Revamped Scoped Custom Element Registries

[annevk]

https://github.com/WICG/webcomponents/blob/gh-pages/proposals/Scoped-Custom-Element-Registries.md is a good proposal, but it ties the functionality too much to shadow roots. This is Ryosuke and I's proposed improvement attempting to account for feedback given in various Web Components issues on this topic: https://github.com/WICG/webcomponents/issues?q=is%3Aissue+label%3A%22scoped+custom+element+registry%22.

First, the IDL, illustrating the new members:

interface CustomElementRegistry {
  constructor();

  ...
  undefined initialize((Element or ShadowRoot) root);
};

partial interface Document {
  [CEReactions, NewObject] Element createElement(DOMString localName, optional (DOMString or ElementCreationOptions) options = {});
  [CEReactions, NewObject] Element createElementNS(DOMString? namespace, DOMString qualifiedName, optional (DOMString or ElementCreationOptions) options = {});
  [CEReactions, NewObject] Node importNode(Node node, optional (boolean or ImportNodeOptions) options = false);
}

dictionary ElementCreationOptions {
  CustomElementRegistry customElementRegistry;
}

dictionary ImportNodeOptions {
  CustomElementRegistry customElementRegistry;
  selfOnly = false;
}

partial interface Element {
  readonly attribute CustomElementRegistry? customElementRegistry;
};

dictionary ShadowRootInit { // used by Element.prototype.attachShadow
  ...
  CustomElementRegistry customElementRegistry;
};

partial interface DocumentOrShadowRoot {
  readonly attribute CustomElementRegistry? customElementRegistry;
};

partial interface HTMLTemplateElement {
  [CEReactions] attribute DOMString shadowRootCustomElementRegistry;
}

Here’s a summary of how the proposal evolved:

• CustomElementRegistry still gains a constructor.
• ShadowRoot still supports a CustomElementRegistry, exposed through a customElementRegistry getter.
  • It seems important for the shadow root to be able to be independent from its host in terms of registries.
  • Interaction with declarative shadow DOM WICG/webcomponents#914 has a proposal for declarative shadow trees to be able to disable inheriting from the global registry. This adopts that with the shadowrootcustomelements attribute, which is reflected as a string for forward compatibility.
  • ElementInternals gains initializeShadowRoot() CustomElementRegistry gains initialize() so a declarative shadow root (or any element) can have its CustomElementRegistry set (when it’s null).
  • The attachShadow() member is now called customElementRegistry for consistency.
• Element should support an associated CustomElementRegistry, exposed through a customElementRegistry getter. This impacts elements created through innerHTML and future such methods, such as setHTMLUnsafe(). This will allow using non-global CustomElementRegistry outside of shadow roots.
  • setHTMLUnsafe() in the future could maybe also set its own CustomElementRegistry. Given the ergonomics of that it makes sense to expose it directly on Element as well.
• document.createElement(), document.createElementNS(), and document.importNode() are updated to account for registries.

I’ll create specification PRs as well to allow for review of the processing model changes. We believe this resolves the remaining issues with the latest iteration of the initial proposal.

I'd like to briefly go over this in the December 19 WHATNOT meeting and will also be available then to answer any questions. Marking agenda+ therefore.

cc @rniwa @justinfagnani @whatwg/components

---

Minor issue tracker:

• Instead of inventing a new createElement() we should attempt to reuse the existing one on document (with a new customElements member) as we're not quite ready to reinvent how to best do element-creation.
• Should initializeSubtree() perform upgrades in the connected case? (If yes, it should probably also update the scoped document set.) (No.)
• Is there a better name for initializeSubtree()? (Also consider the names already exposed on this object.) Just initialize.
• importNode() is better after all as we need a document anyway. So we should go with a document.importNode(node, { subtree, customElements }) overload.

[thepassle]

So to createElement from a scoped registry, inside a custom element with a shadowroot, you would now have to do:

this.shadowRoot.customElements.createElement('my-el');

Is that correct?

[EisenbergEffect]

Will the global customElements also get a cloneSubtree() method and will this also support built-ins? Essentially, I'm looking for a consistent way to clone templates/fragments with different scopes. So, I'd like to be able to do something like this:

function cloneInScope(src: DocumentFragment, scope: Document | ShadowRoot | Element = document) {
  const registry = scope.customElements ?? globalThis.customElements;
  return registry.cloneSubtree(src);
}

Assuming the above, then at first glance, this proposal looks like it will enable all my scenarios.

[rniwa]

@EisenbergEffect : Yes, cloneSubtree method would be exposed on all registries including the global one. And indeed one of the design constraints we had was to allow a consistent way of creating an element regardless of the registry being used.

[rniwa]

So to createElement from a scoped registry, inside a custom element with a shadowroot, you would now have to do:

this.shadowRoot.customElements.createElement('my-el');

Is that correct?

Yes although a more convenient way is to use whatever node you already have in the shadow tree and do:

node.customElements.createElement('my-el');

[thepassle]

Right so from “inside” a custom element you could either do this.customElements or this.shadowRoot.customElements? To give some context, at ING (bank) we make heavy use of the current scoped registries polyfill and @open-wc/scoped-elements so this wil likely be something we need to address in our codebases, since we currently use this.shadowRoot.createElement. I dont think thats a huge problem though.

Additionally, I think the addition of allowing a registry for a node is a good addition 👍

[matthewp]

Why happens in this scenario?

<outer-element>
  <template shadowrootmode="open">
    <inner-element></inner-element>
  </template>
<outer-element>

<inner-element></inner-element>

Assuming that there are different definitions for inner-element in the light DOM vs the outer-element's template. Let's assume that inner-element gets defined before outer-element.

[rniwa]

@matthewp : in that scenario, all the elements will use the global registry since there is nothing on template or any other element to indicate it should use a scoped registry.

[matthewp]

@rniwa Thanks, that was my suspicion. That seems like a show-stopper to me. Can we add something to template or somewhere else to prevent this problem?

[rniwa]

@rniwa Thanks, that was my suspicion. That seems like a show-stopper to me. Can we add something to template or somewhere else to prevent this problem?

In the proposal @annevk made above, there is shadowrootcustomelements content attribute you can add on template to indicate that a declarative shadow DOM will use a scoped custom element registry.

[sorvell]

Thanks very much for working on this @rniwa and @annevk! I think this proposal is an improvement over the original. I have a few questions and refinement suggestions.

initializing a registry

It seems problematic to expose this only via ElementInternals since this tightly couples the ability to control a shadowRoot's registry to it being used on a custom element. In other words, how would this work, assuming the developer wants to associate #host with a specific registry?

<div id="host">
  <template shadowrootmode="open" shadowrootcustomelements="">
    <x-foo></x-foo>
  </template>
</div>

Putting an API to initialize a registry on shadowRoot seems like an obvious alternative, but this would be inconvenient for closed shadowRoots. However, (apparently) you can call attachShadow on an element with a declarative shadow root. This may be ok for now until there is a general solution for getting a closed shadowRoot for a non-custom element (like allowing it to call attachInternals?).

I also think it would be great to be able to create an imperative shadowRoot with a blank customElements for symmetry and expressiveness. If that's the case, perhaps it could be ok to set customElements on anything that has it only if its current value is null?

Might this be workable?

const shadowRoot = element.attachShadow({mode: 'closed', customElements: null}); 
shadowRoot.innerHTML = `
  <x-foo> <x-bar></x-bar>...</x-foo> 
  <x-foo> <x-bar></x-bar>... </x-foo>
`;
const xFoo1 = shadowRoot.firstElementChild;
const xFoo2 = shadowRoot.lastElementChild;
xFoo1.customElements = registryA; // upgrades XFoo1 and its subtree in registryA?
shadowRoot.customElements = registryB; // upgrades XFoo2 in registryB?
//
xFoo1.customElements = registryB // throws.

cloning

1. What does cloneNode do if an element has a customElements set on it? Does it upgrade it in that registry?

console.assert(element.customElements == registryA); // ok
const clone = element.cloneNode(true);
console.assert(clone.constructor == registryA.get(clone.localName)) // ok?

2. How is cloneSubtree different from importNode and if the difference is trivial, perhaps that's a better (slightly more familar) name?

[justinfagnani]

Thanks for making this revision @annevk and @rniwa. I'm very glad that it seems like we can just have elements remember their registry and not have to always defer to shadow roots!

A few questions / concerns:

Element creation

I think that in order to get frameworks and rendering libraries to support for scoped registries we have to make it extremely easy and performance-neutral for them to add.

The way I had proposed this was to add createElement(), importNode(), etc., on ShadowRoot not just as a way to create scoped elements, a way that shared an API subset with Document so that a library could choose or be passed an object to create elements with that's likely compatible with their current callsites.

Because ShadowRoot's optionally had an associated CustomElementsRegistry and fell back to global creation when they didn't have one, a library could always use the shadow root as the creation object, and fall back to the document when not rendering into a shadow root. This simplifies element creation a lot - there's little code or perf overhead to supporting scoped registries.

For example, In lit-html, we pass either document or a ShadowRoot as an option to render(), and so any templates cloned for that render will use the correct registry.

// Use the global scope always. (document is also the default)
render(html`<x-foo></x-foo>`, {creationScope: document});

// In a web component, use registry of the shadow root, which may or may not have a scoped registry
render(html`<x-foo></x-foo>`, {creationScope: this.shadowRoot});

On the library side, support for scopes is as simple as:

const fragment = (options?.creationScope ?? document).importNode(template.content, true);

I worry that an API like cloneSubtree() being separate from importNode() means that element creation code would have to change too much. It's either need an abstraction for creating elements and cloning templates, or type check some object and call either .importNode() or .customElements.cloneSubtree().

I think it'd be an easier lift if instead we made it possible to use a Document or ShadowRoot in more cases as a scope object. They have other useful common APIs like .styleSheets and .adoptedStyleSheets as well.

This wouldn't preclude element creation APIs from also existing on CustomElementsRegistry.

Non-shadow DOM usage and SSR

I think that like shadowRootCustomElements option on templates, we also need a way to disable upgrades for light DOM subtrees. Consider a page like:

<body>
  <x-feature-1>
    <x-foo></x-foo>
  </x-feature-1>
  <x-feature-2>
    <x-foo></x-foo>
  </x-feature-2>
</body>

Where <x-feature-1> and <x-feature-2> have independently versioned dependencies and render to light DOM, not shadow DOM. <x-foo> may be have different versions within the features. We'd like to defer upgrades of those subtrees until each feature element can setup the appropriate custom element registry. This also means that we'd need Element. customElements to be settable once like with ShadowRoots.

[rniwa]

The way we are envisioning this API will be used is that we'd use CustomElementRegistry as scoping object instead of ShadowRoot. It's more natural that way since you'd often construct a tree without necessarily having access to the future root node. You can easily fallback to document like this: (customElements || document).createElement('~').
If you wanted to use the shadow root as a scoping object, you still can. You just need to do: (shadowRoot.customElements || document).createElement(~) or (root.customElements.?createElement || root.createElement)('~').

It's possible to add convenience functions on ShadowRoot as well but it did seem like something we can wait for the community feedback.

[rniwa]

It's possible to extend this API to support an element with null registry in a document tree as a thing by introducing a new content attribute for the parser to consume but I don't think we should include that in the initial version unless we can find very important use cases that require that.

[rniwa]

It seems problematic to expose this only via ElementInternals since this tightly couples the ability to control a shadowRoot's registry to it being used on a custom element. In other words, how would this work, assuming the developer wants to associate #host with a specific registry?

Is that a common scenario? I can't think of a situation in which author uses a declarative DOM in conjunction with a scoped custom element without using a custom element on the shadow host. What are concrete use cases?

[rniwa]

I also think it would be great to be able to create an imperative shadowRoot with a blank customElements for symmetry and expressiveness. If that's the case, perhaps it could be ok to set customElements on anything that has it only if its current value is null?

We had considered that option but concluded that a setter which allows setting once then starts throwing is an exotic behavior we want to avoid. We also had hopes to make it so that elements are never exposed to scripts until its registry is initialized. However, now we realize this is not possible since end user could interact with such an element and trigger a composed event before scripts had a chance to define its registry (or else it sort of defeats the whole point of SSR). So given that, we can revisit this alternative design.

[rniwa]

How is cloneSubtree different from importNode and if the difference is trivial, perhaps that's a better (slightly more familar) name?

The primary way cloneSubtree differs from importNode is that it does deep cloning by default as Mozilla had advocated in the past (since that's what you want in most cases anyway) when we were standardizing cloneNode's default argument to be optional. We thought using the same method name would be confusing given that distinction.

[sorvell]

It seems problematic to expose this only via ElementInternals since this tightly couples the ability to control a shadowRoot's registry to it being used on a custom element. In other words, how would this work, assuming the developer wants to associate #host with a specific registry?

Is that a common scenario? I can't think of a situation in which author uses a declarative DOM in conjunction with a scoped custom element without using a custom element on the shadow host. What are concrete use cases?

We've seen a lot of interest in using scoped registries for micro-frontends (MFE) where a subtree might be managed in a framework that either doesn't use custom elements and/or doesn't want Shadow DOM. The general problem with these MFE use cases is that tend to be very over-constrained so the platform must be expressive and flexible to handle them. We can definitely try to get more feedback on these issues. Consider this scenario...

  <div id="svelte-app">
    <template shadowrootmode="closed" shadowrootcustomelements>
       My svelte MFE
      <design-system-button>version 1.2.3 so must be that registry</design-system-buttton>
    </template>
  </div>

  <div id="vue-app">
    <template shadowrootmode="closed" shadowrootcustomelements>
       My vue MFE
      <design-system-button>version 1.1.7 so must be that registry</design-system-buttton>
    </template>
  </div>

  <div id="react-app" customelements="">
    My react MFE (needs global styling!)
    <design-system-button>version 1.3.8 so must be that registry</design-system-buttton>
  </div>

[justinfagnani]

@rniwa

The way we are envisioning this API will be used is that we'd use CustomElementRegistry as scoping object instead of ShadowRoot.

One reason I (mildy) prefer at least the option of using a ShadowRoot as the scoping object is that it has other scope-related APIs, like .adoptedStyleSheets, and .getElementById(). This makes the union type of Document | ShadowRoot useful as a scope object for several purposes.

Of course, as you point out, both Document and ShadowRoot would have .customElements, but it's also somewhat weird to me that you would need to use the CustomElements interface to create elements, even built-in ones, to get scoping correct. .createElement() on Document or ShadowRoot (or maybe Element?) feels more generic.

I know similar arguments were made about the getName() API. As of now, there would be an asymmetry with .get() where customElements.createElement('div') works, but customElements.get('div') returns undefined.

Another issue for me is cloneSubtree() vs importNode(). This would seem to require more conditional code, like:

(root.customElements?.cloneSubtree?.(template.content) ?? document.importNode(template.content, true)

vs

(root.importNode ?? document.importNode)(template.content, true)

It might not seem like much, but we've seen pushback over similar things.

---

I also have a question about the value .customElements - when is it defined?

• If you call attachShadow() without customElements, does shadowRoot.customElements point to the global registry, or is it undefined?
• If it's undefined, is there a way to tell the difference between a root that uses the global registry, and a DSD that's await it's registry to be initialized?
• Similar questions for Elements, and those created within DSD with shadowrootcustomelements

One nice thing about .createElement() on ShadowRoot is that it could throw if shadowrootcustomelements was set but initializeShadowRoot() was not yet called.

[rniwa]

Is that a common scenario? I can't think of a situation in which author uses a declarative DOM in conjunction with a scoped custom element without using a custom element on the shadow host. What are concrete use cases?

We've seen a lot of interest in using scoped registries for micro-frontends (MFE) where a subtree might be managed in a framework that either doesn't use custom elements and/or doesn't want Shadow DOM.

I can see MFE may not want to use shadow DOM. But the combination of waiting to use shadow DOM and scoped custom element registry but not custom elements for the host seems like odd combination to me. What are examples of frameworks / libraries / websites that do this?

[rniwa]

Of course, as you point out, both Document and ShadowRoot would have .customElements, but it's also somewhat weird to me that you would need to use the CustomElements interface to create elements, even built-in ones, to get scoping correct. .createElement() on Document or ShadowRoot (or maybe Element?) feels more generic.

To us, it seemed weird that ShadowRoot gets a method to create a custom / builtin element with the new design where each element is associated with a scoped custom element regardless of its root node. Why is ShadowRoot special compared to other root nodes in this new world?

I know similar arguments were made about the getName() API. As of now, there would be an asymmetry with .get() where customElements.createElement('div') works, but customElements.get('div') returns undefined.

That might be an argument for making getName and get work with builtin elements.

Another issue for me is cloneSubtree() vs importNode(). This would seem to require more conditional code, like:

(root.customElements?.cloneSubtree?.(template.content) ?? document.importNode(template.content, true)

vs

(root.importNode ?? document.importNode)(template.content, true)

Over time (with any polyfill), the former will simplify to just element.customElements.cloneSubtree(template.content). We're envisioning that the future will be custom element registry centric so that most frameworks and libraries will take registry as an argument / configuration option to create a DOM tree.

I also have a question about the value .customElements - when is it defined?

• If you call attachShadow() without customElements, does shadowRoot.customElements point to the global registry, or is it undefined?

It points to the global registry. element.customElements will always point to a valid registry except the case of "null registry" (i.e. for elements in DSD awaiting registry initialization), in which case, it should probably return null.

One nice thing about .createElement() on ShadowRoot is that it could throw if shadowrootcustomelements was set but initializeShadowRoot() was not yet called.

That is tautologically true of createElement on CustomElementRegistry as well since there is no createElement method to call until customElements starts to return a valid registry.

[matthewp]

It seems really odd to me to have shadowRoot as a general feature but then hide some APIs behind custom element only. It's hard to answer a use-case question because it seems like it's already answered for why you use shadow DOM outside of custom elements in general. Shadow DOM is a lightweight scoping mechanism. I have used Shadow DOM outside of custom elements to render email HTML, for example. I would like to enhance this capability to run custom elements and I want to be able to version them.

[michaelwarren1106]

Is that a common scenario? I can't think of a situation in which author uses a declarative DOM in conjunction with a scoped custom element without using a custom element on the shadow host. What are concrete use cases?

We've seen a lot of interest in using scoped registries for micro-frontends (MFE) where a subtree might be managed in a framework that either doesn't use custom elements and/or doesn't want Shadow DOM.

I can see MFE may not want to use shadow DOM. But the combination of waiting to use shadow DOM and scoped custom element registry but not custom elements for the host seems like odd combination to me. What are examples of frameworks / libraries / websites that do this?

here’s the MFE use case that is of interest to me. as usual it’s a design system use case. let’s say i have a design system written in web components. so all my buttons, modals, tooltips, etc are custom elements but the app i’m writing is a react app that consumes those elements.

and my app is an MFE remote app that gets loaded async on the same page as another MFE remote app also in react AND the “host” app which is the parent of all the MFE remotes. my app is not the whole page shown to users, but just a part of it. and my app is rendered by react (a shared dependency from
the host app) into some root div.

my app contains v1.0.0 of the design system button, x-button. and the host app has v0.0.2 of x-button and another MFE remote app has v2.0.0 of x-button all at the same time.

scoping is needed so we go to set it up so that my MFE app version of the design system components can’t conflict with the host app or other MFEs remote apps that might be rendered into the same page as my MFE remote.

under the existing proposal, i’d have to:

• render my whole MFE app in a shadow root
• list each definition that needs scoping and register them in the registry
• somehow tell react that it needs to use the registry for any WCs it’s rendering in my MFE

it would be easier if registries and shadow roots were disconnected because i wouldn’t have render my MFE app in shadow root. if there was a way to programmatically just “apply a registry to some div perpetually” then an MFE setup could just create the registry and the react render root separately with js, then link them together without having to involve react internals at all.

if i could do something like:

const registry = new Registry();
//add els to registry
registry.define(‘x-button’);
const root = React.createRoot(‘div’);

// tell the root that all WCs in it should use the registry first, global as a fallback
root.attachRegistry(registry);

root.render(<MyApp/>);

and not involve react at all that would be amazing for MFEs

[vospascal]

I agree with @michaelwarren1106 like I tried to make clear in discord I would love to have a similar way to how forms work currently if you wrap a form around input elements they register to that unless you set the form attribute on the input to something else. You can also apply this attribute form I think to an input outside the form tag.
This way moving an element doesn’t really matter if you have assigned the form attribute (didn’t test) same goes probably for cloning. Sure it doesn’t cover everything but it’s a well known pattern that could bing us a long way.

<!-- this registereds all to the form / could this method also work for custom elements? -->
<form id="myForm" action="/submit" method="post">
  <button type="submit">Submit</button>
  <input type="password" name="password" placeholder="password">
</form>

<input type="text" name="username" form="myForm" placeholder="Username">
<input type="text" name="email" form="myForm" placeholder="Email">

[tbroyer]

Not a big issue (more like a detail), but it looks like with the proposal in the current form, it's not possible to have an associated CustomElementRegistry to a Document returned from DOMImplementation.createHTMLDocument.

IMO, it should either inherit the custom element registry of the DOMImplementation's associated document, or createHTMLDocument should optionally take a CustomElementRegistry as argument.

[annevk]

You can use initialize(). Depending on how common that is going to be we might offer it as a constructor argument to new Document() at some point.

[tbroyer]

Oh, thanks, overlooked that!
(doesn't look like there's a corresponding WPT though, unless I just failed to find it)

[annevk]

@tbroyer good point. web-platform-tests/wpt#51976

[trusktr]

Is there a polyfill for the new Scoped Custom Element Registries yet? And if I use it, will I be shot?

It is currently a bit difficult to search for information on Scoped Registries. The top result on Google,

has not been updated in 5 years:

@justinfagnani when you have a moment, or if someone else does, can we please link that to the latest spec?

I don't know if it is just me, but anything that goes from its own spec space and into the html or dom specs seems to get burried and undiscoverable on search engines. This is a problem.

[rniwa]

FWIW, the latest scoped custom element registry API has been implemented and enabled in Safari Technology Preview.