Namespace unwrapping support in JS API as a follow-up to direct global exports

[guybedford]

This is an addition to #104 (and based to that PR) that was brought up when this was discussed at the 25.03 Wasm CG meeting.

Specifically, Andreas Rossberg mentioned that having global instance bindings inaccessible for further use in the JS API could be overly restrictive.

To resolve that issue, this PR applies similar logic to what we do in the ESM binding logic but at the JS API level instead, keeping the APIs more in line. When a module namespace is passed on the importObj in the WebAssembly.instantiate API, we internally unwrap any WebAssembly global from the namespace when one is found.

The added checks somewhat repeat what we have for bindings on the WebAssembly module record already, so could be split out into a shared function if going this approach. The overall distinction between the ESM and JS API in operating on module records versus namespaces still seems to warrant the overall logic separation to me though.

//cc @rossberg, review very welcome as well, and if we have support for this change I can then look to merge it in with #104 and refactor some duplication.

[rossberg]

Ah, this isn't quite what I was asking about. I may have misunderstood what's going on with the implicit unwrapping, but the scenario I wondered about was something like:

;; a.wasm
(module
  (global (export "i") (mut i32) ...)
  (global (export "g") (mut i32) ...)
  ...
)

;; b.wasm
(module
  (global (import "m" "g") (mut i32))
  ... ;; other imports
)

// c.js
import * as a from "a"
...
++a.i;
...
WebAssembly.instantiate(b_wasm, {m: {g: a.g, ... /* other imports unrelated to a */}})

Or, which I would assume ought to be equivalent:

// c.js
import {i, g, ...} from "a"
...
++i;
...
WebAssembly.instantiate(b_wasm, {m: {g, ... /* other imports unrelated to a */}})

How can both the use of a.i and a.g work at the same time? My understanding (possibly incorrect) is that the latter would no longer work with #104, and that would be a problem from my perspective, especially if there isn't even an explicit way to get back the actual global.

[guybedford]

Thanks for clarifying the scenario you had in mind here.

If the "m" import expected by the b_wasm only expected a g export name to be present, then it could still be fulfilled by the module A, even though it exports both i and g, per the approach in this PR via something like:

import source b_wasm from './b.wasm';
import * as m from './a.wasm';
WebAssembly.instantiate(b_wasm, { m, ...otherImports });

Or is the problem you are referring to the case where the "m" expected by the b_wasm needs say a g and a p instead of an i?

In general module interfaces should be consistent and backwards compatible, I think that's the overall convention that's in place. Are there useful scenarios in which this kind of per-module interface restructuring is necessary, or is this just considering edge cases?

Binding remixing is done at the syntax level in ESM, so would require a separate module to achieve this kind of mixing. It would also be possible to do this inline with module expressions in JavaScript in future though, to support such a g and p mashup:

import source b_wasm from './b.wasm';
const m = await import(module {
  export { g } from './a.wasm';
  
  // this WebAssembly.Global gets treated just like in the JS API already today
  // #104 only changes exports handling not imports handling
  export const p = new WebAssembly.Global(...);
});
WebAssembly.instantiate(b_wasm, { m, ...otherImports });

Therefore, I still think the change in this PR answers your use case and exception, with the caveat that composition outside the module boundary level is usually preferable to avoid the above (although it is possible as shown).

[rossberg]

AFAICS, the problem with your solution is that it only is applicable when we can pass m to instantiate as an entire module/namespace. But that isn't always the case. The grouping of imports into modules is ultimately just a convention on the Wasm level. Sometimes the individual imports need to be supplied individually, and in that case, we really need a.g individually, as a global instance, instead of the whole namespace it resides in.

Concretely, assume that the "other imports" of b in my example include more from the same virtual namespace "m", but which are not supplied by a. That is, you'd really need to construct the imports object with a nested record, as shown in my example.

Is that still expressible at all under #104? If not, I think that clearly creates an expressivity hole.

As a general principle, it should always be possible to equivalently expand out the full imports object in place. And I think that implies that its semantics shouldn't depend on any magic for specific types of import modules.

(FWIW, I don't understand ESM well enough to tell, but assuming it also allows imports in the other direction, i.e., Wasm directly importing from a JS module, then I should be able to construct a similar counter example even without using instantiate.)

[guybedford]

The last code example in my last comment demonstrates exactly what you are asking, constructing a new m with both a {p, g} namespace, out of an m that was formerly a {p, i} namespace. Does this not resolve your concern? And if so why not?

[rossberg]

I guess I didn't really understand what that example tries to do. As far as I can see, it creates a new global, so new state. But b is supposed to get access to a's global, not a different one. Maybe I'm missing something obvious, but how does the example achieve that?

[guybedford]

In the example, B is getting A's original g global, exactly shared with all other importers of A in the module registry. The extra global for p is to demonstrate "mixing in" a new global which was not previously present on the exports of A. The example could equally well use a memory or table here. g remains the "shared" global though.

[guybedford]

(updated lettering, sorry typo on first go)

[rossberg]

Not sure I follow. Let me check my understanding. Are you saying that g remains wrapped when I write

export {g} from "x.wasm"

but is unwrapped when I write

import {g} from "x.wasm"

? If so, wouldn't that completely go against the spirit of export-from, which really just is a short-hand for reexporting an import?

[guybedford]

No, it is that rather by synthesizing a new module namespace via const ns = await import(module { export { g } from './x.wasm' }) that the spec change here could allow that new namespace to unwrap the module record back to the original Wasm global address.

[guybedford]

Here is another simpler example without the inline module if it helps:

remix.js

export { g } from './a.wasm';
export var extra = new WebAssembly.Global(...);

app.js

import source b_wasm from './b.wasm';
import * as remix_ns from './remix.js';
WebAssembly.instantiate(b_wasm, { m: remix_ns });

And the counter argument to your last question is that this works equally well via:

remix-with-import.js

import { g as foo } from './a.wasm';
// (uses foo here)
export { foo as g };

Because bindings are individually traced at link time.

To break the binding chain you would need to redeclare the variable by assignment, which by definition then creates a new binding:

remix-new-binding.js

import { g as foo } from './a.wasm';
export const g = foo; // assignment to a new binding slot

[rossberg]

Ah, I see, thanks for the clarification. So it all comes down to module namespaces being treated very specially (if not magically) when used as members of an import object. And I have to funnel things through auxiliary modules to trigger the desired/expected behaviour.

I think my point stands that this is problematic. It is highly unintuitive, plus the change is making the semantics of ESM leak into instantiate and thereby pierce through a separate abstraction, just to work around a problem that it created itself.

As far as I am concerned, the design principle for import objects is that they behave as simple records, such that

instantiate(x, {a, b, c})

is always expandable to

instantiate(x, {a: {...a}, b: {...b}, c: {...c}})

without changing meaning. Only that guarantees that the import object is properly composable, and can be treated as convenient syntax for the flat list of imports that it actually represents.

[guybedford]

Yes exactly, so this question is entirely about design. The ESM Integration design goal is to create an ES Module embedding for WebAssembly modules. Because of the frictions in mashing together conventions, we came up with the source phase as a way to avoid the major frictions here in allowing the JS instantiation API with the ESM Integration. This design is only around the remaining instance integration which is exactly still a process of squeezing Wasm semantics into an ESM shaped hole. And yes that is not always pretty, but we still have the source phase opt-out. My own personal opinion though is that we should aim for creating the closest matching of semantics in the instance phase ESM integration, so that JS developer expectations and ergonomics are maintained allowing Wasm to be used transparently in interop.

In JavaScript:

• Modules have binding slots, and when linking modules, imports are resolved possibly via any number of transitive links through reexports to their original binding slot. The ResolvedBinding record in ECMA-262 is a { [[Module]], [[BindingName]] } pair which can be entirely removed from what the original module and name were.
• There is no way in JavaScript to get a handle on a binding slot itself - importers only ever get a "view" into the value at the mutable slot and can't mutate it.
• There have been discussions around instantiation APIs for JavaScript modules like new Compartment(hooks).import('module'), where the hook for getting a hold of another module here is a load (url) -> Namespace | ModuleSource | ModuleInstance hook which returns another module, possibly a namespace or source object or maybe even instance representation in future.

Module load hooks only work at module boundaries since they operate at the level of providing a single module for each single import string. Treating imports as flat pairs of strings is simply not the design of JS modules. Also in JavaScript, if you want the exact binding slot you need to provide a handle to the module record via the namespace or otherwise risk capturing non-live bindings. First-class binding slot handles simply don't exist in JS.

The problems you are describing are therefore just the design of JS modules. I don't think JS developers would look at the above and call this situation problematic or unintuitive, so the question is how do we approach the design problem of integrating these ecosystems in a way that finds the right middle ground.

My argument is that first-class global bindings are critical to ergonomics for JS developers if we want them to use WebAssembly modules.

In the meeting you raised the concern that not having binding slot handles to globals from the instance integration would be a loss of expressibility, to which I opened this PR, in order to resolve your concern in making the expressibility possible. I agree the solution isn't elegant, but if it is a solution to a real problem, then perhaps that is needed.

In terms of the cost of wrapping, I think a closer ESM Integration would actually reduce the wrapping. For example, this Wasm reexports PR in #105 can reduce duplicate bindings between Wasm and JS, and also unnecessary global wrapping and unwrapping on the interfaces between Wasm modules, allowing Wasm modules to more easily share Wasm exports from a root module, without extra wrapping. So in general I think that closer ties also lead to more ergonomic and optimization benefits by working with and not fighting the JS module system, even if there may be some questionable design decisions there we have to tolerate!

[rossberg]

Yes, I’m on board with that, it is a reasonable design for integrating with JS modules. Where I think ESM integration is taking it too far is when it also starts redefining the interpretation of canonical parts of the Wasm API proper — such as reinterpreting import objects as JS-style modules, which they are not. It is an API for accessing Wasm, and ought to primarily reflect the semantics of Wasm modules, not that of JS modules.

Anyway, even if you consider that okay, it still won’t really solve the problem I was pointing out. Because the code you showed is a workaround for limited cases only.

Correct me if I'm wrong, but it still provides no way to get an actual handle on the imported global instance object in JS. And hence, it still cannot express any kind of logic that needs such a handle. For example:

• What if somebody wants to use JS type reflection on the global?
• What if somebody needs to select a global instance to configure some instantiation programmatically, like cond ? g1 : g2 or something along these lines? [*]
• What about future forms of Wasm globals that cannot be mapped to JS variables, e.g., shared globals?

None of this appears to be workable when the global instance is defined away. I would view that as a serious hole, and indicative of a design that is not future-proof.

So, even if we are okay with the changes in question, there nevertheless needs to be a way to get a hold of the actual global. And once there is, wouldn't this PR become unnecessary?

[*] I suspect this case could be encoded awkwardly using module expressions, but the result hardly is the kind of code pattern we should force people to discover and write.

[guybedford]

None of this appears to be workable when the global instance is defined away. I would view that as a serious hole, and indicative of a design that is not future-proof.

It's very important not to forget that this entire discussion on predicated on the edge case of wanting to provide instance phase WebAssembly module record global exports in the JS API. The major use case remains the source phase imports and JS API case where explicit global handles continue to work with the direct global PR. The JS API still has first-class global handles, and that isn't changing here.

To try and clearly describe the sort of scenario we are looking at:

• Some library uses instance phase Wasm in its implementation.
• If they wanted an instance phase global handle, they could just switch to the source phase and JS API instantiation.
• So let's say the problem here is someone else wants to get access to the instance phase global handle, and they don't want to patch the other library code.

Given this kind of a scenario, to answer your questions:

What if somebody wants to use JS type reflection on the global?

Are there already specs that specify JS type reflection for WebAssembly.Global? I think the ideal here would be to source phase import the module, which would give them the already-compiled and cached WebAssembly.Module record, which they could then do introspection on. i.e. I would encourage extending type info onto WebAssembly.Module.exports output rather than dynamically on WebAssembly.Global.

If there are already APIs for type info on WebAssembly.Global then yes that's trickier.

For type imports, being able to import a type as an explicit export name seems like it would fit this model, but let me know if there are any conflicts here I'm missing too.

What if somebody needs to select a global instance to configure some instantiation programmatically, like cond ? g1 : g2 or something along these lines? [*]

Yes, this would be await import(module { export * from './mod.wasm'; export const cond ? g1 : g2; }).

What about future forms of Wasm globals that cannot be mapped to JS variables, e.g., shared globals?

This is a good question. The constraint here is that we do want Wasm to Wasm instance linkage to still support linking these types of values, even though they don't have a JS representation. The best solution that I can come up with for this (and what I specified in the other PR) was to leave these bindings in their TDZ so that accessing them is a reference error. It's not ideal, but it allows the Wasm to Wasm linking, while leaving the door open for possible future representations.

TDZ is quite common in JS when cycles are in play, and as such they don't give any major footguns. In Node.js, for example, modules in TDZ log fine to the console without an error, eg [Module: null prototype] { g: <uninitialized> }. In browsers they always display exports like Module p: (...) where you need to click into the ... to account for TDZ.

In addition, in most cases, these would probably be considered internal or private modules in the sense that authors would use them to structure Wasm to Wasm linkage, and library systems would avoid put them on Wasm interfaces intended for JS consumers directly.

So, even if we are okay with the changes in question, there nevertheless needs to be a way to get a hold of the actual global. And once there is, wouldn't this PR become unnecessary?

I still don't see in any of the above an irrefutable argument that we need the global handle yet in JS. But I'm still happy to be convinced otherwise yet.

Not landing direct global bindings just means a different future where core Wasm modules must always be wrapped in JS wrappers. Given the potential major bandwidth benefit for browsers in not needing wrappers (every extra module in the JS module system is another request and round trip), I just want to make sure we're properly having that conversation as opposed to not having it, so thanks for engaging.

[rossberg]

Are there already specs that specify JS type reflection for WebAssembly.Global?

Yes, see https://github.com/WebAssembly/js-types/blob/main/proposals/js-types/Overview.md

This proposal has been around for years and is implemented (behind a flag) in all browsers. There is no reasonable design alternative to it either.

But more importantly, this is just one example of a new method on the Global class. It is not unlikely that more might be added for other purposes — a future-proof design needs to assume that. I don't see how that general problem can be addressed other than by making the global instance accessible by some means. A random set of work-arounds for specific cases is insufficient.

Yes, this would be await import(module { export * from './mod.wasm'; export const cond ? g1 : g2; }).

Unless I'm misunderstanding something, that does not work. I was assuming that g1 and g2 refer to mutable globals, and I want to pick one or the other. AFAICS, your code would be copying their state, not share it. I was expecting something along the lines of

await (cond ? import(module { export {g1 as g} from './mod.wasm'  }) : import(module { export {g2 as g} from './mod.wasm'  }))

but that would explode real quick when you have more than one of these in the same import namespace.

Another issue with this workaround of course is that it introduces completely artificial asynchronicity, which is rather undesirable.

Not landing direct global bindings just means a different future

I understand the desire for this convenience. But still, it ought to be added in a way that is not at odds with other API or Wasm's structure. Global instances are a thing and exist for a reason.

That said, it's not necessarily mutually exclusive. The programmer just needs to be given a choice. Not quite sure how that could materialise here. As a random shot, there could be API for accessing a wasm module namespace object and extract the proper global instances from there. That at least gives the programmer some way to get a hold of globals, even if it is not the obvious or intuitive or default way.

[guybedford]

Thanks for sharing re JS types, I wasn't aware that was implemented yet as well. That helps a lot for context.

Yes you would need to change the import or use dynamic import if desiring a specific mutable global in the namespace trick there. And I very much agree that is hideous.

We may yet get beyond the asynchronicity constraint though in https://github.com/tc39/proposal-import-sync too.

From the perspective of the linking model treating instance reflection as its own separate phase is probably the correct way to view this theoretically. That is, the issues we are discussing are a consequence of effectively forcing runtime reflection into value slots because we don't have another layer to manage instance reflection at.

As a random shot, there could be API for accessing a wasm module namespace object and extract the proper global instances from there.

There was also talk of having an instance phase explicitly specified, and that JS might benefit from that too. While this still remains part of the overall design space for JS modules, it's probably not the time to discuss adding another phase though.

In lieu of adding another phase, supporting a WebAssembly.Instance.namespaceInstance(ns) reflection API might make sense. I've posted #108 for further discussion on this. Would be interested to know if that would resolve your concern. We can continue discussing here or there, hopefully keeping threads organized on the design space, where this one is a good place to continue the more general discussion.

[rossberg]

FWIW, a very simple but probably sufficient alternative would be to only unwrap immutable global exports, i.e., value exports. That would be symmetric to the fact that only immutable global imports can be supplied with plain values when instantiating. I would assume that covers the vast majority of cases where the convenience matters.

[guybedford]

@rossberg I sympathize with the sentiment, but unfortunately the example that I gave in the presentation would require mutable global exports. Ideally initializers would be possible to use, but in most cases in JavaScript, initialization logic of exported values does require real code execution. Therefore the sort of interoperability being described requires mutable globals.

;; Wasm module for ESM that exports the namespace:
;; Namespace { "obj": [Object: null prototype] { prop: 'hello world' } }
(module
  (import "./wasm-object.js" "create" (func $create (param externref) (result externref)))
  (import "./wasm-object.js" "setProperty" (func $set (param externref externref externref) (result externref)))
  (import "./wasm-string-constants.js" "prop" (global $prop externref))
  (import "./wasm-string-constants.js" "hello world" (global $hello_world_str externref))

  ;; Define a mutable global for $obj
  (global $obj (mut externref) (ref.null extern))

  ;; Export the "iface" global
  (export "iface" (global $obj))

  (func $start
    ;; $obj = Object.create(null)
    ref.null extern
    call $create
    global.set $obj

    ;; Reflect.set($obj, 'prop', 'hello world')
    global.get $obj
    global.get $prop
    global.get $hello_world_str
    call $set
    drop
  )
  (start $start)
)