Adding WASM lib to NX monorepo

Updated: at 09:12 AM

Adding WASM lib to NX monorepo

Based on my old notes from 2020-08-19, might be outdated.

Why?

I wanted to add run the rust wasm game-of-life example as a lib in nx monorepo. Here’s a hacky way to do it.

Full rust+wasm game of life example with GitHub actions / GitHub pages integrations can be found here. I’ve used this method also in my Advent of Code solutions

Install dependencies

Install all dependencies:

Create workspace with one app and one lib.

[jakub@jakub-pc side]$ npx create-nx-workspace@latest
[jakub@jakub-pc side]$ npx g @nrwl/react:app wasm-app
[jakub@jakub-pc side]$ npx g @nrwl/react:lib wasm-lib

The type of the lib is irrelevant, because most of its contents will be replaced anyway with wasm-pack template.

Create rust lib using wasm-pack

[jakub@jakub-pc example]$ cd libs/wasm-lib/
[jakub@jakub-pc wasm-demo]$ ~/.cargo/bin/wasm-pack new tmp

Navigate to the created libs/wasm-lib directory and generate it. Both the nx lib and the wasm-pack created lib cannot be created in an existing folder, so you need to first generate the nx lib. Then, within that lib, generate the wasm-pack lib and copy the files from the generated wasm-pack lib to the parent folder (overwriting if necessary). You can also remove the js/ts related config files: jest.config.js, .eslintrc, .babelrc, tsconfig*, babel-jest-config.json.

After generating nx lib

After generating nx lib

After generating wasm-pack lib

After generating wasm-pack lib

After removing files and copying tmp to root

After removing files and copying tmp to root

The rust code generated by by wasm-pack will look like this - a simple function that calls alert with some text.

// wasm-lib/src/lib.rs
mod utils;

use wasm_bindgen::prelude::*;

// When the `wee_alloc` feature is enabled, use `wee_alloc` as the global
// allocator.
#[cfg(feature = "wee_alloc")]
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;

#[wasm_bindgen]
extern {
    fn alert(s: &str);
}

#[wasm_bindgen]
pub fn greet() {
    alert("Hello, wasm-demo!");
}

To generate js/wasm code, build the lib. To change the name of out files to start with index use the --out-name build flag.

[jakub@jakub-pc wasm-demo]$ ~/.cargo/bin/wasm-pack build --out-name index
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
   Compiling proc-macro2 v1.0.21
   Compiling unicode-xid v0.2.1
   Compiling syn v1.0.41
   Compiling wasm-bindgen-shared v0.2.68
   Compiling log v0.4.11
   Compiling cfg-if v0.1.10
   Compiling bumpalo v3.4.0
   Compiling lazy_static v1.4.0
   Compiling wasm-bindgen v0.2.68
   Compiling quote v1.0.7
   Compiling wasm-bindgen-backend v0.2.68
   Compiling wasm-bindgen-macro-support v0.2.68
   Compiling wasm-bindgen-macro v0.2.68
   Compiling console_error_panic_hook v0.1.6
   Compiling wasm-demo v0.1.0 (/home/jakub/dev/side/example/libs/wasm-demo)
warning: function is never used: `set_panic_hook`
 --> src/utils.rs:1:8
  |
1 | pub fn set_panic_hook() {
  |        ^^^^^^^^^^^^^^
  |
  = note: `#[warn(dead_code)]` on by default

warning: 1 warning emitted

    Finished release [optimized] target(s) in 32.57s
[INFO]: Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended
[INFO]: :-) Done in 32.64s
[INFO]: :-) Your wasm pkg is ready to publish at /home/jakub/dev/side/example/libs/wasm-demo/pkg.

Build result

We can import the greet function in the App component like that:

import React from 'react';

import './app.scss';

import { ReactComponent as Logo } from './logo.svg';
import { greet } from '../../../../libs/wasm-demo/pkg';

export const App = () => {
  greet();
  return (...);
};

export default App;

Adjusting tsconfig

Edit tsconfig.json file in root directory, so that @wasm-example/wasm-lib resolves to the build output directory.

{
  "compileOnSave": false,
  "compilerOptions": {
    "rootDir": ".",
    "sourceMap": true,
    "declaration": false,
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "target": "es2015",
    "module": "esnext",
    "typeRoots": ["node_modules/@types"],
    "lib": ["es2017", "dom"],
    "skipLibCheck": true,
    "skipDefaultLibCheck": true,
    "baseUrl": ".",
    "paths": {
      "@wasm-example/wasm-lib": ["libs/wasm-lib/pkg"]
    }
  },
  "exclude": ["node_modules", "tmp"]
}

This will make the imports a little bit cleaner

import { greet } from "../../../../libs/wasm-lib/pkg/index_bg.wasm"; // before
import { greet } from "@wasm-example/wasm-lib"; // after

Importing and running

Importing the function normall will result in the following error after running npm run start

The webpack tells explicitly that we have to change the import to async:

This is not allowed, because WebAssembly download and compilation must happen asynchronous. Add an async splitpoint (i. e. import()) somewhere between your entrypoint and the WebAssembly module:

Change the greet import to async. After running npm run start you should see the alert generated by the webassembly code.

// wasm-app/src/app/app.tsx fragment
export const App = () => {
	import('@wasm-example/wasm-lib').then(wasm => wasm.greet());

	return (...);
}

Build result

However, this type of import is somewhat annoying, as the result needs to be promisified. Some approaches I’ve found suggest using a useEffect hook to load the .wasm file and then use setState with the loaded .wasm. I’m not a fan of this method because it requires managing a flag to check whether the .wasm has loaded. Luckily, the async split point can occur at any time before the greet call, like a whole component, and once loaded it can be used like normal synchronous code. My preferred approach is to load .wasm at the top of the route, using React.lazy and React.Suspense.

Create a component for the route. Within this component or its child components, you can call .wasm functions synchronously. The export default statement is necessary for working with the async import().

// wasm-app/src/app/wasm-route.tsx
import React from "react";
import { greet } from "@wasm-example/wasm-lib";

const WasmRoute: React.FC = () => {
  greet();

  return <div>Wasm Route!</div>;
};

export default WasmRoute;

Make that component load async using React.lazy and React.Suspense

import React, { lazy, Suspense } from 'react';
import './app.scss';
import { ReactComponent as Logo } from './logo.svg';
import { Route, Link } from 'react-router-dom';

const wasmRoute = lazy(() => import('./wasm-route')); // lazy component with wasm

export const App = () => {
  return (
    <div className="app">
      <header className="flex">
        <Logo width="75" height="75" />
        <h1>Welcome to wasm-app!</h1>
      </header>
      <Suspense fallback={'loading...'}>
        <Route
          path="/"
          exact
          render={() => (
            <div>
              This is the generated root route.{' '}
              <Link to="/page-2">Click here for page 2.</Link>
            </div>
          )}
        />
				{// and place the lazy component on desired route}
        <Route
          path="/page-2"
          exact
          component={wasmRoute}
        />
      </Suspense>
      <div role="navigation">
        <ul>
          <li>
            <Link to="/">Home</Link>
          </li>
          <li>
            <Link to="/page-2">Page 2</Link>
          </li>
        </ul>
      </div>
    </div>
  );
};

export default App;

Testing WASM components with enzyme.

To test components that use wasm code synchronously, wrap them with Suspense and use shallow instead of mount.

Troubleshooting

wasm-pack command does not do anything

On one of my laptops after installing wasm-pack seemed to be broken. Running the wasm-pack binary directly: ~/home/jakub/.cargo/wasm-pack seemed to fix the issue.

“Naming constants with ’_’ is unstable” when running wasm-pack build

[jakub@jakub-pc wasm-demo]$ ~/.cargo/bin/wasm-pack build --out-name index
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
   Compiling proc-macro2 v1.0.21
   Compiling unicode-xid v0.2.1
   Compiling wasm-bindgen-shared v0.2.68
   Compiling syn v1.0.41
   Compiling log v0.4.11
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling bumpalo v3.4.0
   Compiling wasm-bindgen v0.2.68
error[E0658]: naming constants with `_` is unstable (see issue #54912)
   --> /home/jakub/.cargo/registry/src/github.com-1ecc6299db9ec823/bumpalo-3.4.0/src/lib.rs:298:1
    |
298 | const _: [(); _FOOTER_ALIGN_ASSERTION as usize] = [()];
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.
error: Could not compile `bumpalo`.
warning: build failed, waiting for other jobs to finish...
error: build failed
Error: Compiling your crate to WebAssembly failed
Caused by: failed to execute `cargo build`: exited with exit code: 101
  full command: "cargo" "build" "--lib" "--release" "--target" "wasm32-unknown-unknown"

For me RC of this was very old version of rustup (1.31) I think this will fork with 1.37^ as per: https://users.rust-lang.org/t/how-to-resolve-e0658-underscore-naming-in-rusts-cargo-check/40098.

Run rustup update to download latest toolchain versions. After updating, check whether the default toolchain is set to the updated one, if not make it so.