rs-jest and rollup-plugin-rust Update: Integrating with webassembly-loader and Some Bug Fixes

in #utopian-io6 years ago (edited)

Repository

https://github.com/DrSensor/rollup-plugin-rustPR#23, PR#26
https://github.com/DrSensor/rs-jestPR#7, PR#8

Since it's been past 1 month, I need to update this 2 packages 😄

Introduction

This 2 npm packages is some of my attempts to make Rust can be integrated in Javascript/Typescript base project. Actually, there is an effort from rustwasm group itself called wasm-pack which is a tool to pack Rust so that it can be published to NPM. Hopefully I can integrate that to this 2 packages when wasm-pack become more stable and have support for external tools just like cargo does.

New Features

Because this 2 packages now depend on webassembly-loader, most of the features in webassembly-loader are now rs-jest and rollup-plugin-rust features. Most of the features can be follow up in the CHANGELOG.md of that loader which are NOT marked as a library.

😞 Problems and Workaround

Starting from why there is a small adjustment in webassembly-loader, some packages that was suggested by official Webpack documentation can't be tree-shaked by Rollup.

small adjustment in webassembly-loader so it can be used in rs-jest and rollup-plugin-rust

Those library (schema-utils & loader-utils) can't be tree-shaked when used in a project that use Rollup which make rs-jest and rollup-plugin-rust bloated. The only solution I can think of is to separate bundle results of webassembly-loader between when used as a webpack loader and used as a library[code] which make webassembly-loader behave differently when the bundler/transpiler do es6 import because those bundler/transpiler usually respect module field rather than main field.

🦁 Implementation (PR#26 & PR#8)

In rollup-plugin-rust, I use webassembly-loader with option esm [code] which generate ECMAScript export (export default ...) because I can't ever imagine rollup-plugin-rust to be used in a project that doesn't respect pkg.module or es6 import. Actually Rollup itself only care es6 module and disregard any module convention, including CommonJS.

While in rs-jest, I use webassembly-loader with option esm [code] which generate CommonJS export (module.exports = ...) because Jest only support CommonJS module, not the others like es6 module. Also, I don't register custom error handler like in rollup-plugin-rust [code] because Jest doesn't have that.

Bug Fixes

🚬 Fixing smoke test (PR#23 & PR#7)

dependabot

Thanks to dependabot (which I set to check the dependencies every month), I notice that the smoke-test behave strangely because I forgot to remove the lockfile in examples repository. Before this fix/refactor, I use git submodule to manage/separate the smoke-test.

Repo#branchUse on
example-stencil-rust#smoke-testrollup-plugin-rust
example-node-rust-wasm#smoke-testrollup-plugin-rust
example-stencil-rust#rs-jestrs-jest

However, on this fix, I remove those git submodule and change it to normal project folder. The reason I use git submodule before is to avoid resetup/prepare the smoke-test on each project that use it. Now I change my mind because I need to track the changes on each project when the smoke-test fail again.

I also took this chance to see if CircleCI jobs smoke-test for stenciljs is fail again [ref]. After this fix seems it's not causing RangeError: Maximum call stack size exceeded again in CircleCI (maybe they fix that) so I bring back the smoke-test for stenciljs from Travis to CircleCI. I just disable the CI in Travis and not remove travis.yml config in case CircleCI limit the nodejs stack-size again.

CI workflow of rollup-plugin-rust

Before fix

After fix

Ah yes, while at it 👇

CI workflow of rs-jest vs rollup-plugin-rust vs webassembly-loader

CI rs-jest
CI rollup-plugin-rust
CI webassembly-loader

which one is better? 😂

🔏 Effect of locking Rust version to 1.28.0 (PR#26 & PR#8)

First of, the reason why I lock the testing to use 1.28.0 not later is because webassembly-loader will try to throw an Error when wasm code is over 4KB when using export mode instance or module at compile time see [commit-msg]. Because Rust v1.28.0 not yet support macros #[wasm_import_module], I need to update the test suites [code]

what the error in the chrome browser look like when compiling wasm code over 4KB synchronously at runtime using Webassembly.compile

🍐 Weird stuff (a058cb9)

Since Jest only care about main field [#module][#browser], I need to use this resolver script to make Jest respect pkg.module first. To make that script works, I need to make Jest ignore modules that support pkg.module [code]. Excluding webassembly-loader, I found out only rollup and estree that support pkg.module. Even though I have renamed .babelrc into babel.config.js to make it also lookup on node_modules folder, seems like babel-jest will not pick it up if the path doesn't get listed in transformIgnorePatterns option.

error cause by babel-jest doesn't transform es6 export in node_modules/*

TODO

  • Need to investigate the output result of wasm-pack to know if it produce smaller bundle size. It's a bit downer that latest stable Rust targeting wasm32-unknown-unknown produce wasm with the most minimum size 57KB :sad:
  • Integrate with wasmsnip, wasmopt, etc.
  • Investigate if wasm-pack can replace neon.

What I learn when updating this 2 packages

  • There is a trick in wasm-pack-plugin to install wasm-pack if it's not yet installed [ref]. Seems I can use that in the future update.
  • There is an ongoing PR for supporting Webassembly in Rollup [ref] :yay:. Hopefully it can tree-shake WebAssembly exports function. Seems I can begin to support code-splitting or dynamic import for wasm when it's merged. Less headache for me because I don't need to support exporting as WebAssembly.instantiateStreaming 😂
  • There is a stalled PR to make Jest support for the "module" package.json field [ref].
  • Babel configuration file have different behaviour depend on how you name the file (e.g .babelrc vs babel.config.js) [comment][docs].

You may notice that I never mentioned about wasm-bindgen. My other purpose creating this 2 packages is to streamline the web and embedded development, so I will avoid wasm-bindgen when creating a project that use this 2 packages. However, PR for integrating wasm-bindgen is always welcome 🤗.

GitHub Account

https://github.com/DrSensor


Sort:  

Thank you for your contribution. Again a great improvement to the existing codebase. I love the explanations. It's great to see a lot of tests in the repo, says a lot about best practices.


Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Thank you for your review, @codingdefined!

So far this week you've reviewed 8 contributions. Keep up the good work!

Hi @drsensor!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Hey, @drsensor!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!