Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for emscripten (wasm) #7

Merged
merged 2 commits into from
Dec 6, 2023
Merged

Add support for emscripten (wasm) #7

merged 2 commits into from
Dec 6, 2023

Conversation

bytedream
Copy link
Contributor

@bytedream bytedream commented Nov 24, 2023

This adds support for wasm via the wasm32-unknown-emscripten toolchain.

Behavioral changes

Given the following lua function:

int lua_getglobal (lua_State *L, const char *name);

The following implementation which just works fine on Tier 1 platforms (at least on those I've tested), will cause a linking error when compiling to wasm32-unknown-emscripten:

#[no_mangle]
pub extern "C" fn lua_getglobal(state: *mut c_void, k: *const c_char);

If such a mismatch is the case, the linkage will always fails with a "function signature mismatch" error. The extern "C" fn must always return the same value as the original C function.
This also applies for the ! type, for example the lua_error C function returns a int although it never actually returns. In Rust, this function could be represented as pub extern "C" fn lua_error(lua_State *L) -> ! (which is how e.g. mlua does it), but the linker also forbids this.

Other wasm targets

Because lua uses libc, other wasm targets cannot be targeted natively (atm):

  • wasm32-wasi has a libc port which is built on top of WASI system calls (wasi-libc) but this port doesn't fully support all calls lua requires to work. The error I ran into was the missing setjmp call which lua uses to handle errors.
  • wasm32-unknown-unknown & wasm64-unknown-unknown have no libc implementation/wrapper at all.

Notes

When testing via cargo test, the compilation will fail with "undefined symbol: <some libc symbol>". The error can be ignored by declaring the env variable RUSTFLAGS="-C link-args=-sERROR_ON_UNDEFINED_SYMBOLS=0". With this set, testcrate compiles without issues.

cargo test also tries to execute the generated test binary, which is actually just a .js file that imports the compiled wasm binary, but this will result in an error. To prevent this, you can set the env variable CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_RUNNER="node" to use nodejs to execute the js file.

The asmjs-unknown-emscription toolchain is also supported because it uses the same tools as wasm32-unknown-emscripten but as it is considered depreacted in favor of wasm I didn't add any tests for it.

@bytedream
Copy link
Contributor Author

bytedream commented Nov 25, 2023

Okay, after a bit of testing with mlua I'm not sure anymore if supporting emscripten is that useful/great.
I've ran into several issues which aren't present when compiling to native code, the modified mlua crate with emscripten support (I've only added lua51 compatibility every "normal" lua version is supported) is available at https://github.com/ByteDream/mlua.

When running the mlua test suite, a huge amount of stack and heap memory is required in order to pass successfully. This must be set via the RUSTFLAGS env variable which isn't that great for the user experience of 3rd party developers which simply want to use this crate out of the box without manual tweaking

# without --release the memory size must be increased even more
$ CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_RUNNER="node" RUSTFLAGS="-C link-args=-sERROR_ON_UNDEFINED_SYMBOLS=0 -C link-args=-sSTACK_SIZE=50MB -C link-args=-sTOTAL_MEMORY=100MB" cargo test --tests --features lua51 --target wasm32-unknown-emscripten --release

Also the lua_error (and maybe luaL_error too, haven't tested it) sometimes return even if it shouldn't. This causes the wrapper function I've wrote to reach an unreachable!() statement which should never be called (and doesn't get called if using a "normal" toolchain). This might be related to emscripten-core/emscripten#13166.

# the "guided_tour" example triggers this
$ CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_RUNNER="node" RUSTFLAGS="-C link-args=-sERROR_ON_UNDEFINED_SYMBOLS=0" cargo run --features "lua51 async serialize macros" --target wasm32-unknown-emscripten --example guided_tour

@khvzak
Copy link
Member

khvzak commented Dec 3, 2023

Thanks for the PR! I'll take a look to what required for this target too.

a huge amount of stack and heap memory is required in order to pass successfully

This possibly could be related to some tests that explicitly try to overflow Lua stack to check the behaviour.

Also the lua_error (and maybe luaL_error too, haven't tested it) sometimes return even if it shouldn't.

Actually Lua can be compiled with custom LUAI_TRY / LUAI_THROW impls to use rust panic/catch_unwind functions. I tried it (just as an experiment) couple of years ago and it worked.

@tari
Copy link

tari commented Dec 6, 2023

I bumped into the same function signature mismatch errors in an application I'm working on, which I resolved in mlua-rs/mlua#337 using what turn out to be the same techniques (#[link_name]). I haven't tried running the resulting binary yet, but I think this is a fine approach.

@bytedream
Copy link
Contributor Author

I switched the emscripten build process to use the C++ compiler and now lua_error behaves as it should.

Actually Lua can be compiled with custom LUAI_TRY / LUAI_THROW impls to use rust panic/catch_unwind functions. I tried it (just as an experiment) couple of years ago and it worked.

Thanks for pointing this out, I actually tried to compiled it with C++ before I opened this PR too but ran into an issue where C++ exceptions could not be caught... turns out I had to set the -fexceptions flag to make it work. I probably wouldn't have found out/searched for a solution if you hadn't written that it had already worked for you.

@khvzak khvzak merged commit 37fbf28 into mlua-rs:master Dec 6, 2023
9 of 27 checks passed
@khvzak
Copy link
Member

khvzak commented Dec 7, 2023

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants