Throw an error with mismatched Apache Arrow version

[willium]

What happens?

Right now, if you have a mismatched apache arrow version, DuckDB WASM fails silently or throws very confusing errors. I was trying to create a table, and nothing happened.

To Reproduce

Target Arrow 15 or 16, and try to use DuckDB.

Browser/Environment:

Chrome

Device:

Mac

DuckDB-Wasm Version:

Latest

DuckDB-Wasm Deployment:

App

Full Name:

William Wolf

Affiliation:

Gestalt

[domoritz]

This keeps coming up. I wonder what the best way to resolve this is. Maybe only include arrow as a peer dependency or a more robust check for isArrowTable?

[domoritz]

Related vega/vega-loader-arrow#350

[rob-blackbourn]

Is the arrow esm vs csj problem I reported here #1708 ?

[domoritz]

That's related. The problem can also happen when you have e.g. arrow 14 but duckdb wasm uses arrow 15.

[carlopi]

Problem seems quite hard, it's the problem of being successful, but instanceof feel like the wrong solution here. Would it not be possible to duck-type this and assume that if something looks like an arrow table has to be an arrow table?

Or possibly using stuff like obj.constructor.someStaticFieldThatEncodesTheArrowVersion ?

[carlopi]

One note is that DuckDB-Wasm has possibly a simpler job that regular arrow, since we can implement detection our side, once a semi-reliable detection function is approved.

[stefnotch]

One option might be to re-export apache arrow? At least that's what some Rust libraries do for their dependencies.

[domoritz]

That doesn't help if another library has arrow as a dependency. This really needs to be solved at the npm/yarn/etc level but we can try to warn people when they may have multiple versions.