-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for native compilers #87
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tremendous amount of thoughts and ideas! I realized that it is still WIP, still I would contribute some review findings, that would help polish the functionlity. I would take some of them for myself to address further.
…_refactor Re-factor of compile* logic and other cleanup
…tion for all samples in a directory
…typed-ast into native_compilers
…ports parser from coverage report.
Codecov Report
@@ Coverage Diff @@
## master #87 +/- ##
==========================================
+ Coverage 92.23% 92.26% +0.03%
==========================================
Files 246 254 +8
Lines 5033 5195 +162
Branches 800 817 +17
==========================================
+ Hits 4642 4793 +151
- Misses 249 253 +4
- Partials 142 149 +7
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking more like "final", except few TODOs that are still need to be addressed (compiler cache location, native compiler list updates).
…ey affect nyc / wasm solc crashes.
…s between src/ and dist/ during development.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome work on expanding the tests and cleanin up Pavel!
Found a couple of small bugs and missing docstrings in my code.
Also made some general comments on improving some tests in later PRs.
Will fix my comments, file issues for anything in later PRs and then re-run tests. Other than that I am good with it too.
Final self-review fixes
…er in identifiers
…typed-ast into native_compilers
This is a WIP PR for collaborating on implementing support for using the native compiler binaries instead of the wasm ones.
WASM binaries sometimes take a lot more time (likely loading or first JIT-ing). This causes us to have VERY long test cycles, with CI often taking an hour.
This PR has several things implemented already:
Adds a new option "frontend" to all the compile entry points. Valid values are
CompilerFrontend.{Default, WASM, Native}
. (Note that the name 'CompilerFrotend' is terrible. We should pick something more accurate. CompilerBackend maybe? Idk)Re-factors a lot of the compiling logic into new classes in
frontend/wasm.ts
andnative_compilers.ts
(note -native_compilers.ts
should be moved underfrontend
)Makes all of the
compile*
entry-points async (as they may need to fetch a native compiler on-the-fly)Adds logic to download compiler binaries for the current architecture in
native_compilers.ts
Adds a new partial Soliidty parser for top-level definitions only in
compile/inference/top_level_definitions_parser.pejgs
. We can use this to detect all import statement in a file (along with all compiler pragmas as well)Adds some unit testing for the new parser
And its missing several more things:
CompilerFrontend
is the wrong name - should probably be morebackend
for every solidity file X.sol in the samples, that doesn't import anything:
let a = top level definitions in X.sol after running the WASM frontend
let b = top level definitions in all files discovered after running the top-level definitions parser on X.sol
expect(a).toEqual(b);
Run the unit tests from (4) on a sizable subset of flattened files from the contract archive repo (to sanity check the parser)
Write the logic in the
native_compilers.ts
that uses the top-level definitions parser, and theImportFinder
provided by the callers, to find all imported files by the compiler. If any of those files live require a path-remapping (e.g. @open-zeppelin imported from node_modules/), then build a proper path-remapping option to be passed to the native compilerAdd a unit test that runs both the native compiler and the wasm compiler for every sample that we have under tests, asserting that they produce the same ASTs
Run test 7 for a good subset of the contract archive repo to sanity-check the whole logic.
I think at this point we can trust the code enough that it works. @blitz-1306 what do you think?