Preprocess and transpile without bundling

[joeljeske]

First, thanks for your forward thinking and hard work!

I currently have a project using JSPM. In development, I am using system.js to transpile and wrap modules JIT in the browser.

For production, I want to serve every JS file individually, taking advantage of depCache and HTTP/2 to download deps in parallel. My issue, is that I do not see the systemjs builder command to preprocess and transpile all of my JS and other assets into static and individual JS files. I only see options to build and bundle an entire dependency chain given an entry point into a single file.

I am looking for an API to pass in an entry point and it prepares all of my assets as individual JS files to be loaded in production.

I am likely simply missing something, as this seems like the primary advantage of SystemJS.

Thanks!

[guybedford]

Thanks! Yes exactly polishing the workflows around this use case is important. There is builder.compile exactly for this purpose. Also see https://github.com/Munter/express-systemjs-translate for a project that uses this to cache translations as a single-file development server.

[joeljeske]

Ahhhh!! Interesting. I will take a look at that. Are there any docs for .compile? I do not see any reference to that function.

[joeljeske]

With this strategy, I would want all of my plugin loaders to run at this compile time. How should I handle non-JS files being wrapped into JS files?

For example, I have a file "resource.txt" that I am importing into a JS file. At production build time, should I builder.compile('resource.txt') to "resource.txt.js" or "resource.txt"?

I think I want to save it as a new file in my build directory as "resource.txt.js", but then how do I handle the import in the file that requests it? It would still be looking for "resource.txt".

Also, I think there are depCache considerations..

I am just wondering if there is a preferred or suggested way of doing this.

[guybedford]

This is always a tricky problem, and with a "blind" single-file compile you're best bet is to just keep the extension resource.txt even if it's being compiled by the text plugin into JS.

Lots of people seem to think thats odd, but I quite like that sort of workflow.

Renaming / moving options for modules in the builder are limited, but this is an area that could certainly be explored in future (eg relocateModules: { '*.txt': '*.js' } or something like that with that being by no means a suitable syntax or feature proposal).

[joeljeske]

Awesome, thanks for your suggestion on that.

So, it appears not many people (if any) are trying to do what I want, especially since the compile method is undocumented and there isn't many complaints. Is there a happy path for my use case? Or is there a roadmap toward working to this use case? I do not yet see tooling to ease this. I would love to contribute to the tooling, but am relatively new to SystemJS and want to ensure my efforts are well directed.

[joeljeske]

Do you have any workflow advice for processing without bundling? Currently, I am trace()ing through all of my "entry point" files, and compile()ing each file & dependencies individually, and writing out to a dist directory so that src to dist files are 1:1.

However, at production runtime, the jspm's generated metata .json file is still being requested. I would think that the info needed from the .json files in jspm_packages would be resolved at build time.

For example:

// Source
- app.js  <-- imports angular
- jspm_packages
  - github
    - angular
      - bower-angular@1.5.8.json
      - bower-angular@1.5.8
        - angular.js

// Built
- app.js  <-- imports angular
- github:angular
  - bower-angular@1.5.8
    - angular.js

404's on /path/to/baseURL/github:angular/bower-angular@1.5.8.json

FYI, I'm using JSPM 0.17 beta-31

[joeljeske]

Ha! I continued my search through the source code and found the answer to that piece of the puzzle:

There is an option to get the configs,
Builder.trace(file, {tracePackageConfig:true})

In the resulting array, the parsed object has a flag, "isPackageConfig" that I can key off of to determine whether to compile or to copy.

Also, when I have a good solution, I plan to create a gulp wrapper for this use case.

[lastmjs]

@joeljeske I believe I'm looking for nearly the same functionality you are. I'm working on a development/production server that automatically transpiles your scripts without any code changes needed. You can just <script src="path/to/file.ts"></script> and the .ts file will return transpiled. Here it is: https://github.com/lastmjs/zwitterion I've proven the concept, and now I'm going to build it out. I'd like to know if we're both thinking on the same page. Thanks!

[lastmjs]

@guybedford Is there a way to use compile to create a self-executing file?

[asapach]

@lastmjs, that's what buildStatic() is for: https://github.com/systemjs/builder#self-executing-sfx-bundles

[lastmjs]

@asapach I want the semantics of compile though. I don't want a bundle, I just want each file compiled into a self-executing file without its dependencies included. Is that what buildStatic does? The way I've been using it, it bundles.

[asapach]

@lastmjs, I'm confused. If you're not including the dependencies into your self-executing file, how are they resolved? SFX bundles don't use SystemJS runtime, they include their own.

[lastmjs]

@asapach Maybe I'm confused then. Here's what I do know, if I use compile on a file, I know that only that file will be transpiled (that's mostly what I'm using the builder to achieve) and ready for use in the browser. When I send it up the browser executes it, and all of its es6 imports are then retrieved over HTTP with one individual xhr request per import. That is working great for me. But, it is important to my use case to not have to include SystemJS. I would like the same behavior as compile, where only one file is "bundled", but I then want each es6 import dependency to be retrieved over HTTP. Is that unreasonable? I would think it is possible with buildStatic by statically building a file without its dependencies, and then somehow including all of the code that would retrieve the rest of the imports over HTTP. What I would like is compileStatic. I want the same functionality as compile, but I want it to be self-executing. Just like buildStatic is the same as build, but it is self-executing.

[asapach]

When I send it up the browser executes it, and all of its es6 imports are then retrieved over HTTP with one individual xhr request per import.

That's incorrect: the module will not be executed until you do System.import(). And you need SystemJS to load the dependencies - browser will not do it for you.

Static bundles don't load any dynamic dependencies (hence the name), because all their dependencies need to be included in the bundle at build time.

You can't have it both ways. You can try webpack with their code splitting and chunking, etc.

[joeljeske]

@lastmjs, as @guybedford mentioned in his initial response to my issue, there already exists middle-ware that supports server-side just-in-time transpilation using compile. It is a little outdated, but if you are interested in something similar, I would suggest making some PRs to that project. It has a decent starting point.

https://github.com/Munter/express-systemjs-translate

[lastmjs]

@asapach Okay, I see. Thank you for explaining that. Thanks @joeljeske That project is only for express. I'm working on a general purpose HTTP server.

[joeljeske]

@lastmjs, it appears it is already general purpose or is close to it.
Munter/express-systemjs-translate#169