distribution

[stefanpenner]

• cdnjs Add RSVP to cdnjs #206
• duo Support DuoJS #314
• npm
• bower
• support spm stefanpenner/es6-promise#73 ???

@trek I know you have some thoughts on this topic, want to share?

[trek]

Oh, I have many unpleasant thoughts about this issue.

This is basically the same issue as timrwood/moment-es6#2 and slightly related to jquery/jquery#1691 so @timrwood and @scottgonzalez,@dmethvin,@mzgol should chime in as well.

The full list of distribution channels we should support as good citizens is probably at least: cdnjs, Duo, Bower, and npm. I suspect this list will grow over time.

cdnjs and npm are easy because they have a distinct publication phase where we can author in ES6 and automate transpilation to other formats. Bower and Duo complicate the problem because the want to use a source control system for asset hosting.

The two competing problems with this approach are

1. Most JavaScript is not authored in the format that people consume it.
2. Modularity only Just Works™ when your each item in your dependency tree has a standard way to expose itself.

So, basically we have to author in one format and transpile to multiple others, with several possible variations:

@timrwood laid them out in his issue, but I'll duplicate it here:

Environment	Bundled?	Minified?
amd	Y	Y
commonjs	N	N
es6	N	N
globals	Y	Y

I've removed UMD from this list because UMD is a lie

I'm increasingly uncomfortable with committing built assets into source control like we do on router.js. It was a pragmatic step but it's time to have a real solution.

I think it's important that we work with Bower and Duo to come up with a solution, so it would be great if @matthewmueller from Duo and @desandro/@sheerun from Bower would participate. Library authors simply can't bear the burden of this on their own. At best it's a giant amount of duplicated work. At worst every library will come up with their own incompatible solution further fragmenting the community. You know what JavaScript people are like.

Bower and Duo need some strategy for multi-module support.

Solutions that have already been tried but don't work:

• tell everyone to author in the same module format (maybe it would work with ES6? It hasn't worked for AMD/CJS for the last 5 years).
• come up with a new module format that allows interop (UMD didn't work).
• create shim repos (https://github.com/components/) pollutes Github with random read-only repos. Is a maintenance headache for authors (e.g. Issues opened), is a discoverability headache for consumers. Only happened for maybe 1% of all libraries, about 20% of top quartile by install.
• Check in transpiled code into source control. Same as shim repos basically. Makes the main entry of a manifest meaningless, complicates tag-based publication, etc

Taking the above options off the table what do we try next?

I proposed one solution in bower/bower#1606 (making the package manager release asset aware) @sheerun suggested a https://github.com/Homebrew/homebrew style meta-repo where a third group (not authors, not package managers) can commit transpilation recipes.

I'd love to hear other ideas too. This really, really needs to get solved in some solid way for 2015-2016. My secret hope is that by 2017 we'll all just be using ES6 modules, but two years is a long time. Plus, I'm probably being overly optimistic.

[trek]

Oh, and I think one major issue with traspiling is the ugly ['default'] problem

But it looks like @eventualbuddha and @guybedford think it can be side-stepped if authors choose to opt-in.

Another note: @ef4 has been using Github assets for publication for a while. He might have useful insight into the results https://github.com/ef4/liquid-fire/releases

[sheerun]

As Trek mentioned, I think it's far more important to give developers standardized way of building 3rd party library in different formats, rather than standardized way of downloading it in different formats.

Standardizing both would be great, though.

[stefanpenner]

I've removed UMD from this list because UMD is a lie

UMD will work fine for this project as it is self contained and exposes a vary minimal extend interface.

Library authors simply can't bear the burden of this on their own.

Yes, currently this is all horseshit I don't have time to constantly add flavor-of-the-week distributions.

come up with a new module format that allows interop (UMD didn't work).

I think the Interopt story between different modules is killing the eco-system. As soon as you add deps to your project, the many 2 many situation this creates is often more complicated then what the library itself is doing. Actually it gets worse its really a n^3 problem when you add different distribution systems to the picture.

Honestly, the solution is for a distribution systems to handle this burden for library authors.

Said distribution system would require authorship in es6, and it could emit the module variants needed, also it could theoreticallyy pass along your module to other distributions but honestly though, if it makes module authors lives easier hopefully it just becomes the defacto distribution system.

Now, I really don't want someone to create yet another distribution system, instead as a community we need to re-invest in something that exists. IMHO NPM is the best candidate for this.

[trek]

UMD will work fine for this project as it is self contained and exposes a vary minimal extend interface.

If there is a way to transpile, publish, and consume both AMD and CJS, UMD isn't needed.

Honestly, the solution is for distribution systems to handle this burden for library authors.

I agree, but the folks at npm and Bower seem to disagree. No idea what Duo plans for solving this.

Basically nobody wants to take responsibility. It for sure should not be a consumer's problem. It's just irresponsible to force ever consumer to deal with this.

Putting this in the package manager seems like the least overall effort. There are hundreds of thousands of consumers, thousands of authors, and a only a handful of distribution systems. Distribution systems are also the medium that binds authors and consumers.

As an author, I'm happy to use whatever mechanism they suggest that solves this instead of inventing more one-off solutions that further fragment the landscape for consumers.

https://github.com/bower/bower.json-spec has languished for more than a year. I don't think it will ever succeed because it needs to convince all authors to reorganize their source in similar ways. This just won't happen. It's too much work coordinated between too many people.

Having a publication format spec could succeed because that space is wide open. Nobody's really married to any specific pattern yet and the proliferation of node-based build tools makes it trivially easy to take your source – organized in a way that makes sense to you – and transform it into a specific formal output structure.

[ef4]

I'm looking at it from a marketing and adoption standpoint. It shouldn't be the consumer's problem, but today it is, so those are the people who are motivated to adopt a better pattern. Once you have consumer adoption, you'll have the leverage to convince many (but never all) authors to make their packages more friendly to your tooling.

I think a homebrew-like model fits this pattern. Doubly so if somebody is willing to host build & distribution infrastructure so that the consumer-side tool can remain dead simple.

It also creates a smooth upgrade path to the happy shiny future, because you can let authors opt-in to providing their own compatible metadata/recipes from within their source.

And you could probably roll out the top 100 clientside packages on day one, without filing a single PR anywhere.

[ef4]

(I realize that what I just proposed may sound like "More fragmentation!", but I think that can be avoided by being Borg-like and assimilating all the existing packages.)

[trek]

@ef4 I don't think you're describing a solution that makes it a consumers problem. You're describing Yet Another Package Manager. A very heavy-handed one where the package manager contributors crowd-source source-to-build transforms. I'm not against such an idea but I think adding more package managers into the mix makes it worse for consumers, not better.

I know it's part of JavaScript's Ethos that "if one solution is good, 10 mostly identical solutions must be better!" but just this once I'd like to see if we can convince existing tools maintainers to solve this problem before further complicating things.

npm has already said "no, just use browserfiy/come up with your own solution using npm". If Bower and Duo also say "no", then it's time to look elsewhere.

[ef4]

Yeah, I'm just cynically skipping ahead to the part where none of the package manager maintainers want to change. Because that's what I fully expect to happen. But I'm glad you're not as cynical. :-)

[guybedford]

I'm not that aware of the direct publishing implications for rsvp right now, so it's difficult to be sure if this is answering the direct issue or going off-topic, apologies in advanced for stating the obvious or irrelevant. Yes there are a lot of publishing workflows, but our publishing workflows have to take into account all the usage workflows we want to support, and it's an unfortunate fact that does take some effort to cover. There could definitely be a role to document / standardise these workflows for npm-first or global-first publishing from ES6 authoring. We're not at ES6 publishing though and I don't think we should think about that at all until we're using ES6 environments.

I've also suggested 6to5 consider implementing the default collapsing in their latest interop work (babel/babel#95 (comment)).

With ES6 it can actually work out that the development resources become the production resources in many cases, just like CommonJS. I don't think loading from source repo has to be seen as invalid entirely for JavaScript. The only important thing is that sources are compiled (jsx -> js, less -> css), but it is not important that they are minified or bundled, which is the job the production workflow (with help from the package manager tooling or not). If there are single-file bundle approaches being used for other reasons (perhaps to support globals, or for collapsing module trees for performance with es6-module-transpiler), then that is fine too though.

In the scenarios where the sources + uncompiled files are small enough, a dist folder in the repo that is only updated on tagged releases works if you're happy with that. Alternatively set up a standardised TravisCI build that automatically deploys to GitHub releases (http://docs.travis-ci.com/user/deployment/releases/). jspm will download from GitHub releases if there is one available for the given tag, allowing for release workflows to work. Separate production repos on GitHub does seem ugly, but if people are happy to maintain that I don't see an issue with that either.

Note that jspm is trying to capture a lot of these issues by supporting loading packages from any location in any format. So you can point the jspm metadata to a global CommonJS AMD, or ES6 main or folder etc and just set up the dependency configuration correctly for things to work with the right circular-reference supported inter-format interops. From there it can offer builds etc. As an ES6 environment it requires buy-in though yes. The point is that we don't put requirements on authors, but support existing publishing methods.

[dmethvin]

Honestly, the solution is for a distribution systems to handle this burden for library authors.

Yes. The Internet is good at disintermediation. What we need this time is some intermediation. I want a middleman to be a wholesaler and deal with packaging for me. All these packages and wrappers do not belong in a development repo. They are a distraction from real project goals.

It's as if the potato farmer is responsible for delivering potatoes in any form someone might want them. That's not my job. I grew the damn things and put 'em on a truck. You bag 'em, store 'em, mash 'em, bake 'em, serve 'em any way you want. I do not want to clog my farm with warehouses full of potatoes processed and bagged in every possible way so that you can drop by and pick them up on the way home from work. That's not my job. I've got enough crap to do making this code work right.

But this is a bit of a digression from rsvp. Let's figure out a place where the mob can meet, light torches, and storm some appropriate castle.

[stefanpenner]

But this is a bit of a digression from rsvp. Let's figure out a place where the mob can meet, light torches, and storm some appropriate castle.

+1

[trek]

@dmethvin

Yes..... yes... let hate flow through you

to this issue bower/bower#1606

[stefanpenner]

npm
bower

basically these two are fine