Make data loading sufficiently random

[dlwh]

I wrote a design doc here outlining desiderata and the current status, along with a potential plan for moving forward. (Very open to other designs!)

The basic issue is that if docs are super long or not randomized when they come in, performance seems to suffer substantially.

Sub issues assuming we go with the plan above:

• implement a shuffle buffer
• implement seek in IndexedDataset
• implement JumpingDataset
• figure out serialization of datasets

[dlwh]

one random note is that according to https://medium.com/@duhroach/optimal-size-of-a-cloud-storage-fetch-8c270b511016 you should aim to read ~1MB chunks, though that's against the HTTP api so presumably the native API is more efficient?

[dlwh]

(also maybe i should just accept pre-declaring how big the sequences are and reprocessing whenever you change that so we can shuffle up front.)

[dlwh]

Updated design doc:
https://gist.github.com/dlwh/88ae0ef4efdc03fff2d3ade8d2f11ee6