riker for hash table

[thedavidmeister]

IMPORTANT: tests are failing CI because the docker box uses stable and we need nightly. test locally for review

fixes #135
fixes #137

this introduces the Riker actor system for #135

the problem

the state tree (redux style state) is great for small, well known items of data but struggles with:

• generic traits: trait info propagates to the root of the state tree and everything that touches it
• lifetimes: the same issue as generic traits
• large data: the state tree is cloned on every "mutation" by design
• external resources: any external state/service must be a black box resource ID, so "time travel" is meaningless in this context

(actual photo of generic traits spreading through our application state)

this solution: riker actors

http://riker.rs/ implementing the riker library for actors

the first implementation is for hash tables

riker core concepts:

• protocol: a set of valid messages that can be sent (e.g. an enum)
• actor system: manages all the actors for a given protocol
• actor: anything implementing Actor that creates new actor instances and defines message receive
• actor instance: an instance of the actor struct that has internal state and is tracked by the actor system
• actor ref(erence): an ActorRef<MyProtocol> that can tell messages to the actor instance it references via. the actor system

the approach in this PR is to implement the HashTable trait for actor refs. the actor ref is passed an "inner table" that also implements the same HashTable trait at construction time. the actor ref becomes a standardised, transparent wrapper around the inner table implementation.

this means that calling table.commit() and table_actor_ref.commit() do the same thing

the benefit of table actor refs:

• known size at compile, safe as properties of structs/enums
• small size, almost free to clone
• safe to share across threads and copy, no Arc reference counting, no locks, etc.
• safe to drop (the actor system maintains a URI style lookup)
• known type, no onerous generic trait handling
• no onerous lifetimes

implementation deets

the 1:1 API implementation between actors and their inner table is achieved by internally blocking on an ask from riker patterns - https://github.com/riker-rs/riker-patterns

the actor ref methods implementing HashTable send messages to itself

calling table_actor_ref.commit(entry) looks like this:

0. the actor ref constructs a HashTableProtocol::Commit message including the entry
1. the actor ref calls its own ask method, which builds a future using riker's ask
2. the actor ref blocks on its internal future
3. the referenced actor receives the Commit message and matches/destructures this into the entry
4. the entry is passed to the commit() method of the inner table
5. the actor's inner table, implementing HashTable, does something with commit (e.g. MemTable inserts into a standard Rust, in-memory HashMap)
6. the return value of the inner table commit is inserted into a CommitResult message
7. the CommitResult message is sent by the actor back to the actor ref's internal future
8. the actor ref stops blocking
9. the CommitResult message is destructured by the actor ref so that the return of commit satisfies the HashTable trait implementation

riker ask returns a future from the futures 0.2.2 crate, table_actor.ask calls block_on and unwrap against this ask. both the block and the unwrap should be handled better in the future.

limitations, tradeoffs

i wish that this were free

nightly rust

IIRC riker (or futures, or both) needs nightly rust (TODO: double check this, document why)

rust futures

the futures story for rust is "WIP"

• futures started as a crate that started as a spin off from tokio
• syntax like async/await exists as macros in crates that monkey patch and re-export futures
• async/await definitely needs nightly as it relies on at least 3 experimental nightly features, one (generators) documented as "extra-unstable" (https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html#generators)
• some macros like async/await have started landing as experimental nightly features
• the experimental nightly landings in the compiler thoroughly break the crates of the same name, often with no reasonable workaround await is ambiguous after updated to nightly-2018-06-24 and after alexcrichton/futures-await#106
• the experimental breakages have gotten so bad/frequent that the async/await maintainers basically abandoned the crate (in favour of waiting on core to catch up) Is anyone interested in maintaining this crate? alexcrichton/futures-await#109
• there are 3 flavours of futures in active development, 0.2.x, 0.3.x and the nightly compiler
• riker targets 0.2.2 atm, but the README says it will shift to newer futures branches as they mature
• the futures API/syntax changes a lot across versions
• the documentation/blogs are typically unclear which version they are targetting

even so, futures look far more pragmatic than thread/channel juggling for many situations

for example, the observer/sensor/event-loop state model we implemented ad-hoc looks a lot like some of the future/poll/task system internals

dependency on riker

once/if we merge this, we're pretty much in bed with riker moving forward:

• it makes little sense to reinvent our approach, the same problems in our state tree apply to logging, network state, etc.
• riker is a relatively new crate
  • sub 20 stars on github
  • ambitious but incomplete roadmap
  • incomplete docs
  • ??? team, 1 person? funded? motivated by? open to collaboration?
• the same things that make riker powerful (opinionated, implementing a very specific approach) also mean we must pay attention along the way (discussion, careful prototyping, etc.)
• riker is clearly the type of thing that we'd find ourselves reaching for a lot (at least for external state, possibly even for some internal state too) once it is in there

riker ask actor/future vs. futures poll

rust has limited/awkward callback support so futures in rust (unlike basically every other language with futures) is poll based, and must be "driven" externally

the native future model is designed primarily to be composable on the level of poll. the whole thing only works because nested poll calls in nested futures can bubble their results.

the usefulness of futures comes in large part through the various abstractions for controlling how nested/parallel poll results are are called, merged, blocked on, etc.

riker on the other hand is all about independent actors sending and receiving messages asynchronously. the need for futures to make this work is equal parts implementation detail and "adapter" for the broader rust ecosystem. i get the feeling that if there was a viable non-futures approach, then riker would use that instead.

for example, i couldn't figure out how to usefully nest ask futures across multiple actors like we could nest poll calls in a vanilla future. the underlying futures task context (needed by poll) is hidden somewhere in the riker internals. nested blocking on ask is a compiler error.

at this point i'm willing to chalk the friction up to my own inexperience with futures/riker and the relative immaturity of both libraries. it's certainly not clear that we have a hard requirement for nested actors... after all, i managed to find a way around it for the HashTable use-case.

Why not use...

Structs

We could fix the generic trait issues by:

• making a wrapper struct with an inner hash table
• the wrapper struct goes in the state chain with a known type
• the new method of the wrapper takes a <HT: HashTable> and acts as a buffer

the problem (ignoring likely unknown size issues for the inner table) is that cloning the wrapper struct also clones the inner table. some inner implementations might be a stateless reference (e.g. a URL) and be safe to clone. Many will be stateful (e.g. MemTable) and so can't be cloned safely.

riker actor references are always stateless. the actor + actor system quarantines state for us, regardless of the inner implementation.

Actix

Actix looks great:

• actor system
• stable rust (i think)
• many github stars (~1500)
• nested actors seem to work well and clearer support for call/response actor comms

but has these limitations that looked like dealbreakers when i reviewed with the team:

• roadmap is not covering what we would want in a generalised actor framework (e.g. persistence, event logs, actors over network, pluggable backends, etc.)
• seems more monolithic, wasn't as clear how to plug it into our existing systems
• the API broadly doesn't match our mental model of what we want to achieve here
  • for example, it doesn't seem to have actor references to pass around and plug into our state tree
• docs are surprisingly unmaintained, riker has far more info, most of the actix docs are "TODO"

changes

💥 💥 💥

💥 💥 💥

• adds riker
• adds futures 0.2.2
• adds riker config symlinked into place as toml
• defines a protocol for HashTable actors
• creates an actor system for HashTable through lazy static
• more explicit naming, e.g. get vs. get_entry and get_pair
• extends trait bounds on HashTable to be actor ref friendly
• implements HashTable for actor ref
• implements an actor for hash tables
• refactor Chain to use the HashTable actor ref instead of the inner implementation
• implement MemTable for agent state
• test that commit/get actions and zome functions can round trip data

followups

• riker config tweaks review riker config #228
• use actors for context can context be upgraded with riker? #229
• partial equality for chains review PartialEq for Chain in context of actor refs #257
• refactor top pair move set_top_pair validation logic inside set_top_pair #258
• atomic set top pair handle the case where commit to a table succeeds but setting the top pair fails #259
• source chain as hash table should SourceChain have a bound on HashTable for consistency? #261
• file table file system backed hash table #231
• remove function call id should reduce_zfr store state using function call or action as key? #198
• do we need HashTable setup and teardown? do we need setup/teardown for HashTable? #262

[thedavidmeister]

this actually does a round trip!

time to start polishing..

[sphinxc0re]

Maybe not do this. This can get out of hand when used overly. better return an immutable reference to self.chain

[thedavidmeister]

@sphinxc0re ok i'll change it, i have some questions about this and mutability but they can wait until later

[zippy]

Fabulous. Big step forward.

[zippy]

can we make this a Result?

[zippy]

I like this much more that the state has a chain than a top_pair

[zippy]

nice test!!

[zippy]

I think his will conflict with #268 so someone will have to do the merge...

[lucksus]

I am a bit concerned of all those commit()s here which actually should be put()s.
I've created ticket #274 as a follow up task to make these function names match the mental modal we have...