Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAG Service integration #9

Open
hannahhoward opened this issue Mar 30, 2022 · 0 comments
Open

DAG Service integration #9

hannahhoward opened this issue Mar 30, 2022 · 0 comments

Comments

@hannahhoward
Copy link
Collaborator

hannahhoward commented Mar 30, 2022

Largely putting this one off for last, but:

The DAGService and asssociated walking functions are essentially the core traversal methods in go-IPFS

For Lightning storm to work, it needs to work with the DAG service.

My first version thought is:

  • Lightning storm is essentially an indepedent library that is used to supercharge the speed of DAGService dag walks -- it's kicked off in pararel to the dag walk and hopefully writes to the local blockstore before the DAGService gets to it
  • if it doesn't get ahead of the DAG walk, no worries -- they're complementary actually -- lightning storm will use the BlockService so that if blocks are already fetched by the dag service, we'll just read them from the dag store.
  • we integrate this into all of the walk functions in go-merkledag
  • there are probably other places walks happen in go-ipfs that we'll have to add to eventually
  • this works well for backwards compatibility as once Backward compatibility for graphsync 2.1 protocol go-graphsync#376 ships we can just short circuit the lightning storm work if Graphsync 2.1 is not present on the other end

There's a penalty paid in terms of double disk I/O but I think that's probably ok until we can delete the DAGService in a much later endeavor.

@hannahhoward hannahhoward mentioned this issue Mar 30, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant