Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dependency to bufio.Reader in internal carv1 package #160

Merged
merged 1 commit into from
Jul 15, 2021
Merged

Remove dependency to bufio.Reader in internal carv1 package #160

merged 1 commit into from
Jul 15, 2021

Conversation

masih
Copy link
Member

@masih masih commented Jul 15, 2021

Remove dependency to bufio.Reader in internal carv1 package that
seems to be mainly used for peeking a byte to return appropriate error
when stream abruptly ends, relating to #36. This allows simplification
of code across the repo and remove all unnecessary wrappings of
io.Reader with bufio.Reader. This will also aid simplify the
internal IO utilities which will be done in future PRs. For now we
simply remove dependency to bufio.Reader.

Relates to:

Remove dependency to `bufio.Reader` in internal `carv1` package that
seems to be mainly used for peeking a byte to return appropriate error
when stream abruptly ends, relating to #36. This allows simplification
of code across the repo and remove all unnecessary wrappings of
`io.Reader` with `bufio.Reader`. This will also aid simplify the
internal IO utilities which will be done in future PRs. For now we
simply remove dependency to `bufio.Reader`

See:
- #36
@masih masih marked this pull request as ready for review July 15, 2021 09:29
@masih masih requested review from mvdan and rvagg July 15, 2021 09:31
Copy link
Member

@rvagg rvagg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, so we have a carv1 inside carv2? is that going to be deduped at some point?

I have some memory of this Peek() code, trying to make it give proper errors on EOF vs "can't decode this uvarint". Other than that, I don't see a good reason to hold on to bufio.Reader; so go ahead and yeet it! This looks good to me. I'd be happy to see this migrate into the top level package too, I don't think there's a strong reason to keep it otherwise since that Peek() is not protecting an unbounded reader or anything.

@masih
Copy link
Member Author

masih commented Jul 15, 2021

oh, so we have a carv1 inside carv2? is that going to be deduped at some point?

Yep the context behind this is captured in #104. TLDR; go-ipld-prime dependency in FileCoin side needs to be upgraded before we can deduplicate. The plan is certainly to deduplicate. The same dependency issue on FileCoin side is blocking the implementation of Selective Car in v2. Forking the v1 into v2 also allows us to do such refactors.

I'd be happy to see this migrate into the top level package too

Yeah that would be nice; the bad thing is we can't roll out breaking changes so we have to leave the buffo.Reader taking APIs in v1 in place. But we can certainly mark them as deprecated and roll out new versions. Not sure how nice the naming of new functions would be since existing ones are named things like ReadHeader.

@masih masih merged commit a7291a8 into ipld:wip-v2 Jul 15, 2021
@masih masih deleted the v1-remove-bufio-dep branch July 15, 2021 09:49
@masih masih added this to the CAR v2 milestone Jul 15, 2021
@mvdan
Copy link
Contributor

mvdan commented Jul 15, 2021

SGTM. For what it's worth I'm not convinced we can simply deduplicate the carv1 packages; the v0 ones in master have APIs that limit what we can do, like func ReadHeader(br *bufio.Reader) (*CarHeader, error) still requiring a buffered reader.

We can certainly try to deduplicate some of it though.

@rvagg
Copy link
Member

rvagg commented Jul 15, 2021

the bad thing is we can't roll out breaking changes

.. but isn't bufio.Reader a superset of io.Reader? if we switched to just r *io.Reader wouldn't it still work if someone passed in a bufio.Reader?

@masih
Copy link
Member Author

masih commented Jul 15, 2021

the bad thing is we can't roll out breaking changes

.. but isn't bufio.Reader a superset of io.Reader? if we switched to just r *io.Reader wouldn't it still work if someone passed in a bufio.Reader?

mm good point; is that OK as far as Go API etiquette goes? @mvdan what do you think?

@mvdan
Copy link
Contributor

mvdan commented Jul 15, 2021

It's technically a breaking change, as one could e.g. pass ReadHeader as a parameter to a func that expects the bufio type.

It's likely that it would break noone in practice though, so we could always try and see. It's a v0 after all. I haven't checked if any other bits of the API needed changing for their uses in carv2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants