Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement local puller and pusher #1871

Closed
wants to merge 30 commits into from
Closed

Conversation

thesayyn
Copy link
Collaborator

@thesayyn thesayyn commented Jan 25, 2024

Take two on #1433

Most of the crane cli surface will be able to work with images on oci-layout.

pkg/v1/local/layer.go Outdated Show resolved Hide resolved
pkg/.DS_Store Outdated Show resolved Hide resolved
@imjasonh
Copy link
Collaborator

imjasonh commented Feb 9, 2024

Can you fix the build errors so we can see if any other tests are affected?

Error: pkg/v1/local/puller.go:63:21: undefined: errors.ErrUnsupported

@thesayyn thesayyn changed the title poc puller feat: implement local puller and pusher Feb 17, 2024
@thesayyn
Copy link
Collaborator Author

Can you fix the build errors so we can see if any other tests are affected?

Error: pkg/v1/local/puller.go:63:21: undefined: errors.ErrUnsupported

Done

pkg/v1/layout/pusher.go Outdated Show resolved Hide resolved
pkg/v1/remote/image.go Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Feb 22, 2024

Codecov Report

Attention: Patch coverage is 43.12796% with 240 lines in your changes are missing coverage. Please review.

Project coverage is 70.43%. Comparing base (a0658aa) to head (9a1cd79).
Report is 3 commits behind head on main.

Files Patch % Lines
pkg/v1/layout/sink.go 42.10% 91 Missing and 19 partials ⚠️
pkg/crane/image.go 40.57% 30 Missing and 11 partials ⚠️
pkg/v1/layout/source.go 40.62% 30 Missing and 8 partials ⚠️
pkg/v1/remote/puller.go 0.00% 15 Missing ⚠️
pkg/crane/options.go 42.85% 8 Missing ⚠️
internal/cmd/edit.go 36.36% 5 Missing and 2 partials ⚠️
pkg/v1/layout/layer.go 36.36% 7 Missing ⚠️
pkg/v1/remote/artifact.go 0.00% 6 Missing ⚠️
pkg/crane/get.go 62.50% 3 Missing ⚠️
pkg/v1/layout/write.go 72.72% 2 Missing and 1 partial ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1871      +/-   ##
==========================================
- Coverage   71.65%   70.43%   -1.22%     
==========================================
  Files         123      128       +5     
  Lines        9928    10320     +392     
==========================================
+ Hits         7114     7269     +155     
- Misses       2115     2310     +195     
- Partials      699      741      +42     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

path Path
}

func NewPuller(path Path) remote.Puller {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to define a new interface in v1 that both remote.Puller and layout.Source would implement.

In my mind I'd want:

package v1

type Source interface{
 // minimal subset of methods needed to read things
}

type Sink interface{
 // minimal subset of methods needed to write things

}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would remove the dependency between layout + remote and avoid the need to convert remote.Puller into a struct below. Spitballing the set of methods:

type Source interface {
  Layer(...)
  Head(...)
  Get(...)
  Artifact(...)
}

type Sink interface {
  Push(...)
  Upload(...) // Maybe not?
}

The verbs are a bit clunky for non-remote implementors, but I don't see a clean way around that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, i can't put source and sink into v1 root folder as it creates cycle between v1 -> partial.Artifact -> v1

Would putting this into its own directory be acceptable?

package github.com/google/go-containerregistry/cmd/crane
        imports github.com/google/go-containerregistry/cmd/crane/cmd
        imports github.com/google/go-containerregistry/internal/cmd
        imports github.com/google/go-containerregistry/internal/verify
        imports github.com/google/go-containerregistry/pkg/v1
        imports github.com/google/go-containerregistry/pkg/v1/partial
        imports github.com/google/go-containerregistry/pkg/v1: import cycle not allowed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay i have created the sinksource (name TBD) directory.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now ready for another look

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonjohnsonjr what do you think about v1.Artifact to get around the import cycle?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that one also sounds fair, though now v1.Artifact would depend on bunch of partial interfaces.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anything i could do to push this over the edge @jonjohnsonjr? I'd really like to land this, as i have some people who's been waiting for it for a long time.

pkg/v1/remote/puller.go Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants