Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use gogo proto from jsuereth/protobuf fork for optional support #4259

Closed

Conversation

codeboten
Copy link
Contributor

Description:
Draft PR only to capture the work needed to update gogoproto to support optional fields

Link to tracking Issue: Fixes #4258

@@ -3269,6 +3273,19 @@ func (m *HistogramDataPoint) MarshalToSizedBuffer(dAtA []byte) (int, error) {
return len(dAtA) - i, nil
}

func (m *HistogramDataPoint_Sum) MarshalTo(dAtA []byte) (int, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a bug in the upstream gogoproto fork, let me work on fixing that. This should not be generated, instead the above if m.Sum != nil should have been generated.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 2, 2021

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Nov 2, 2021
@codeboten codeboten removed the Stale label Nov 2, 2021
@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Nov 20, 2021
@codeboten codeboten removed the Stale label Nov 22, 2021
@jsuereth
Copy link
Contributor

@codeboten Sorry for long delay, just did a round of bug fixes to gogoproto around optional generation for gogofaster.

  • Size, Marshal + Unmarshal generation should work w/ optional
  • Theoretically this will work with optional and setting gogo.nullable=false, but that kind of breaks my head why you'd do this.
  • There's a LOT of features in GoGo proto I need to expand test coverage for. I'll keep churning in the background BUT this should be able to pull in the optional fields we want in OTLP.

@codecov
Copy link

codecov bot commented Nov 23, 2021

Codecov Report

Merging #4259 (9196432) into main (c0e8f31) will increase coverage by 2.65%.
The diff coverage is n/a.

❗ Current head 9196432 differs from pull request most recent head 702d6db. Consider uploading reports for the commit 702d6db to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4259      +/-   ##
==========================================
+ Coverage   88.07%   90.73%   +2.65%     
==========================================
  Files         173      178       +5     
  Lines       10188    10360     +172     
==========================================
+ Hits         8973     9400     +427     
+ Misses        975      743     -232     
+ Partials      240      217      -23     
Impacted Files Coverage Δ
config/configtest/configtest.go 93.61% <0.00%> (-3.05%) ⬇️
service/zpages.go 79.38% <0.00%> (-0.82%) ⬇️
config/config.go 100.00% <0.00%> (ø)
config/exporter.go 88.88% <0.00%> (ø)
config/receiver.go 88.88% <0.00%> (ø)
config/extension.go 88.88% <0.00%> (ø)
config/processor.go 88.88% <0.00%> (ø)
config/configgrpc/configgrpc.go 93.91% <0.00%> (ø)
model/pdata/generated_common.go 100.00% <0.00%> (ø)
model/pdata/generated_resource.go 100.00% <0.00%> (ø)
... and 59 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c0e8f31...702d6db. Read the comment docs.

@codeboten
Copy link
Contributor Author

@jsuereth Thanks for the update, I rebuilt the container using this commit jsuereth/gogoproto@9c8da7e and regenerated the protos here

The Sum field in the Histogram is now a *float64. Not sure if we need the floatPtr convenience method or if there's a better alternative.
@bogdandrutu
Copy link
Member

@codeboten while discussing with @jsuereth we came with an alternative simpler solution, which is to use custom types for the "optional" fields that implement the "optional" part (since only double values). What do you think?

@codeboten
Copy link
Contributor Author

@bogdandrutu you're thinking we could use the sed script to substitute the optional field with a custom type instead and use that to generated the protos or are you thinking of changing the field to be a custom type instead?

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2021

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Dec 2, 2021
@bogdandrutu bogdandrutu removed the Stale label Dec 8, 2021
@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Dec 16, 2021
@codeboten codeboten removed the Stale label Dec 16, 2021
@jack-berg
Copy link
Member

@bogdandrutu / @jsuereth wondering if you could answer @codeboten's question. If I'm not mistaken the answer dictates whether we need to make changes to the protos or to the collector proto generation code.

@codeboten
Copy link
Contributor Author

@jack-berg the conversation continued on this other PR (for the 2nd prototype). I'm now working through a third prototype to move away from gogo and use the standard protobuf library + the vtprotobuf extension

@jack-berg
Copy link
Member

Thanks for the update and for connecting the dots @codeboten! 👍

@github-actions
Copy link
Contributor

github-actions bot commented Jan 5, 2022

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@codeboten
Copy link
Contributor Author

Closing this for now.

@codeboten codeboten closed this Jan 13, 2022
@codeboten codeboten deleted the codeboten/support-optional branch June 22, 2023 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support optional fields in proto
4 participants