Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Common: fix PII enrichment adding empty objects instead of missing properties #351

Closed
chuwy opened this issue Sep 16, 2020 · 1 comment
Closed
Assignees
Labels
bug Something isn't working
Milestone

Comments

@chuwy
Copy link
Contributor

chuwy commented Sep 16, 2020

If we specify JSONPath like $.['a', 'b'] and proocess JSON that miss at least one of those properties, the another one will be replaced with an empty object, e.g.

{"a": 1}

on input, and

{"a": 1, "b": {}}

on output.

This can break downstream consumers as b is supposed to be non-object.

It seems to be regression since R117, but cannot see the reason yet, no major changes has been made in PII enrichment and JSONPath lib remained on the same 2.4.0 version.

It seems we just re-introduced snowplow/snowplow#3636 during R118 refactoring.

@chuwy chuwy added the bug Something isn't working label Sep 16, 2020
@benjben benjben self-assigned this Sep 17, 2020
@benjben
Copy link
Contributor

benjben commented Sep 17, 2020

It happened when we replaced json4s with circe : 4d0b947#diff-991a6b97fb9371add7ff0ace34b8c91fR298-R300

@chuwy chuwy closed this as completed in 8d47268 Oct 21, 2020
@chuwy chuwy added this to the 1.4.0 milestone Oct 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants