Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Types key inside the json parser #1563

Closed
JGiola opened this issue Sep 11, 2019 · 2 comments
Closed

Support Types key inside the json parser #1563

JGiola opened this issue Sep 11, 2019 · 2 comments

Comments

@JGiola
Copy link

JGiola commented Sep 11, 2019

Is your feature request related to a problem? Please describe.

A common configuration shared by all the parsers is the Types configuration that instruct it on the types of certain fields. This was originally intended to use in conjunction with the regex parser to cast fields from the string type.

This knowledge is totally missing in the documentation, and the user is led to believe that can be used on every parser to force types on the parsed data even with the json parser.
This is not true if you search in the code for the flb_parser_typecast function is called inside all the concrete implementation of the parser except in src/flb_parser_json.c.

Describe the solution you'd like
A nice solution is to add the same typecast function in the json parser code to share the same function across all the parsers.
This can be useful for cast types from different json log to common types (the more common, cast to string a timestamp saved as a number to pass it to strptime)

Describe alternatives you've considered
Clearly states in the documentation that this is working as intended and it can be used only with the other parser (like the Decode_Fieldkey that on the contrary work only on json parsers).

Additional context
I was trying to cast some values inside a json log but they where never picked up. I've gained some frustration in exchange of time to find why.

@nokute78
Copy link
Collaborator

Types is an option for the formats which don't support data types. e.g. ltsv.
On the other hand, JSON can represent data types. (integers, floats, strings, booleans.)
So, JSON parser casts them automatically and it doesn't support types option.

@nokute78
Copy link
Collaborator

@JGiola

Clearly states in the documentation that this is working as intended and it can be used only with the other parser (like the Decode_Fieldkey that on the contrary work only on json parsers).

Sorry for undocumented it.
I sent a patch for document to make clear which parser supports Types option.

fluent/fluent-bit-docs#211

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants