Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid byte sequence for encoding "UTF8": 0x00 #52

Closed
PopFlamingo opened this issue Oct 17, 2019 · 2 comments
Closed

Invalid byte sequence for encoding "UTF8": 0x00 #52

PopFlamingo opened this issue Oct 17, 2019 · 2 comments
Labels
question Further information is requested

Comments

@PopFlamingo
Copy link

Fluent attempts to save strings that contains the NULL character, it seems to be linked to JSON serialised values as this repo demonstrates:
https://github.com/adtrevor/FluentPosgresNullCharBug

The resulting error is:

ERROR:  22021: invalid byte sequence for encoding "UTF8": 0x00
LOCATION:  report_invalid_encoding, wchar.c:2017

Thank you

@tanner0101 tanner0101 added the question Further information is requested label Oct 18, 2019
@tanner0101
Copy link
Member

tanner0101 commented Oct 18, 2019

Where does the null character (0x00) appear in the JSON?

If it's appearing in a value string, that's not valid according to spec. Null characters should be represented as \u0000. For example:

{"string": "foo\u0000bar"}

String value spec from RFC 7159 pasted here:

string = quotation-mark *char quotation-mark

char = unescaped /
  escape (
      %x22 /          ; "    quotation mark  U+0022
      %x5C /          ; \    reverse solidus U+005C
      %x2F /          ; /    solidus         U+002F
      %x62 /          ; b    backspace       U+0008
      %x66 /          ; f    form feed       U+000C
      %x6E /          ; n    line feed       U+000A
      %x72 /          ; r    carriage return U+000D
      %x74 /          ; t    tab             U+0009
      %x75 4HEXDIG )  ; uXXXX                U+XXXX

escape = %x5C              ; \

quotation-mark = %x22      ; "

unescaped = %x20-21 / %x23-5B / %x5D-10FFFF

@tanner0101
Copy link
Member

It turns out the problem was attempting to submit TIMESTAMPTZ formatted data to a query expecting JSON formatted data. A failure here is expected.

However, this does bring up a possible enhancement to postgres nio:
#53

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants