[BUG] exception: Reason: java.io.OptionalDataException

[StefanSa]

Describe the bug
All this time v2 ran without a problem.
As of today, opensearch-dashboard can no longer be started.

error message:

2022-06-03T15:47:25.866695+02:00 os-dash01 opensearch-dashboards[1388]: {"type":"log","@timestamp":"2022-06-03T13:47:25Z","tags":["error","opensearch","data"],"pid":1388,"message":"[exception]: java.io.OptionalDataException"}
2022-06-03T15:47:25.869685+02:00 os-dash01 opensearch-dashboards[1388]: {"type":"error","@timestamp":"2022-06-03T13:47:25Z","tags":[],"pid":1388,"level":"error","error":{"message":"java.io.OptionalDataException: exception: [exception] Reason: java.io.OptionalDataException","name":"ResponseError","stack":"ResponseError: java.io.OptionalDataException: exception: [exception] Reason: java.io.OptionalDataException\n    at onBody (/usr/share/opensearch-dashboards/node_modules/@opensearch-project/opensearch/lib/Transport.js:374:23)\n    at IncomingMessage.onEnd (/usr/share/opensearch-dashboards/node_modules/@opensearch-project/opensearch/lib/Transport.js:293:11)\n    at IncomingMessage.emit (events.js:412:35)\n    at endReadableNT (internal/streams/readable.js:1334:12)\n    at processTicksAndRejections (internal/process/task_queues.js:82:21)"},"url":"https://os-dash01.feltengroup.local:5601/bootstrap.js","message":"java.io.OptionalDataException: exception: [exception] Reason: java.io.OptionalDataException"}
2022-06-03T15:47:25.884527+02:00 os-dash01 opensearch-dashboards[1388]: {"type":"log","@timestamp":"2022-06-03T13:47:25Z","tags":["error","opensearch","data"],"pid":1388,"message":"[exception]: java.io.OptionalDataException"}
2022-06-03T15:47:25.887033+02:00 os-dash01 opensearch-dashboards[1388]: {"type":"error","@timestamp":"2022-06-03T13:47:25Z","tags":[],"pid":1388,"level":"error","error":{"message":"java.io.OptionalDataException: exception: [exception] Reason: java.io.OptionalDataException","name":"ResponseError","stack":"ResponseError: java.io.OptionalDataException: exception: [exception] Reason: java.io.OptionalDataException\n    at onBody (/usr/share/opensearch-dashboards/node_modules/@opensearch-project/opensearch/lib/Transport.js:374:23)\n    at IncomingMessage.onEnd (/usr/share/opensearch-dashboards/node_modules/@opensearch-project/opensearch/lib/Transport.js:293:11)\n    at IncomingMessage.emit (events.js:412:35)\n    at endReadableNT (internal/streams/readable.js:1334:12)\n    at processTicksAndRejections (internal/process/task_queues.js:82:21)"},"url":"https://os-dash01.feltengroup.local:5601/bootstrap.js","message":"java.io.OptionalDataException: exception: [exception] Reason: java.io.OptionalDataException"}

To Reproduce
systemctl start dashboards

OpenSearch Version
v2.0

Dashboards Version
v2.0

Screenshots

If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: [openLeap 15.3]

[StefanSa]

A bit strange.
After i started dashboards in the console "./bin/opensearch-dashboards" and then via systemctl, it works again.

[StefanSa]

Possibly the same problem?

opensearch-project/security#1961

[kavilla]

Hello @StefanSa, thanks for opening.

Just to clarify the issue you are seeing is intermittent?

Based on the Java exception and the logs the error should be coming from OpenSearch. It could be coming from the OpenSearch JS Client that is hooked into OpenSearch Dashboards since we upgraded that version.However, if you are seeing an intermittent failure then I agree with you and I believe it could be related to the same issue you are addressed above. If so I will re-route it to that repo.

In the meantime, would be able to provide details on how you upgrade? I do believe there's some lack of documentation on known issues when upgrading that keep a stale version of files around.

Thanks again!

[StefanSa]

hi @kavilla
Has only occurred once so far. I do the upgrade / installation with the tar file.
First back up the certificates, configuration files and data folders (master nodes).
After that i rename the old folders and unpack the tar file and restore the config / certificates etc.
So far, this upgrade strategy has worked without any problems.
The problem appeared suddenly.
However, each of the three Client Nodes was accessible via WebUI and a login was successful.
So i can't really say if it was a server or client (dashboard) problem.

[kavilla]

Hello @StefanSa, I will re-route this to OpenSearch repo as the linked issue above has the same issue and it was for 1.3.

Thank you!

[kavilla]

Issue moved to opensearch-project/OpenSearch opensearch-project/OpenSearch#3586 via ZenHub