Upcoming Transition to Node.js v18: Plugin Compatibility Test Required Overall Issue #4058

ananzh · 2023-05-17T18:33:44Z

Introduction

Based on the announcement of Node.js , here is a list of end of security support for different node.js package.

Node.js 14 (LTS): Security Support Ends in 1 month on 30 Apr 2023
Node.js 16 (LTS): Security Support Ends in 6 months on 11 Sep 2023
Node.js 18 (LTS): Security Support Ends in 2 years on 30 Apr 2025
Node.js 19 (latest version but no LTS): Security Support Ends in 3 months on 01 Jun 2023

As we strive to ensure the highest quality and performance for OpenSearch Dashboards, we are planning to upgrade from Node.js v14 to Node.js v18. This major version bump comes with numerous benefits including performance improvements, new features, and enhanced security. However, we understand that such changes may also impact the existing plugins built on the OpenSearch Dashboards platform.

Therefore, we kindly request all plugin owners to test the compatibility of their plugins with the new Node.js version. A branch with Node.js v18 updates has been provided for this purpose:
https://github.com/AMoo-Miki/OpenSearch-Dashboards/tree/node-18-webpack-4

Steps to proceed

Pull the provided branch that includes the Node.js v18 upgrade and OpenSearch 3.0.0.
Hook up your plugin(s) with the updated OpenSearch Dashboards. We have opened a list of Plugin Compatibility Test issues. See the next section.
Run all your existing test suites and perform manual testing as necessary. If there is no problem, close the issue.
Report any issues or incompatibilities you encounter during your tests. Any questions or issues encountered should be documented in the respective plugin's compatibility test issue and also posted in the comments of this overall issue. The purpose of adding questions and issues here is to enable visibility and transparency among all plugin owners. One plugin's issue might be a common problem affecting other plugins as well. This shared knowledge base will promote collaborative problem-solving and prevent duplication of effort.

Plugin Compatibility Test issues

Build Team Notification

Please ensure to carry out these tests and report your findings as soon as possible. This will help us identify potential issues early and mitigate them promptly, ensuring a smooth transition for the entire community and release. We understand this requires additional efforts, but your collaboration is crucial to maintain the robustness and reliability of OpenSearch Dashboards. We appreciate your support and commitment to the project.

ananzh · 2023-05-17T20:41:43Z

Currently we have hooked up 5 plugins:

anomaly-detection-dashboards-plugin
dashboards-alerting
dashboards-reporting
security-dashboards-plugin
index-management-dashboards-plugin

bootstrap and start 5 plugins

All bootstrap and start success. Plugin pages can load successfully.

ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ nvm use 18.15.0
Now using node v18.15.0 (npm v9.5.0)
ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ yarn osd bootstrap
yarn run v1.22.19
$ scripts/use_node scripts/osd bootstrap
 info [opensearch-dashboards] running yarn

$ scripts/use_node ./preinstall_check
[1/4] Resolving packages...
warning Resolution field "typescript@4.0.2" is incompatible with requested version "typescript@~4.5.2"
success Already up-to-date.

 info [opensearch-alerting-dashboards] running yarn

[1/4] Resolving packages...
warning Resolution field "loader-utils@1.4.2" is incompatible with requested version "loader-utils@^2.0.4"
success Already up-to-date.

 info [anomaly-detection-dashboards] running yarn

[1/4] Resolving packages...
success Already up-to-date.

 info [reports-dashboards] running yarn

[1/4] Resolving packages...
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^4.1.0"
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^2.0.0"
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^2.0.0"
warning Resolution field "json-schema@0.4.0" is incompatible with requested version "json-schema@0.2.3"
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^3.0.0"
success Already up-to-date.
$ node ./scripts/postinstall.js
Modified files for html2canvas Safari support:
 
Downloaded eng.traineddata.gz for tesseract.js

 info [opensearch_index_management_dashboards] running yarn

[1/4] Resolving packages...
success Already up-to-date.

 info [opensearch-security-dashboards] running yarn

[1/4] Resolving packages...
success Already up-to-date.

 succ yarn.lock analysis completed without any issues
 warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
 warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
 warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
 warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
 warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
 succ 24 bootstrap builds are cached
Done in 4.91s.

ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ yarn start
yarn run v1.22.19
$ scripts/use_node scripts/opensearch_dashboards --dev
  log   [20:28:08.067] [info][plugins-service] Plugin "dataSourceManagement" has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]
  log   [20:28:08.072] [info][plugins-service] Plugin "dataSource" is disabled.
  log   [20:28:08.073] [info][plugins-service] Plugin "visTypeXy" is disabled.
  log   [20:28:08.155] [info][plugins-service] Plugin initialization disabled.
  log   [20:28:08.165] [warning][savedobjects-service] Skipping Saved Object migrations on startup. Note: Individual documents will still be migrated when read or written.
np bld    log   [20:28:09.336] [info][@osd/optimizer] initialized, 53 bundles cached
np bld    log   [20:28:09.336] [warning][@osd/optimizer] only building [v7dark,v7light] themes, customize with the OSD_OPTIMIZER_THEMES environment variable
np bld    log   [20:28:09.336] [success][@osd/optimizer] all bundles cached, success after 0.6 sec
 watching for changes  (2364 files)
  log   [20:28:09.707] [info][server] basepath proxy server running at http://localhost:5601/pic
server    log   [20:28:12.449] [info][plugins-service] Plugin "dataSourceManagement" has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]
server    log   [20:28:12.452] [info][plugins-service] Plugin "dataSource" is disabled.
server    log   [20:28:12.453] [info][plugins-service] Plugin "visTypeXy" is disabled.
server    log   [20:28:12.565] [info][plugins-system] Setting up [42] plugins: [alertingDashboards,usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,indexManagementDashboards,reportsDashboards,anomalyDetectionDashboards,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,visBuilder,tileMap,regionMap,inputControlVis,visualize,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,securityDashboards,bfetch]
server    log   [20:28:12.780] [info][savedobjects-service] Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations...
server    log   [20:28:12.851] [info][savedobjects-service] Starting saved objects migrations
server    log   [20:28:12.952] [info][plugins-system] Starting [42] plugins: [alertingDashboards,usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,indexManagementDashboards,reportsDashboards,anomalyDetectionDashboards,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,visBuilder,tileMap,regionMap,inputControlVis,visualize,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,securityDashboards,bfetch]

all 5 plugins are installed in OpenSearch Dashboards

dashboards-reporting page

index-management-dashboards-plugin

security-dashboards-plugin

dashboards-alerting

anomaly-detection-dashboards-plugin

amitgalitz · 2023-05-23T16:47:36Z

Is it expected that we can't run yarn start on macos with node 18.16.0?

  throw new Error('No native build was found for ' + target + '\n    loaded from: ' + dir + '\n')
        ^
Error: No native build was found for platform=darwin arch=x64 runtime=node abi=108 uv=1 libc=glibc node=18.16.0
    loaded from: /Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/lmdb-store

    at Function.load.path (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/node-gyp-build/index.js:59:9)
    at load (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/node-gyp-build/index.js:21:30)
    at Object.<anonymous> (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/lmdb-store/index.js:6:49)
    at Module._compile (node:internal/modules/cjs/loader:1254:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Function.Module._load (node:internal/modules/cjs/loader:958:12)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at Module.Hook._require.Module.require (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/require-in-the-middle/index.js:80:39)
    at Module.Hook._require.Module.require (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/require-in-the-middle/index.js:80:39)
    ```

ananzh · 2023-05-23T22:06:44Z

@amitgalitz The error message indicates that the lmdb-store module is looking for a prebuilt binary compatible with your current environment but is unable to find one. The environment is identified by a combination of platform (darwin), architecture (x64), Node.js runtime version (node=18.16.0), and the libc version (glibc). However we are not using lmdb-store any more

ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ yarn why lmdb-store
yarn why v1.22.19
[1/4] Why do we have the module "lmdb-store"...?
[2/4] Initialising dependency graph...
warning Resolution field "typescript@4.0.2" is incompatible with requested version "typescript@~4.5.2"
[3/4] Finding dependency...
error We couldn't find a match!
Done in 0.85s.

We updated to lmdb. Seems your node_modules is not cleaned and it fetched old modules. I have verified using Mac

Could you pls run yarn osd clean first? Make sure use 18.16.0 and checkout Miki's PR. After bootstrap, double check in node_modules folder that lmdb is there, not lmdb-store.

joshuali925 · 2023-05-23T23:38:54Z

the portal to save settings in advanced settings doesn't seem to work, had to scroll to bottom to see it. not sure if it's intentional

ananzh · 2023-05-24T17:11:50Z

@joshuali925 could you be more specific about this issue?

I think you could either search experimental visualizations

Or you will need to scroll down. Any differences from usage perspective?

joshuali925 · 2023-05-24T20:19:41Z

Not the actual setting, i'm talking about the black banner appended to the bottom of the screen by react portal. It shows 1 unsaved setting, Cancel changes, Save changes. Now user have to scroll down to see the banner.

i tried again using miki's branch https://github.com/AMoo-Miki/OpenSearch-Dashboards/tree/node-18-webpack-4 and saw the same issue. I do have observability OSD plugin main branch installed, didn't check if that affects it.

ananzh · 2023-05-25T17:27:00Z

@joshuali925 The issue is caused by observability plugin modifying global style classes. This is not related to Node.js 18.

https://github.com/opensearch-project/dashboards-observability/blob/main/public/components/event_analytics/explorer/explorer.scss#L7

https://github.com/opensearch-project/dashboards-observability/blob/main/public/components/metrics/index.scss#L7

This opensearch-dashboards-body is used to wrap the entire body of application. Opened an issue: opensearch-project/dashboards-observability#485

SuZhou-Joe · 2023-05-26T06:36:01Z

Hi, dashboards-notification repo encounters an issue that with Node 18, the workflow failed for 5601 port do not respond: opensearch-project/dashboards-notifications#49, while with normal main branch, the workflow passes: opensearch-project/dashboards-notifications#50

ananzh · 2023-05-30T23:46:46Z

This issued is caused Node.js 18, DNS puts IPv6 addresses before IPv4, as IPv6 gets widespread usage.

➜  index-management-dashboards-plugin git:(main) ✗ netstat -an | grep 5601
tcp6       0      0  ::1.5601               *.*                    LISTEN

Cypress doesn't support ipv6. Therefore, one work-around is to bind OpenSearch Dasbhoards to 0.0.0.0. So OSD will listen on all IPv4 addresses. In opensearch_dashboards.yml file, the setting is:

server.host: "0.0.0.0"

When you run workflow or cypress using node 18, should config opensearch_dashboards.yml and update the server.host.

ananzh · 2023-06-06T19:10:34Z

Node 18 has been released in 2.8
close this issue

ananzh added the NODE 18 ⚙ label May 17, 2023

ananzh self-assigned this May 17, 2023

github-actions bot added the untriaged label May 17, 2023

ananzh removed the untriaged label May 17, 2023

This was referenced May 22, 2023

[Proposal] Support AL2023/NodeJS18 for OpenSearch/Dashboards Releases opensearch-project/opensearch-build#1563

Closed

[RELEASE] Release version 2.8.0 opensearch-project/opensearch-build#3434

Closed

ananzh closed this as completed Jun 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upcoming Transition to Node.js v18: Plugin Compatibility Test Required Overall Issue #4058

Upcoming Transition to Node.js v18: Plugin Compatibility Test Required Overall Issue #4058

ananzh commented May 17, 2023 •

edited by kavilla

Loading

ananzh commented May 17, 2023

amitgalitz commented May 23, 2023

ananzh commented May 23, 2023 •

edited

Loading

joshuali925 commented May 23, 2023

ananzh commented May 24, 2023

joshuali925 commented May 24, 2023

ananzh commented May 25, 2023

SuZhou-Joe commented May 26, 2023

ananzh commented May 30, 2023

ananzh commented Jun 6, 2023

Upcoming Transition to Node.js v18: Plugin Compatibility Test Required Overall Issue #4058

Upcoming Transition to Node.js v18: Plugin Compatibility Test Required Overall Issue #4058

Comments

ananzh commented May 17, 2023 • edited by kavilla Loading

Introduction

Steps to proceed

Plugin Compatibility Test issues

Build Team Notification

ananzh commented May 17, 2023

bootstrap and start 5 plugins

all 5 plugins are installed in OpenSearch Dashboards

dashboards-reporting page

index-management-dashboards-plugin

security-dashboards-plugin

dashboards-alerting

anomaly-detection-dashboards-plugin

amitgalitz commented May 23, 2023

ananzh commented May 23, 2023 • edited Loading

joshuali925 commented May 23, 2023

ananzh commented May 24, 2023

joshuali925 commented May 24, 2023

ananzh commented May 25, 2023

SuZhou-Joe commented May 26, 2023

ananzh commented May 30, 2023

ananzh commented Jun 6, 2023

ananzh commented May 17, 2023 •

edited by kavilla

Loading

ananzh commented May 23, 2023 •

edited

Loading