Install minimum set of dependencies for dashboards-plugin integration tests

[cwperks]

The security-dashboards-plugin has been facing issues when running integration tests on Github runners because the runners are running out of free disk space: opensearch-project/security-dashboards-plugin#1611

To resolve the issue, the security-dashboards-plugin Github action for integration tests is removing pre-installed software on the runners that are not needed to run the integration tests: opensearch-project/security-dashboards-plugin#1613

OSD core faced similar issues and was resolved by disabling watermarking: #5172

Is there a better solution than either of these approaches that can address the problem of using up too much space for integration tests?

The pseudo-code of the integration tests for security-dashboards-plugin is:

1. Download latest SNAPSHOT of min distro of OpenSearch
2. Download latest SNAPSHOT zip of security plugin
3. Install security plugin
4. Run OS Node
5. Clone OpenSearch-Dashboards <- space intensive
6. yarn osd bootstrap <- space intensive
7. Checkout security-dashboards-plugin
8. build dashboards plugins
9. Run security-dashboards-plugin integ tests

Is there a way to minimize step 6? Security-dashboards-plugin added a filter during the checkout in step 5 to omit the cypress/ and tests/ directories. The cypress directory is over 650MB in this repo.

After running yarn osd bootstrap, the node_modules directory is 1.4GB.

I tried running NODE_ENVIRONMENT=production yarn osd bootstrap to only install prod dependencies, but this command fails with error TS2688: Cannot find type definition file for 'node'..

[cwperks]

@opensearch-project/engineering-effectiveness could you also advise on this issue?

[rishabh6788]

I have a limited understanding of the dashboards build process but is there a way we can avoid step 5 and 6, download last successful built dashboards min snapshot (if there is any) and use that to install security-dashboards-plugin in that and run the integration tests?

[cwperks]

If that were available that would be very helpful for dashboards-plugins. They are currently not being published to artifacts.opensearch.org so I filed an issue in this repo: #5336