[meta] Core Team 8.0 Projects

[joshdover]

This meta issue is to meant capture all work on the Core team that needs to be completed for the 8.0 release.

Elasticsearch

• Require scripts to be enabled in Elasticsearch Require scripts to be enabled in Elasticsearch #96106
• Migrate to new ES client [Meta] Switch to the new Elasticsearch client #83910

Saved Objects

Public HTTP API Changes

• Add deprecation docs for dashboard/import API Add deprecation docs for dashboard import/export API #102223
• [deferred] Remove dashboard import/export API Remove Dashboard Import/Export API #41439
• [deferred] Change saved objects HTTP API to follow snake_case naming convention Use snake case for saved objects import/export API #54123
• Remove legacy format for /api/status API Remove legacy /api/status endpoint response format #94419
  • Blocker: Adapt status page UI to 8.0 format Adapt status page UI to 8.0 /api/status format #77624
• [deferred] Formalize HTTP error response format Formalize HTTP error response format #57296
• Remove legacy /api/settings [Breaking] Remove xpack_legacy plugin & /api/settings route #105695

Remove support for legacy exports

We deprecated the legacy export format in 7.0 and we need to remove the UI code from master for supporting this:

• Remove support for legacy imports from the UI #103921

Migrating to System Indices

Meta Issue

All Elastic-internal indices will be converted to "system indices" in 8.0. The goal of system indices is to prevent accidental tampering/breakage/corruption of data & index settings by users when using the user-facing REST APIs in Elasticsearch.

Related Issues

• Deprecate kibana.index setting Deprecate kibana.index setting #82521
• ESArchiver + System Indices ESArchiver + System Indices == 🥰 or 😭 #83592

Removing multi-tenancy support

Overall issue

The new "Kibana ES Plugin" will only support a single Kibana installation per Elasticsearch cluster. Due to this change, we will be removing support for the kibana.index & related settings which allowed users to have multiple, separate Kibana installs on the same cluster.

Instead, we are encouraging users to utilize the Spaces functionality and we want to assist in migrating to Spaces as best we can. To this end, we plan to leverage the Saved Objects import/export features to allow users to export all of their Kibana data and then import it into a space on a shared Kibana install. However, today there are several limitations to this functionality that need to be addressed before users will be able to successfully export all of their data:

• Support exporting and importing hidden SavedObjects SavedObjects should support importing/exporting even if they're hidden from the API #82027
• Encrypted SavedObject attributes cause issues for import/export Encrypted saved-object attributes cause issues for import/export #82086
• [deferred] SavedObject references used to model a "composition" relationship do not work with exports well Saved-object references used to model a "composition" relationship #82064
  • I'm not sure this is necessarily a blocker for allowing users to migrate to Spaces

Logging

As part of the migration to the Kibana Platform, we introduced a new logger that supports many new features to bring Kibana's logging capabilities more inline with Elasticsearch's logging. Internally, virtually all log messages flow through the new system, but we are still relying on the legacy logging system to actually write to the final output. In 8.0, we'd like to remove support for the old logging system entirely, including its configuration values so that we no longer have to support this (untested) code.

Blockers to removing the legacy logging system (meta issue)

• Use Elastic Common Schema (ECS) [logging] Use Elastic Common Schema (ECS) #52226
• Migrate log rotation [KP] Implement Log rotation appender #56291
• Consume logging config via CLI Consume logging config via CLI #57548
• Provide substitution for elasticsearch.logQueries in the new logging config Provide substitution for elasticsearch.logQueries in the new logging config #57546
• Log request, response and ops data in the new platform Log HTTP requests, responses #13241
• Implement censoring mechanism in the Logging service for sensitive request data Implement filtering mechanism in the Logging service #57547
• Migrate internal logging infrastructure [8.0] Remove @kbn/legacy-logging #50660
• [core.logging] Update ECS typings to 8.0.0 #112175

Cleanup

• Documentation Add user-facing docs for new logging configuration #57502
• Unify Kibana & Elasticsearch logging config keys Unify Kibana & Elasticsearch logging config keys #57551
• Deprecate legacy logging configuration in favor of new logging system Deprecate legacy logging configuration in favour of new logging system #82431

Configuration

kibana.yml

• Expose config deprecations to Upgrade Assistant Expose detected config deprecations to plugins #60302
• [deferred] Refactor naming format of the config settings in kibana.yml file (will done by Operations team, but we will have to update out code Settings should use snake_case #7444)
• Remove deprecated server.xsrf.whitelist setting Allow disabling xsrf protection per an endpoint #58717
• Remove deprecated Core settings #103915
• [deferred] Strict domain separation in kibana.yml file [New Platform] 8.0 Refactor kibana.yml #46705
• Use 78 error code for unknown config keys instead of the 'legacy' 64 (see Kibana should use proper exit signal when stops due to a configuration error #96376 and Fix config validation when elastic.apm is defined #96502)
• Remove telemetry url configs in favor of a telemetry.sendUsageTo: staging | prod config. (See [Telemetry] Deprecate telemetry url configs #96761 and [Telemetry] Add telemetry.sendUsageTo config #107396)

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[lukeelmers]

I believe all remaining items on this list have been deferred, so I've updated the checklist accordingly. Now that we are past 8.0 FF, I'll go ahead and close this.