Add ability to configure RewritePolicy that can rewrite any fields

[rudolf]

When closing #57547 it was assumed we would only ever want to rewrite the LogMeta fields. However, there's just as much reason to want to rewrite other fields like message and error.

MetaRewritePolicy has a very strict schema and defensive implementation, i.e. it focusses only on meta fields and you cannot set a new field or update a field that doesn't exist.

I wonder if we should not expose much more power through the RewritePolicy so that users can rewrite any field. Taking inspiration from ingest and filebeat processors we'd want to be able to set or remove a field if an array of conditions are true (AND). contains conditions would test if a given field contains the provided string (case insensitive). A match condition would match the given field against the provided regexp.

Instead of introducing new policies for every field like MessageRewritePolicy I wonder if we should not introduce a new GenericRewritePolicy and default rewritePolicy.type to 'generic'.

As a first step we should design a config schema that's flexible/powerful enough while being very simple.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[pgayvallet]

I wonder if we should not expose much more power through the RewritePolicy so that users can rewrite any field. Taking inspiration from ingest and filebeat processors we'd want to be able to set or remove a field if an array of conditions are true (AND)

My main concern here is, wouldn't we be just reimplementing (a part of, at least) filebeat within Kibana doing so? I understand why we're opening the discussion around this feature, but I can't avoid thinking we might be looking at reinventing the wheel here?

Customers ingesting Kibana logs (in ES or elsewhere) will likely always have an integration to do so. And I assume all of those log ingestion integration will have those features of data preprocessing that could be used, right?

[rudolf]

wouldn't we be just reimplementing (a part of, at least) filebeat within Kibana doing so

++