[libbeat] Support custom analyzers in fields.yml

[andrewkroh]

Describe the enhancement:

Fields.yml fields currently support using the built-in analyzers on text fields. But it's not possible to declare a custom analyzer in fields.yml and have it installed as part of the index settings in the index template.

Describe a specific use case for the enhancement or feature:

Using a custom analyzer or search_analzyer for text fields in fields.yml. This will give users a better experience when searching data indexed by Beats.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

Open questions:

• Where in fields.yml should analyzer be allowed?
• What is the merge behavior when multiple analyzer blocks are declared in separate files (e.g. error out on analyzer name collisions)? For example, imagine both the zeek module and suricata module declaring a custom_url_analzyer, what happens?

Examples:

- key: powershell
  title: PowerShell module
  description: >
    These are the event fields specific to the module for the Microsoft-Windows-PowerShell/Operational and Windows PowerShell logs.
  release: beta
  analyzer:
    powershell_script_analyzer:
      type: pattern
      pattern: "[\\W&&[^-]]+"
  fields:
    ...

[efd6]

To address the issue of name collisions, I think that collisions should error out and that packages that define a single location analyzer should namespace the analyzer by their package, essentially as we have here. For the example, suricata_url_analyzer and zeek_url_analyzer.

The docs should probably also advertise any custom analyzer so that users can use them for extending.