Beats: Add default_field: true to fieldsets (#1633)

CHANGELOG.next.md

@@ -45,6 +45,7 @@ Thanks, you're awesome :-) -->
 #### Improvements
 
 * Remove remaining Go deps after removing Go code generator. #1585
+* Add explicit `default_field: true` for Beats artifacts. #1633
 
 #### Deprecated
 

experimental/generated/beats/fields.ecs.yml

@@ -63,6 +63,7 @@
       not change if data is sent through queuing systems like Kafka, Redis, or processing
       systems such as Logstash or APM Server.'
     type: group
+    default_field: true
     fields:
     - name: build.original
       level: core
@@ -127,6 +128,7 @@
       behalf of a single administrative entity or domain that presents a common, clearly
       defined routing policy to the internet.
     type: group
+    default_field: true
     fields:
     - name: number
       level: extended
@@ -163,6 +165,7 @@
       in that category, you should still ensure that source and destination are filled
       appropriately.'
     type: group
+    default_field: true
     fields:
     - name: address
       level: extended
@@ -438,6 +441,7 @@
       in the cloud, the field contains cloud data from the machine the service is
       running on.'
     type: group
+    default_field: true
     fields:
     - name: account.id
       level: extended
@@ -528,6 +532,7 @@
     group: 2
     description: These fields contain information about binary code signatures.
     type: group
+    default_field: true
     fields:
     - name: digest_algorithm
       level: extended
@@ -615,6 +620,7 @@
 
       These fields help correlate data based containers from any runtime.'
     type: group
+    default_field: true
     fields:
     - name: cpu.usage
       level: extended
@@ -703,6 +709,7 @@
       names cannot include `\`, `/`, `*`, `?`, `"`, `<`, `>`, `|`, ` ` (space character),
       `,`, or `#`. Please see the Elasticsearch reference for additional https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params[restrictions].'
     type: group
+    default_field: true
     fields:
     - name: dataset
       level: extended
@@ -750,6 +757,7 @@
       transaction. If the event also contains identification of the client and server
       roles, then the client and server fields should also be populated.'
     type: group
+    default_field: true
     fields:
     - name: address
       level: extended
@@ -1029,6 +1037,7 @@
 
       * Dynamic library (`.dylib`) commonly used on macOS'
     type: group
+    default_field: true
     fields:
     - name: code_signature.digest_algorithm
       level: extended
@@ -1427,6 +1436,7 @@
       query details as well as all of the answers that were provided for this query
       (`dns.type:answer`).'
     type: group
+    default_field: true
     fields:
     - name: answers
       level: extended
@@ -1591,6 +1601,7 @@
     group: 2
     description: Meta-information specific to ECS.
     type: group
+    default_field: true
     fields:
     - name: version
       level: core
@@ -1609,6 +1620,7 @@
     group: 2
     description: These fields contain Linux Executable Linkable Format (ELF) metadata.
     type: group
+    default_field: true
     fields:
     - name: architecture
       level: extended
@@ -1798,6 +1810,7 @@
       protocols that send and receive email messages such as SMTP are outside the
       scope of the `email.*` fields.'
     type: group
+    default_field: true
     fields:
     - name: attachments
       level: extended
@@ -1967,6 +1980,7 @@
       Use them for errors that happen while fetching events or in cases where the
       event itself contains an error.'
     type: group
+    default_field: true
     fields:
     - name: code
       level: core
@@ -2012,6 +2026,7 @@
       temperature. See the `event.kind` definition in this section for additional
       details about metric and state events.'
     type: group
+    default_field: true
     fields:
     - name: action
       level: core
@@ -2334,6 +2349,7 @@
       services). File fields provide details about the affected file associated with
       the event or metric.'
     type: group
+    default_field: true
     fields:
     - name: accessed
       level: extended
@@ -3231,6 +3247,7 @@
       This geolocation information can be derived from techniques such as Geo IP,
       or be user-supplied.'
     type: group
+    default_field: true
     fields:
     - name: city_name
       level: core
@@ -3315,6 +3332,7 @@
     description: The group fields are meant to represent groups that are relevant
       to the event.
     type: group
+    default_field: true
     fields:
     - name: domain
       level: extended
@@ -3347,6 +3365,7 @@
       a range of generic bytes. Entity-specific hashes such as ja3 or imphash are
       placed in the fieldsets to which they relate (tls and pe, respectively).'
     type: group
+    default_field: true
     fields:
     - name: md5
       level: extended
@@ -3383,6 +3402,7 @@
       event happened, or from which the measurement was taken. Host types include
       hardware, virtual machines, Docker containers, and Kubernetes nodes.'
     type: group
+    default_field: true
     fields:
     - name: architecture
       level: core
@@ -3645,6 +3665,7 @@
     description: Fields related to HTTP activity. Use the `url` field set to store
       the url of the request.
     type: group
+    default_field: true
     fields:
     - name: request.body.bytes
       level: extended
@@ -3759,6 +3780,7 @@
       a single observer interface (e.g. network sensor on a span port) only the observer.ingress
       information should be populated.
     type: group
+    default_field: true
     fields:
     - name: alias
       level: extended
@@ -3795,6 +3817,7 @@
       The details specific to your event source are typically not logged under `log.*`,
       but rather in `event.*` or in other ECS fields.'
     type: group
+    default_field: true
     fields:
     - name: file.path
       level: extended
@@ -3906,6 +3929,7 @@
       The network.* fields should be populated with details about the network activity
       associated with an event.'
     type: group
+    default_field: true
     fields:
     - name: application
       level: extended
@@ -4065,6 +4089,7 @@
       or metric. Message queues and ETL components used in processing events or metrics
       are not considered observers in ECS.'
     type: group
+    default_field: true
     fields:
     - name: egress
       level: extended
@@ -4378,6 +4403,7 @@
     description: Fields that describe the resources which container orchestrators
       manage or act upon.
     type: group
+    default_field: true
     fields:
     - name: api_version
       level: extended
@@ -4449,6 +4475,7 @@
       These fields help you arrange or filter data stored in an index by one or multiple
       organizations.'
     type: group
+    default_field: true
     fields:
     - name: id
       level: extended
@@ -4469,6 +4496,7 @@
     group: 2
     description: The OS fields contain information about the operating system.
     type: group
+    default_field: true
     fields:
     - name: family
       level: extended
@@ -4536,6 +4564,7 @@
       It contains general information about a package, such as name, version or size.
       It also contains installation details, such as time or location.
     type: group
+    default_field: true
     fields:
     - name: architecture
       level: extended
@@ -4632,6 +4661,7 @@
     group: 2
     description: These fields contain Windows Portable Executable (PE) metadata.
     type: group
+    default_field: true
     fields:
     - name: architecture
       level: extended
@@ -4905,6 +4935,7 @@
       from a log message.  The `process.pid` often stays in the metric itself and
       is copied to the global field for correlation.'
     type: group
+    default_field: true
     fields:
     - name: args
       level: extended
@@ -7680,6 +7711,7 @@
     group: 2
     description: Fields related to Windows Registry operations.
     type: group
+    default_field: true
     fields:
     - name: data.bytes
       level: extended
@@ -7755,6 +7787,7 @@
       to `related.ip`, you can then search for a given IP trivially, no matter where
       it appeared, by querying `related.ip:192.0.2.15`.'
     type: group
+    default_field: true
     fields:
     - name: hash
       level: extended
@@ -7792,6 +7825,7 @@
       application firewalls, url filters, endpoint detection and response (EDR) systems,
       etc.'
     type: group
+    default_field: true
     fields:
     - name: author
       level: extended
@@ -7894,6 +7928,7 @@
       in that category, you should still ensure that source and destination are filled
       appropriately.'
     type: group
+    default_field: true
     fields:
     - name: address
       level: extended
@@ -8166,6 +8201,7 @@
 
       These fields help you find and correlate logs for a specific service and version.'
     type: group
+    default_field: true
     fields:
     - name: address
       level: extended
@@ -8279,6 +8315,7 @@
       transaction. If the event also contains identification of the client and server
       roles, then the client and server fields should also be populated.'
     type: group
+    default_field: true
     fields:
     - name: address
       level: extended
@@ -8554,6 +8591,7 @@
       \ which kind of approach is used by this detected threat, to accomplish the\
       \ goal (e.g. \"endpoint denial of service\")."
     type: group
+    default_field: true
     fields:
     - name: enrichments
       level: extended
@@ -11774,6 +11812,7 @@
       protocol itself and intentionally avoids in-depth analysis of the related x.509
       certificate files.
     type: group
+    default_field: true
     fields:
     - name: cipher
       level: extended
@@ -12384,6 +12423,7 @@
     description: URL fields provide support for complete or partial URLs, and supports
       the breaking down into scheme, domain, path, and so on.
     type: group
+    default_field: true
     fields:
     - name: domain
       level: extended
@@ -12531,6 +12571,7 @@
       Fields can have one entry or multiple entries. If a user has more than one id,
       provide an array that includes all of them.'
     type: group
+    default_field: true
     fields:
     - name: changes.domain
       level: extended
@@ -12844,6 +12885,7 @@
 
       They often show up in web service logs coming from the parsed user agent string.'
     type: group
+    default_field: true
     fields:
     - name: device.name
       level: extended
@@ -12952,6 +12994,7 @@
       specific information when observer events contain discrete ingress and egress
       VLAN information, typically provided by firewalls, routers, or load balancers.'
     type: group
+    default_field: true
     fields:
     - name: id
       level: extended
@@ -12973,6 +13016,7 @@
     description: The vulnerability fields describe information about a vulnerability
       that is relevant to an event.
     type: group
+    default_field: true
     fields:
     - name: category
       level: extended
@@ -13111,6 +13155,7 @@
       use the x509 fields under the relevant TLS fields: `tls.server.x509` and/or
       `tls.client.x509`.'
     type: group
+    default_field: true
     fields:
     - name: alternative_names
       level: extended