[Auditbeat] System module 6.6

[cwurm]

This is the meta issue for the release of the first version of the Auditbeat system module.

Further tasks are tracked in the backlog issue.

General

• Implement host.id for darwin (done: Implement UniqueID on Darwin go-sysinfo#31)
• Populate message (done: [Auditbeat] Add message field to system module #9483)
• Add new fields to fields.ecs.yml (Add Auditbeat system module fields to fields.ecs.yml #9318)
• Templates combining fields from both auditbeat and x-pack/auditbeat (done: Add CI testing to x-pack/auditbeat #9362)
• Merge feature branch feature-auditbeat-host (done: [Auditbeat] Add system module #9546)
• Rename "metricset" to "dataset" everywhere (config done: [Auditbeat] Change module configuration from "metricsets" to "datasets" #10018, docs done: [Auditbeat] Rename "metricset" to "dataset" in docs #10101)

1. Host

• Initial implementation (done: [Auditbeat Host] Add host, packages, and processes metricsets #8436)
• Implement change detection (done: [Auditbeat] Update host metricset #9421)
• State reporting (done: [Auditbeat] Update host metricset #9421)
• Documentation (done: [Auditbeat] System module documentation #9512)

2. Process

• Initial implementation (done: [Auditbeat Host] Add host, packages, and processes metricsets #8436)
• Rename to process, implement scheduled state reporting, and change to single documents (merged: [Auditbeat] Update process metricset #9139)
• Documentation (done: [Auditbeat] System module documentation #9512)
• Resilient data collection (done: [Auditbeat] Report process errors #9693)
• Fix on Windows ([Auditbeat] Failing x-pack/auditbeat tests on Windows #9748) (done: [Auditbeat] Process metricset: Skip permission errors on Windows #9863)

3. Socket

• Implement metricset incl. state reporting (done: [Auditbeat] Socket metricset #8834)
• Documentation (done: [Auditbeat] System module documentation #9512)

4. User

• Implement metricset incl. state reporting (done: [Auditbeat] Add user metricset #8835)
• Make reading /etc/shadow opt-in, and do multiple rounds of SHA-512 hashing (done: [Auditbeat] Opt-in to detecting password changes #9461)
• Documentation (done: [Auditbeat] System module documentation #9512)
• Fix collection of group data ([Auditbeat] Flaky user test #9679) (done: [Auditbeat] User metricset: Fetch groups by user #9732)

Main PRs (no longer maintained)

#8356 (MERGED - Rename sysinfo module to system)
#8436 (MERGED - Add host, packages, and processes metricsets)
#8835 (MERGED - Add user metricset)
#8834 (MERGED - Socket metricset)
#9139 (MERGED - Update process metricset)
#9362 (MERGED - Add CI testing)
#9421 (MERGED - Update host metricset)
#9461 (MERGED - Opt-in to detecting password changes)
#9483 (MERGED - Add message field)
#9512 (MERGED - System module documentation)
#9546 (MERGED - Add system module)

[tsg]

Pinging @elastic/secops (mostly testing).

[cwurm]

fyi, I made a bunch of updates above to reflect the current state

[cwurm]

updated and added a backlog

[cwurm]

Updated to include only what will be in 6.6. Added new issue for 6.7 / 7.0.

[cwurm]

The System module was released with four datasets in 6.6. Closing.

Next releases 6.7 and 7.0 are tracked in #10103.

Backlog in #9344.