Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[event log] query should be over all version indices, not just the current version indices #81274

Closed
pmuellr opened this issue Oct 21, 2020 · 4 comments · Fixed by #81920
Closed
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Feature:EventLog Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@pmuellr
Copy link
Member

pmuellr commented Oct 21, 2020

In the code below, we are querying the event log using the alias we create to write event docs to the indices:

return await this.esContext.esAdapter.queryEventsBySavedObject(
this.esContext.esNames.alias,
namespace,
type,
id,
findOptions
);

That alias name - and other es-related names - are generated here:

export function getEsNames(baseName: string): EsNames {
const eventLogName = `${baseName}${EVENT_LOG_NAME_SUFFIX}`;
const eventLogNameWithVersion = `${eventLogName}${EVENT_LOG_VERSION_SUFFIX}`;
const eventLogPolicyName = `${
baseName.startsWith('.') ? baseName.substring(1) : baseName
}${EVENT_LOG_NAME_SUFFIX}-policy`;
return {
base: baseName,
alias: eventLogNameWithVersion,
ilmPolicy: `${eventLogPolicyName}`,
indexPattern: `${eventLogName}-*`,
indexPatternWithVersion: `${eventLogNameWithVersion}-*`,
initialIndex: `${eventLogNameWithVersion}-000001`,
indexTemplate: `${eventLogNameWithVersion}-template`,
};
}

For v7.10.0, the alias name will be .kibana-event-log-7.10.0. This will limit searches to only the events generated by the current version of Kibana. We should be able to search older versions as well - the mappings have not changed significantly since the beginnings. Clearly we need some thoughts about the future where the mappings could change in incompatible ways, and consider what happens when the event log becomes a datastream.

For now, it seems like we should use EsNames.indexPattern, which would be set to the string .kibana-event-log-*, for these queries.

@pmuellr pmuellr added bug Fixes for quality problems that affect the customer experience Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:EventLog labels Oct 21, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

@mikecote
Copy link
Contributor

For now, it seems like we should use EsNames.indexPattern, which would be set to the string .kibana-event-log-*, for these queries.

++ the alias I believe is mostly used to write event logs to the most recent index (determined by ILM) but index pattern should be used for queries everywhere.

@pmuellr
Copy link
Member Author

pmuellr commented Oct 21, 2020

Besides a jest test (hoping an existing one will need a change), was thinking about a functional one.

I think we can arrange for FTR to create an index that matches the pattern; say .kibana-event-log-${current-iso-date}, and write a "compatible" doc into it - recent @timestamp, message with a unique value in it (again, probably date), a saved object reference (perhaps for a preconfigured action?). Then do a find to see if it gets found.

@mikecote
Copy link
Contributor

The ES archiver may be able to help here but it's been a while so I may be wrong. It would dump data into indices named whatever you want.

@mikecote mikecote mentioned this issue Oct 27, 2020
36 tasks
@gmmorris gmmorris self-assigned this Oct 28, 2020
@kobelb kobelb added the needs-team Issues missing a team label label Jan 31, 2022
@botelastic botelastic bot removed the needs-team Issues missing a team label label Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:EventLog Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants