[Fleet] Give permissions to indices with rolling dates

[afgomez]

Summary

Some of the integrations add a suffix with the date to the indices they need to write in, like:

logs-osquery_manager.result-default-2021.06.23

The new permissions code doesn't account for this. This PR adds a * as a suffix to each entry in the agent policy permissions block to ensure indices with suffixes are possible.

[elasticmachine]

Pinging @elastic/fleet (Feature:Fleet)

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[jen-huang]

@ruflin What do you think about this? It feels a bit odd to me, but I don't think we have any flags today on the integration that can tell us whether we expect the final data streams to be time series or not.

[ruflin]

We want to get away from the * as otherwise we are back to the same problem that any indices could be created. So I don't think this is the right solution. Are these indices or data streams?

@aleksmaus Why are the above indices / data streams created and not the data stream naming scheme is followed? This also has implications on the number of shards etc.

[ruflin]

@afgomez If we need a short term fix here, I rather have osquery as a special case instead of applying the * to all namespaces. But as mentioned before, I think we have a more fundamental issue here with osquerybeat.

[afgomez]

I rather have osquery as a special case instead of applying the * to all namespaces

@ruflin That was my initial idea as well, but when I checked with @aleksmaus we couldn't figure out if this happened with other integrations as well.

If we know for sure this is the only one, then I can add a special case for it.

[ruflin]

Do you know if the above are indices or data streams? Everything MUST use data streams and the data stream naming scheme. But seems like osquerybeat does not so if other integrations do the same, we have a larger issue that we need to investigate quickly.

[afgomez]

Do you know if the above are indices or data streams?

Looking at https://epr.elastic.co/package/osquery_manager/0.2.3/ I'd say it's a data stream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 95c7406
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💚 Build #134142 succeeded 9caf75e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @afgomez

[nchaulet]

🚀

[ruflin]

Please do not merge. Indices should not have rolling dates, we should fix this on the osquerybeat side.

[jen-huang]

Going to close this in favor of elastic/beats#26545.