Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

Add /tags/terms query to get counts of tag values #1582

Merged
merged 4 commits into from
Jan 20, 2020

Conversation

shanson7
Copy link
Collaborator

If the interface / behavior is approved, I'll update the HTTP docs.

Fix #1573

@shanson7
Copy link
Collaborator Author

Example:

curl -s 'http://localhost:6060/tags/terms?expr=namespace=ossh&tags=name'
{
  "totalSeries": 15387,
  "terms": {
    "name": {
      "disk.average.latency.g": 925,
      "disk.average.service.time.g": 927,
      "disk.io.current.g": 961,
      "disk.io.time.g": 875,
      "disk.io.time.weighted.g": 928,
      "disk.reads.completed.g": 921,
      "disk.reads.merged.g": 946,
      "disk.reads.time.g": 985,
      "disk.sectors.read.g": 963,
      "disk.sectors.written.g": 921,
      "disk.writes.completed.g": 970,
      "disk.writes.merged.g": 924,
      "disk.writes.time.g": 953,
      "interrupt.g": 553,
      "network.softnet.cpu.collisions.g": 848,
      "network.softnet.cpu.packets.dropped.g": 899,
      "network.softnet.cpu.time.squeeze.g": 888
    }
  }
}

Copy link
Contributor

@Dieterbe Dieterbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks pretty good, though i didn't go to deep in the index. i'll defer to mauro or robert on that.

idx/idx.go Outdated Show resolved Hide resolved
// is kept for each value of the requested tags.
// The series are not deduplicated and in certain cases it is possible that some
// entries will be double counted.
FindTerms(orgID uint32, tags []string, query tagquery.Query) (uint32, map[string]map[string]uint32)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe personal preference, but i think elsewhere we somewhat try to order args by the order in which they're used.
if we first execute the query, then look at its result based on tags, we should probably put tags after the query.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the other functions, this query seems to come later. In fact, I actually do use tags first (meaningfully, at least). I switched the order of use in the implementation, so §tags§ is used first for real.

m.RLock()
defer m.RUnlock()

// TODO - terms for meta tags?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make up our mind. is it something to do (when?), or is this something we're deciding not to support for now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this TODO was meant to spark discussion (forgot to include it when entering the PR). We don't have a need for meta tags in this spot, but maybe it should be supported anyway?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much work would it be to enrich here? cc @replay

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be easy and relatively fast, basically just this needs to be done:

byPath[nameWithTags].MetaTags = mtr.getMetaTagsByRecordIds(enricher.enrich(def.Id.Key))

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the only remaining point. I'm not entirely sure when we would want to include meta tags for this endpoint, since, as @robert-milan mentions, meta tags are really just tied to other tags (albeit in a non-trivial way). Does anyone have any strong preference?

idx/memory/partitioned_idx.go Show resolved Hide resolved
api/routes.go Outdated Show resolved Hide resolved
@replay
Copy link
Contributor

replay commented Jan 9, 2020

Looks good to me. Once we have a conclusion regarding whether meta tags should be included in the result it's fine for me to merge it.

@robert-milan
Copy link
Contributor

Aside from the other comments it looks fine.

@robert-milan
Copy link
Contributor

Looks good to me. Once we have a conclusion regarding whether meta tags should be included in the result it's fine for me to merge it.

I feel like meta tags is a separate concept and should not be included in the results. Since meta tags themselves are just collections of tags I think it might confuse things a bit.

@Dieterbe
Copy link
Contributor

ok so let's not do meta tags then yet.
when docs are updated, we can merge.

Copy link
Contributor

@Dieterbe Dieterbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

splendid

@Dieterbe Dieterbe merged commit 02376a7 into grafana:master Jan 20, 2020
@shanson7 shanson7 deleted the terms branch January 21, 2020 11:06
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support useful tag-based metadata endpoints
4 participants