Revise "Reference: Glossary" #1181

jdbaldry · 2022-02-12T12:19:03Z

What this PR does

Revises all the definitions in the glossary and fits the page with the documentation information architecture.

Checklist

Documentation added

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

osg-grafana · 2022-02-12T14:06:46Z

Relates to #1102

replay · 2022-02-14T13:52:17Z

docs/sources/reference-glossary.md

+
+## Flushing
+
+Flushing is the operation run by ingesters to offload time series from memory and store them in the long-term storage.


I think one could argue that "flushing" and "shipping" are two separate operations in the ingester, considering that they get triggered by separate tickers and a failure of shipping does not result in a failure of flushing.

Tried a clarification in 56efa6f, what do you think?

I can see you reverted your clarification in 1a11857. Was that intentional?

I reverted that because of #1181 (comment) but perhaps I misunderstood?

Right. I think we never came up with a good definition of "flushing". I've always understood it as "from ingesters to long-term storage" because "flush" makes me think we "remove" it from ingesters. I have no strong opinion on this.

replay

One nit, otherwise this LGTM

osg-grafana · 2022-02-15T17:27:58Z

"Ship" this PR and we have #1150 and #1191 as well.

…ect store Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

09jvilla · 2022-02-16T23:38:46Z

Love that you have user<>tenant<>org in here.

pracucci

Thanks for working on this. I left a couple of comments I would be glad you to take look. Thanks!

pracucci · 2022-02-17T08:30:30Z

docs/sources/reference-glossary.md

+## Blocks storage
+
+Blocks storage is the Mimir storage engine based on the Prometheus TSDB.
+Grafana Mimir stores blocks in object stores such as AWS S3, Google Cloud Storage (GCS), or Azure blob storage.


I would list the main supported ones.

Suggested change

Grafana Mimir stores blocks in object stores such as AWS S3, Google Cloud Storage (GCS), or Azure blob storage.

Grafana Mimir stores blocks in object stores such as AWS S3, Google Cloud Storage (GCS), Azure blob storage or Swift.

Revised in 0828eab

pracucci · 2022-02-17T08:31:56Z

docs/sources/reference-glossary.md

+
+## Flushing
+
+Flushing is the operation run by ingesters to offload time series from memory and store them as blocks on disk.


This is not correct. We're used to refer as "flushing" the operation to offload time series from ingester memory and/or disk to the long-term storage (eg. S3).

Reverted in 1a11857

pracucci · 2022-02-17T08:33:23Z

docs/sources/reference-glossary.md

+
+## Gossip
+
+Gossip is a protocol by which components share data to all members without the need for a central store.


without the need for a central store

I think this is misleading. It still requires a centralized storage (eg. S3). Gossip is a protocol that Mimir use to coordinate between components (yes, we share some data, but it's just about coordination between components).

Revised in ba79490

… the object store" This reverts commit 56efa6f. Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

pracucci

LGTM, thanks! I just left a non blocking comment on gossip.

pracucci · 2022-02-17T11:12:40Z

docs/sources/reference-glossary.md

+
+## Gossip
+
+Gossip is a protocol by which components coordinate with the need for a centralized [key-value store]({{<relref "#key-value-store" >}}).


What do you mean with "with the need for a centralized key-value store"? Maybe it's just my poor english, but I've the feeling it doesn't clarify that when using gossip you don't have to run any centralized key-value store, because Mimir components just use this p2p protocol to talk each other and coordinate.

I agree that this wording is a bit confusing.

Sorry yeah that was a typo. It was meant to be "without" 🤦

Perhaps that still isn't the best wording. Happy to replace with something else.

I think fixing the typo could be a good start. At least it makes it correct ;)

replay · 2022-02-17T20:38:39Z

docs/sources/reference-glossary.md

+
+Churn is the frequency at which series become idle.
+
+A series become idle once it's no longer exported by the monitored targets.


Suggested change

A series become idle once it's no longer exported by the monitored targets.

A series becomes idle once it's no longer exported by the monitored targets.

or

Suggested change

A series become idle once it's no longer exported by the monitored targets.

Series become idle once they're no longer exported by the monitored targets.

replay · 2022-02-17T20:42:43Z

docs/sources/reference-glossary.md

+
+## Hash ring
+
+The hash ring is a distributed data structure used by Grafana Mimir for sharding, replication, and service discovery.


How is it distributed? I thought (except in the case of memberlist) it's pretty centralized in the k/v store.

We're used to call it distributed data structure, with regards to the fact that we use a centralized KV store just to "share" it but then it's kept in each process memory and each process can take local decisions (based on the ring copy they have in memory) without having to lookup the KV store each time.

replay

LGTM, I just added nits

pracucci

LGTM (modulo fixing the typo in "gossip" definition)

pracucci · 2022-02-18T08:10:16Z

docs/sources/reference-glossary.md

+
+## Flushing
+
+Flushing is the operation run by ingesters to offload time series from memory and store them in the long-term storage.


I can see you reverted your clarification in 1a11857. Was that intentional?

pracucci · 2022-02-18T08:10:49Z

docs/sources/reference-glossary.md

+
+## Gossip
+
+Gossip is a protocol by which components coordinate with the need for a centralized [key-value store]({{<relref "#key-value-store" >}}).


I think fixing the typo could be a good start. At least it makes it correct ;)

pracucci · 2022-02-18T08:12:26Z

docs/sources/reference-glossary.md

+
+## Hash ring
+
+The hash ring is a distributed data structure used by Grafana Mimir for sharding, replication, and service discovery.


We're used to call it distributed data structure, with regards to the fact that we use a centralized KV store just to "share" it but then it's kept in each process memory and each process can take local decisions (based on the ring copy they have in memory) without having to lookup the KV store each time.

pracucci · 2022-02-18T08:13:19Z

docs/sources/reference-glossary.md

+
+Flushing is the operation run by ingesters to offload time series from memory and store them in the long-term storage.
+
+## Gossip


I would also add a ## Memberlist, mentioning to refer to ## Gossip.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: replay <mauro.stettler@gmail.com>

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: replay <mauro.stettler@gmail.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

jdbaldry · 2022-02-18T11:24:50Z

Thanks for the thorough review!

jdbaldry added 13 commits February 12, 2022 11:28

Rename glossary to fit with information architecture

952d953

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Fix heading hierarchy

b8a1711

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Blocks storage" definition

f1a62de

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Churn" definition

6305b1a

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Flushing" definition

9407fc0

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "HA tracker" definition

63fdc00

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Hash ring" definition including defining related concepts

2bfe5de

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "HA tracker" definition

d9aa7fa

Revise "Sample" definition

8957abd

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Series" definition

170f322

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Tenant" definition

365bed3

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Revise "Write-ahead log (WAL)" definition

1d4d36d

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Replace instance "See" with "Refer to" and use relref links for checking

7861824

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

jdbaldry added the type/docs Improvements or additions to documentation label Feb 12, 2022

jdbaldry requested a review from osg-grafana February 12, 2022 12:19

osg-grafana approved these changes Feb 12, 2022

View reviewed changes

replay reviewed Feb 14, 2022

View reviewed changes

replay approved these changes Feb 14, 2022

View reviewed changes

Clarify that "flushing" is to local disk and "shipping" is to the obj…

56efa6f

…ect store Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

pracucci reviewed Feb 17, 2022

View reviewed changes

jdbaldry and others added 3 commits February 17, 2022 10:19

Revert "Clarify that "flushing" is to local disk and "shipping" is to…

1a11857

… the object store" This reverts commit 56efa6f. Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Add Swift to primary blocks storage backends

0828eab

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

Clarify that gossip is for coordination without a kv store

ba79490

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

pracucci approved these changes Feb 17, 2022

View reviewed changes

replay reviewed Feb 17, 2022

View reviewed changes

replay approved these changes Feb 17, 2022

View reviewed changes

pracucci approved these changes Feb 18, 2022

View reviewed changes

jdbaldry and others added 3 commits February 18, 2022 10:57

Fix typo

7e9ef70

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: replay <mauro.stettler@gmail.com>

Fix typo

01deed0

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: replay <mauro.stettler@gmail.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

Add definition of memberlist and refer to gossip

fcf86ca

Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

jdbaldry merged commit f8d9087 into main Feb 18, 2022

jdbaldry deleted the jdb/2022-02-glossary branch February 18, 2022 11:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revise "Reference: Glossary" #1181

Revise "Reference: Glossary" #1181

jdbaldry commented Feb 12, 2022

osg-grafana commented Feb 12, 2022

replay Feb 14, 2022

jdbaldry Feb 16, 2022

pracucci Feb 18, 2022

jdbaldry Feb 18, 2022

pracucci Feb 18, 2022

replay left a comment

osg-grafana commented Feb 15, 2022

09jvilla commented Feb 16, 2022

pracucci left a comment

pracucci Feb 17, 2022

jdbaldry Feb 17, 2022

pracucci Feb 17, 2022

jdbaldry Feb 17, 2022

pracucci Feb 17, 2022

jdbaldry Feb 17, 2022

pracucci left a comment

pracucci Feb 17, 2022

replay Feb 17, 2022

jdbaldry Feb 17, 2022

pracucci Feb 18, 2022

replay Feb 17, 2022

replay Feb 17, 2022 •

edited

Loading

replay Feb 17, 2022 •

edited

Loading

pracucci Feb 18, 2022

replay left a comment

pracucci left a comment

pracucci Feb 18, 2022

pracucci Feb 18, 2022

pracucci Feb 18, 2022

pracucci Feb 18, 2022

jdbaldry commented Feb 18, 2022


		## Flushing

		Flushing is the operation run by ingesters to offload time series from memory and store them in the long-term storage.

	Grafana Mimir stores blocks in object stores such as AWS S3, Google Cloud Storage (GCS), or Azure blob storage.
	Grafana Mimir stores blocks in object stores such as AWS S3, Google Cloud Storage (GCS), Azure blob storage or Swift.


		## Gossip

		Gossip is a protocol by which components share data to all members without the need for a central store.


		Churn is the frequency at which series become idle.

		A series become idle once it's no longer exported by the monitored targets.

	A series become idle once it's no longer exported by the monitored targets.
	A series becomes idle once it's no longer exported by the monitored targets.

	A series become idle once it's no longer exported by the monitored targets.
	Series become idle once they're no longer exported by the monitored targets.


		## Hash ring

		The hash ring is a distributed data structure used by Grafana Mimir for sharding, replication, and service discovery.

Revise "Reference: Glossary" #1181

Revise "Reference: Glossary" #1181

Conversation

jdbaldry commented Feb 12, 2022

What this PR does

Checklist

osg-grafana commented Feb 12, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

replay left a comment

Choose a reason for hiding this comment

osg-grafana commented Feb 15, 2022

09jvilla commented Feb 16, 2022

pracucci left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pracucci left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

replay Feb 17, 2022 • edited Loading

Choose a reason for hiding this comment

replay Feb 17, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

replay left a comment

Choose a reason for hiding this comment

pracucci left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdbaldry commented Feb 18, 2022

replay Feb 17, 2022 •

edited

Loading

replay Feb 17, 2022 •

edited

Loading