From dc3d5ed00cc0debb52079060a576f0db7cc6955a Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 18 Jan 2022 12:23:36 -0800 Subject: [PATCH 01/41] allow duplicate instrument reg --- specification/metrics/api.md | 40 ++++++++++++++++++++++++++++-------- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index c5d75d81660..b2c8aa0d678 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -180,14 +180,33 @@ will have the following information: * An optional `unit` of measure * An optional `description` -Instruments are associated with the Meter during creation, and are identified by -the name: - -* Meter implementations MUST return an error when multiple Instruments are - registered under the same Meter instance using the same name. -* Different Meters MUST be treated as separate namespaces. The names of the - Instruments under one Meter SHOULD NOT interfere with Instruments under - another Meter. + + +Instruments are associated with the Meter during creation and are +identified by instrument name. Duplicate registration of +identically-named instruments within a Meter is treated as follows: + +* If the registration is semantically identical, meaning to have the + same `kind` and `unit` as a previously registered instrument of the + same `name`, the implementation MUST return a valid instrument. + It is unspecified whether or under which conditions the same or different + instrument instance will be returned as a result of duplicate registration. +* If the registration is semantically not identical, the + implementation MUST return a registration error to the user when the + instrument is constructed. + +When determining whether a duplicate registration is valid, +language-level features such the distinction between integer and +floating point numbers SHOULD be considered. Equivalently, +implementations SHOULD NOT support an application in emitting both +integer and floating point numbers for the same metric inside the same +instrumentation library. + + + +Different Meters MUST be treated as separate namespaces. The names of the +Instruments under one Meter SHOULD NOT interfere with Instruments under +another Meter. @@ -239,6 +258,11 @@ instrument. It MUST be treated as an opaque string from the API and SDK. * It MUST support at least 1023 characters. [OpenTelemetry API](../overview.md#api) authors MAY decide if they want to support more. +Note the `description` property of an instrument is explicitly +disregarded when considering duplicate registration, because it is not +semantic in nature. Implementations SHOULD use any of the provided +`description` values when emitting metrics that had duplicate registrations. + Instruments can be categorized based on whether they are synchronous or asynchronous: From 80c0e73d06b0510edb96979c3fe6e618ad2a24a7 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 18 Jan 2022 12:23:48 -0800 Subject: [PATCH 02/41] comment about single-writer rule and duplicate instruments --- specification/metrics/sdk.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index f85f92b061b..4b091191931 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -952,3 +952,22 @@ called concurrently. **MetricExporter** - `ForceFlush` and `Shutdown` are safe to be called concurrently. + +## Data model requirements + +The implementation is required to respect the OpenTelemetry Metrics +data model [Single Writer](datamodel.md#single-writer) requirement. +This rule stipulates that the implementation MUST avoid creating +duplicate output streams from a given SDK. + +This rule is the basis of the output-name uniqueness check in for +[Views](#view) above, and it also constrains how duplicate instrument +registration is handled. + +For example, the implementation is not required to return the +identical instrument when a duplicate instrument is registered, but +assuming it does allow separate instances to co-exist, the +implementation MUST eliminate the duplication at a later point using +the [natural merge function](#opentelemetry-protocol-data-model) for +those data points, as otherwise it would risk violating the +single-writer rule. From 64c955f81c6a90c6d9be0f008900d3100ff25936 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 18 Jan 2022 12:30:36 -0800 Subject: [PATCH 03/41] comment about duplicate asynchronous observations --- specification/metrics/api.md | 10 ++++++++++ specification/metrics/sdk.md | 5 +++++ 2 files changed, 15 insertions(+) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index b2c8aa0d678..7267e279cfc 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -288,6 +288,16 @@ Please note that the term _synchronous_ and _asynchronous_ have nothing to do with the [asynchronous pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). +When asynchronous instruments are used with duplicate registrations, +meaning to have more than one callback provided, the order of callback +execution is unspecified. Callers SHOULD avoid making duplicate +observations from asynchronous instrument callbacks. However, if a +user accidentally makes duplicate observations, meaning to provide +more than one value for a given asynchronous instrument and attribute +set, the caller SHOULD expect the last-observed value to be the one +recorded. This is sometimes referred to as "last-value wins" for +asynchronous instruments. + ### Counter `Counter` is a [synchronous Instrument](#synchronous-instrument) which supports diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 4b091191931..945306d4c17 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -971,3 +971,8 @@ implementation MUST eliminate the duplication at a later point using the [natural merge function](#opentelemetry-protocol-data-model) for those data points, as otherwise it would risk violating the single-writer rule. + +As another example, users are encouraged not to make duplicate +observations from asynchronous instrument callbacks. However, +implementations MUST NOT violate the single-writer rule even when +users make duplicate observations. From e70203c2d4e275337b8b54b1d6aa4bbdd6f2241f Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 18 Jan 2022 12:34:26 -0800 Subject: [PATCH 04/41] refer to supp guidelines as well --- specification/metrics/sdk.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 945306d4c17..6825a5aa0ca 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -975,4 +975,6 @@ single-writer rule. As another example, users are encouraged not to make duplicate observations from asynchronous instrument callbacks. However, implementations MUST NOT violate the single-writer rule even when -users make duplicate observations. +users make duplicate observations. This is also covered in the +[supplemental guidelines for handling asynchronous instrument +views](#asynchronous-example-attribute-removal-in-a-view). From fc0cfe9ff313b1685b03dac95c7392c1f3f88177 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Wed, 2 Feb 2022 16:29:44 -0800 Subject: [PATCH 05/41] first draft update --- specification/metrics/api.md | 105 ++++++++++++++++------------- specification/metrics/datamodel.md | 28 ++++++-- 2 files changed, 81 insertions(+), 52 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 7267e279cfc..b431d028899 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -129,11 +129,16 @@ This API MUST accept the following parameters: * [since 1.4.0] `schema_url` (optional): Specifies the Schema URL that should be recorded in the emitted telemetry. -It is unspecified whether or under which conditions the same or different -`Meter` instances are returned from this functions. - -Implementations MUST NOT require users to repeatedly obtain a `Meter` again with -the same name+version+schema_url to pick up configuration changes. This can be +Meters are identified by all of these fields. When more than one +Meter of the same `name`, `version`, and `schema_url` is created, it +is unspecified whether or under which conditions the same or different +`Meter` instances are returned. Meter identity, and the terms +_identical_ and _distinct_ applied to Meters in this specification +describes Meter instances that have identical and/or distinct values +for all three of their `name`, `version`, and `schema_url` attributes. + +Implementations MUST NOT require users to repeatedly obtain a `Meter` with +the same identity to pick up configuration changes. This can be achieved either by allowing to work with an outdated configuration or by ensuring that new configuration applies also to previously returned `Meter`s. @@ -141,7 +146,7 @@ Note: This could, for example, be implemented by storing any mutable configuration in the `MeterProvider` and having `Meter` implementation objects have a reference to the `MeterProvider` from which they were obtained. If configuration must be stored per-meter (such as disabling a certain meter), the -meter could, for example, do a look-up with its name+version+schema_url in a map +meter could, for example, do a look-up with its identity in a map in the `MeterProvider`, or the `MeterProvider` could maintain a registry of all returned `Meter`s and actively update their configuration if it changes. @@ -175,38 +180,50 @@ Instruments are used to report [Measurements](#measurement). Each Instrument will have the following information: * The `name` of the Instrument -* The `kind` of the Instrument - whether it is a [Counter](#counter) or other - instruments, whether it is synchronous or asynchronous +* The `kind` of the Instrument - whether it is a [Counter](#counter) or + one of the other kinds, whether it is synchronous or asynchronous * An optional `unit` of measure * An optional `description` - +Instruments are associated with the Meter during creation. Instruments +are identified by all of these fields. + + + +When more than one Instrument of the same `name`, `kind`, `unit`, and +`description` is created for identical Meters, denoted _duplicate +instrument registration_, the implementation MUST return a valid +Instrument in every case. + +It is unspecified whether or under which conditions the same or +different Instrument instance will be returned as a result of +duplicate instrument registration. Instrument identity, and the terms +_identical_ and _distinct_ applied to Instruments in this +specification describes Instrument instances that have identical +and/or distinct values for all four of their `name`, `kind`, `unit`, +and `description` attributes. Language-level features such the +distinction between integer and floating point numbers SHOULD be +considered as identifying, as well. -Instruments are associated with the Meter during creation and are -identified by instrument name. Duplicate registration of -identically-named instruments within a Meter is treated as follows: +The implementation SHOULD aggregate data from identical Instruments +together in its export pipeline. -* If the registration is semantically identical, meaning to have the - same `kind` and `unit` as a previously registered instrument of the - same `name`, the implementation MUST return a valid instrument. - It is unspecified whether or under which conditions the same or different - instrument instance will be returned as a result of duplicate registration. -* If the registration is semantically not identical, the - implementation MUST return a registration error to the user when the - instrument is constructed. +When through duplicate registration more than one distinct Instrument +is registered with the same `name` for identical Meters, the +implementation SHOULD emit a warning to the user informing them of +duplicate registration conflict(s). -When determining whether a duplicate registration is valid, -language-level features such the distinction between integer and -floating point numbers SHOULD be considered. Equivalently, -implementations SHOULD NOT support an application in emitting both -integer and floating point numbers for the same metric inside the same -instrumentation library. +__Note the warning about duplicate instrumentation registration +conflicts is meant to help avoid the semantic error state described in +the [OpenTelemetry Metrics data +model](datamodel.md#opentelemetry-data-model) when more than one kind +of point is written for a given instrument `name` and Meter identity. -Different Meters MUST be treated as separate namespaces. The names of the -Instruments under one Meter SHOULD NOT interfere with Instruments under -another Meter. +Distinct Meters MUST be treated as separate namespaces for the +purposes of [Instrument type conflict +detection](#instrument-type-conflict-detection). @@ -229,7 +246,7 @@ DIGIT = %x30-39 ; 0-9 -The `unit` is an optional string provided by the author of the instrument. It +The `unit` is an optional string provided by the author of the Instrument. It SHOULD be treated as an opaque string from the API and SDK (e.g. the SDK is not expected to validate the unit of measurement, or perform the unit conversion). @@ -258,12 +275,7 @@ instrument. It MUST be treated as an opaque string from the API and SDK. * It MUST support at least 1023 characters. [OpenTelemetry API](../overview.md#api) authors MAY decide if they want to support more. -Note the `description` property of an instrument is explicitly -disregarded when considering duplicate registration, because it is not -semantic in nature. Implementations SHOULD use any of the provided -`description` values when emitting metrics that had duplicate registrations. - -Instruments can be categorized based on whether they are synchronous or +Instruments are categorized based on whether they are synchronous or asynchronous: @@ -288,15 +300,16 @@ Please note that the term _synchronous_ and _asynchronous_ have nothing to do with the [asynchronous pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). -When asynchronous instruments are used with duplicate registrations, -meaning to have more than one callback provided, the order of callback -execution is unspecified. Callers SHOULD avoid making duplicate -observations from asynchronous instrument callbacks. However, if a -user accidentally makes duplicate observations, meaning to provide -more than one value for a given asynchronous instrument and attribute -set, the caller SHOULD expect the last-observed value to be the one -recorded. This is sometimes referred to as "last-value wins" for -asynchronous instruments. +Considering duplicate registration, of asynchronous instruments, the +implementation MUST register every callback, even for conflicting-type +instruments. + +Callers SHOULD avoid making duplicate observations from asynchronous +instrument callbacks. However, if a user accidentally makes duplicate +observations, meaning to provide more than one value for a given +asynchronous instrument and attribute set, the caller SHOULD expect +the last-observed value to be the one recorded. This is sometimes +referred to as "last-value wins" for asynchronous instruments. ### Counter @@ -430,7 +443,7 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). -* A `callback` function. +* An optional `callback` function. The `callback` function is responsible for reporting the [Measurement](#measurement)s. It will only be called when the Meter is being diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index ea99ae2ba81..d1de9357076 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -263,18 +263,33 @@ can be converted directly into Timeseries, and share the same identity characteristics for a Timeseries. A metric stream is identified by: - The originating `Resource` +- The metric's associated Meter details (`name`, `version`, and `schema_url`) - The metric stream's `name`. - The attached `Attribute`s -- The metric stream's point kind. +- The metric stream's point kind (`Sum`, `Gauge`, `Histogram` `ExponentialHistogram`) It is possible (and likely) that more than one metric stream is created per `Instrument` in the event model. -__Note: The same `Resource`, `name` and `Attribute`s but differing point kind -coming out of an OpenTelemetry SDK is considered an "error state" that SHOULD -be handled by an SDK.__ - -A metric stream can use one of these basic point kinds, all of +Points with identical `name`, `Resource`, and Meter details but +different point kind SHOULD be considered an "error state", due to +semantic disagreement, whether or not they use distinct `Attribute` +values. + +Consumers of OpenTelemetry metrics data streams MAY reject data +containing the multiple point kinds for identical `name`, `Resource`, +and Meter details, even though the [Single-Writer](#single-writer) +rule is only broken when distinct `Attribute` values are involved. + +__Note: The OpenTelemetry SDK specification is written to explicitly +permit duplicate instrumentation registration, which means the +potential to produce the error state. This is considered to be in the +user's best interest. Although the SDK has permission to pass-through +duplicate instrument registration conflicts, consumers of +OpenTelemetry metrics are given equal permission to reject the data +because of semantic disagreement. + +A metric stream can use one of the basic point kinds, all of which satisfy the requirements above, meaning they define a decomposable aggregate function (also known as a “natural merge” function) for points of the same kind. [1](#otlpdatapointfn) @@ -749,6 +764,7 @@ All metric data streams within OTLP MUST have one logical writer. This means, conceptually, that any Timeseries created from the Protocol MUST have one originating source of truth. In practical terms, this implies the following: +@@@ - All metric data streams produced by OTel SDKs MUST be globally uniquely produced and free from duplicates. All metric data streams can be uniquely identified in some way. From c94358f3ff1d67a383242784da27cdd2c9ac7925 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 08:57:46 -0800 Subject: [PATCH 06/41] pre-merge --- specification/metrics/api.md | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index b431d028899..b9c5cdb5bf1 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -190,10 +190,9 @@ are identified by all of these fields. -When more than one Instrument of the same `name`, `kind`, `unit`, and -`description` is created for identical Meters, denoted _duplicate -instrument registration_, the implementation MUST return a valid -Instrument in every case. +When more than one Instrument of the same `name` is created for +identical Meters, denoted _duplicate instrument registration_, the +implementation MUST return a valid Instrument in every case. It is unspecified whether or under which conditions the same or different Instrument instance will be returned as a result of @@ -217,13 +216,14 @@ __Note the warning about duplicate instrumentation registration conflicts is meant to help avoid the semantic error state described in the [OpenTelemetry Metrics data model](datamodel.md#opentelemetry-data-model) when more than one kind -of point is written for a given instrument `name` and Meter identity. +of point is written for a given instrument `name` and Meter identity +from the same process. Distinct Meters MUST be treated as separate namespaces for the -purposes of [Instrument type conflict -detection](#instrument-type-conflict-detection). +purposes of detecting [duplicate instrument registration +conflicts](#instrument-type-conflict-detection). @@ -305,11 +305,7 @@ implementation MUST register every callback, even for conflicting-type instruments. Callers SHOULD avoid making duplicate observations from asynchronous -instrument callbacks. However, if a user accidentally makes duplicate -observations, meaning to provide more than one value for a given -asynchronous instrument and attribute set, the caller SHOULD expect -the last-observed value to be the one recorded. This is sometimes -referred to as "last-value wins" for asynchronous instruments. +instrument callbacks. ### Counter From 806317b24eb19df903d60266c1d25cbe0bb498a4 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 09:52:12 -0800 Subject: [PATCH 07/41] finish draft --- specification/metrics/api.md | 37 ++++++++++++++++++++++++------- specification/metrics/sdk.md | 43 +++++++++++++++++------------------- 2 files changed, 49 insertions(+), 31 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index cc00485d837..ff58ee3dece 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -204,7 +204,7 @@ and `description` attributes. Language-level features such the distinction between integer and floating point numbers SHOULD be considered as identifying, as well. -The implementation SHOULD aggregate data from identical Instruments +The implementation MUST aggregate data from identical Instruments together in its export pipeline. When through duplicate registration more than one distinct Instrument @@ -300,12 +300,24 @@ Please note that the term _synchronous_ and _asynchronous_ have nothing to do with the [asynchronous pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). -Considering duplicate registration, of asynchronous instruments, the -implementation MUST register every callback, even for conflicting-type -instruments. +Considering duplicate registration of asynchronous instruments, the +implementation MUST support multiple callbacks registered per +identical instrument. + +The API MAY provide support for registration of a single callback +having multiple asynchronous instruments, where idiomatic. The +implementation is REQUIRED to maintain a mapping from every regsitered +callback to the associated instrument(s). + +The implementation MUST provide a way to unregister callbacks +associated with asynchronous instruments. + +The implementation SHOULD reject observations that are made outside of +a callback registered for the instrument. Callers SHOULD avoid making duplicate observations from asynchronous -instrument callbacks. +instrument callbacks, as the result of making duplicate observations +is not specified. ### Counter @@ -439,7 +451,10 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). -* An optional `callback` function. + +The API MUST support an idiomatic way to associate one or more +`callback` functions with each instrument, whether through the +instrument constructor or a separate registration method. The `callback` function is responsible for reporting the [Measurement](#measurement)s. It will only be called when the Meter is being @@ -656,7 +671,10 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). -* A `callback` function. + +The API MUST support an idiomatic way to associate one or more +`callback` functions with each instrument, whether through the +instrument constructor or a separate registration method. The `callback` function is responsible for reporting the [Measurement](#measurement)s. It will only be called when the Meter is being @@ -928,7 +946,10 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). -* A `callback` function. + +The API MUST support an idiomatic way to associate one or more +`callback` functions with each instrument, whether through the +instrument constructor or a separate registration method. The `callback` function is responsible for reporting the [Measurement](#measurement)s. It will only be called when the Meter is being diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index c36fdd42ee1..585a4e5f07e 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -955,26 +955,23 @@ concurrently. ## Data model requirements -The implementation is required to respect the OpenTelemetry Metrics -data model [Single Writer](datamodel.md#single-writer) requirement. -This rule stipulates that the implementation MUST avoid creating -duplicate output streams from a given SDK. - -This rule is the basis of the output-name uniqueness check in for -[Views](#view) above, and it also constrains how duplicate instrument -registration is handled. - -For example, the implementation is not required to return the -identical instrument when a duplicate instrument is registered, but -assuming it does allow separate instances to co-exist, the -implementation MUST eliminate the duplication at a later point using -the [natural merge function](#opentelemetry-protocol-data-model) for -those data points, as otherwise it would risk violating the -single-writer rule. - -As another example, users are encouraged not to make duplicate -observations from asynchronous instrument callbacks. However, -implementations MUST NOT violate the single-writer rule even when -users make duplicate observations. This is also covered in the -[supplemental guidelines for handling asynchronous instrument -views](#asynchronous-example-attribute-removal-in-a-view). +The [OpenTelemetry Metrics data +model](datamodel.md#opentelemetry-protocol-data-model) has a [Single +Writer](datamodel.md#single-writer) rule which the SDK is required to +enforce, with one exception noted below. This rule is the origin of +the output-name uniqueness requirement for [Views](#view) and is the +reason the API specifies that implementations MUST aggregate data for +identical instruments in its pipeline. + +The implementation has permission to pass-through violations of the +semantic "error state" described in the [OpenTelemetry Metrics data +model](datamodel.md#opentelemetry-protocol-data-model). This is to +address the exceptional case of duplicate instrument registration +conflicts, which are tolerated with warnings in the API. + +Duplicate instrument registration conflicts can produce this semantic +error state, which may lead to violations of the single-writer rule. +SDKs are permitted to allow these conflicts to take place to avoid +user data loss. Since the semantic conflict may not be resolvable +downstream, Metrics data consumers MAY reject semantically conflicting +data and SHOULD notify the user of the error, somehow, in such cases. From e166181368f17b8d91824f875dd4f6bcb294d334 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 11:36:54 -0800 Subject: [PATCH 08/41] two edits --- specification/metrics/api.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index ff58ee3dece..7a173e938d5 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -200,17 +200,16 @@ duplicate instrument registration. Instrument identity, and the terms _identical_ and _distinct_ applied to Instruments in this specification describes Instrument instances that have identical and/or distinct values for all four of their `name`, `kind`, `unit`, -and `description` attributes. Language-level features such the +and `description` attributes. Language-level features such as the distinction between integer and floating point numbers SHOULD be considered as identifying, as well. The implementation MUST aggregate data from identical Instruments together in its export pipeline. -When through duplicate registration more than one distinct Instrument -is registered with the same `name` for identical Meters, the -implementation SHOULD emit a warning to the user informing them of -duplicate registration conflict(s). +When more than one distinct Instrument is registered with the same +`name` for identical Meters, the implementation SHOULD emit a warning +to the user informing them of duplicate registration conflict(s). __Note the warning about duplicate instrumentation registration conflicts is meant to help avoid the semantic error state described in From 73d3c035fd0c2866f6b0d1c52dbae2865f2ab8fc Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 12:22:15 -0800 Subject: [PATCH 09/41] SDK req --- specification/metrics/sdk.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 585a4e5f07e..7a726f106ab 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -414,6 +414,21 @@ instruments that record negative measurements, e.g. `UpDownCounter` or `Observab - Min (optional) `Measurement` value in population. - Max (optional) `Measurement` value in population. +### Resolving duplicate instrument registration conflicts + +Implementation are required by the API specification to tolerate +duplicate instrument registration conflicts. View configuration +offers the user an opportunity to configure a workaround in cases +where the conflict involves the instrument type. + +To help the user manage such conflicts, the implementation SHOULD warn +about dupliclate instrument registration conflicts after applying the +Views configuration. For conflicts that can be resolved through a +View configuration, which includes instrument-type conflicts but not +unit or description conflicts, the implementation's warning SHOULD +include an example suggesting a View configuration to resolve the +conflict. + ## Attribute limits Attributes which belong to Metrics are exempt from the From 0ffc30589722263f23407785fff9d2325a815718 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 13:00:05 -0800 Subject: [PATCH 10/41] intrinsic properties --- specification/metrics/datamodel.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index d1de9357076..278341871f5 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -267,14 +267,15 @@ characteristics for a Timeseries. A metric stream is identified by: - The metric stream's `name`. - The attached `Attribute`s - The metric stream's point kind (`Sum`, `Gauge`, `Histogram` `ExponentialHistogram`) +- Intrinsic properties of the point, including temporality and monotonicity It is possible (and likely) that more than one metric stream is created per `Instrument` in the event model. Points with identical `name`, `Resource`, and Meter details but -different point kind SHOULD be considered an "error state", due to -semantic disagreement, whether or not they use distinct `Attribute` -values. +different point kind or different point kind intrinsic properties +SHOULD be considered an "error state", due to semantic disagreement, +whether or not they use distinct `Attribute` values. Consumers of OpenTelemetry metrics data streams MAY reject data containing the multiple point kinds for identical `name`, `Resource`, From 7d5163b6ef58371357ecb916910dc45412f6b725 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 13:24:01 -0800 Subject: [PATCH 11/41] from 2270 --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 7a726f106ab..33787bc7099 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -972,7 +972,7 @@ concurrently. The [OpenTelemetry Metrics data model](datamodel.md#opentelemetry-protocol-data-model) has a [Single -Writer](datamodel.md#single-writer) rule which the SDK is required to +Writer](datamodel.md#single-writer) rule that the SDK is REQUIRED to enforce, with one exception noted below. This rule is the origin of the output-name uniqueness requirement for [Views](#view) and is the reason the API specifies that implementations MUST aggregate data for From 0778baf7bac673eaabda8fbc0693c4dfab321129 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 13:45:57 -0800 Subject: [PATCH 12/41] typos --- specification/metrics/api.md | 4 ++-- specification/metrics/datamodel.md | 1 - specification/metrics/sdk.md | 2 ++ 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 7a173e938d5..2f617e6263e 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -180,7 +180,7 @@ Instruments are used to report [Measurements](#measurement). Each Instrument will have the following information: * The `name` of the Instrument -* The `kind` of the Instrument - whether it is a [Counter](#counter) or +* The `kind` of the Instrument - whether it is a [Counter](#counter) or one of the other kinds, whether it is synchronous or asynchronous * An optional `unit` of measure * An optional `description` @@ -305,7 +305,7 @@ identical instrument. The API MAY provide support for registration of a single callback having multiple asynchronous instruments, where idiomatic. The -implementation is REQUIRED to maintain a mapping from every regsitered +implementation is REQUIRED to maintain a mapping from every registered callback to the associated instrument(s). The implementation MUST provide a way to unregister callbacks diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 278341871f5..41e5130d562 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -765,7 +765,6 @@ All metric data streams within OTLP MUST have one logical writer. This means, conceptually, that any Timeseries created from the Protocol MUST have one originating source of truth. In practical terms, this implies the following: -@@@ - All metric data streams produced by OTel SDKs MUST be globally uniquely produced and free from duplicates. All metric data streams can be uniquely identified in some way. diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 33787bc7099..5f1612e5729 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -19,6 +19,7 @@ + [Last Value Aggregation](#last-value-aggregation) + [Histogram Aggregation](#histogram-aggregation) + [Explicit Bucket Histogram Aggregation](#explicit-bucket-histogram-aggregation) + * [Resolving duplicate instrument registration conflicts](#resolving-duplicate-instrument-registration-conflicts) - [Attribute limits](#attribute-limits) - [Exemplar](#exemplar) * [ExemplarFilter](#exemplarfilter) @@ -40,6 +41,7 @@ - [Numerical limits handling](#numerical-limits-handling) - [Compatibility requirements](#compatibility-requirements) - [Concurrency requirements](#concurrency-requirements) +- [Data model requirements](#data-model-requirements) From bd3056ab74fe10e11e1ec6840a6d4fbceb8f2339 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 13:46:36 -0800 Subject: [PATCH 13/41] add summary --- specification/metrics/datamodel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 41e5130d562..64d0418b8ed 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -266,7 +266,7 @@ characteristics for a Timeseries. A metric stream is identified by: - The metric's associated Meter details (`name`, `version`, and `schema_url`) - The metric stream's `name`. - The attached `Attribute`s -- The metric stream's point kind (`Sum`, `Gauge`, `Histogram` `ExponentialHistogram`) +- The metric stream's point kind (`Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) - Intrinsic properties of the point, including temporality and monotonicity It is possible (and likely) that more than one metric stream is created per From 65d614725771f0f9e3ad9fb8594a7d7423f3b748 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 4 Feb 2022 15:30:46 -0800 Subject: [PATCH 14/41] Require a shorter test for probability sampler --- .../trace/tracestate-probability-sampling.md | 97 +++++++++---------- 1 file changed, 44 insertions(+), 53 deletions(-) diff --git a/specification/trace/tracestate-probability-sampling.md b/specification/trace/tracestate-probability-sampling.md index 6cad1c9ff82..700ec46c210 100644 --- a/specification/trace/tracestate-probability-sampling.md +++ b/specification/trace/tracestate-probability-sampling.md @@ -870,7 +870,7 @@ this a strict test for random behavior, we take the following approach: - Generate a pre-determined list of 20 random seeds - Use fixed values for significance level (5%) and trials (20) -- Use a population size of one million spans +- Use a population size of one hundred thousand spans - For each trial, simulate the population and compute ChiSquared test statistic - Locate the first seed value in the ordered list such that the @@ -895,22 +895,19 @@ In this case there are two degrees of freedom for the Chi-Squared test. The following table summarizes the test parameters. | Test case | Sampling probability | Lower, Upper p-value when sampled | Expectlower | Expectupper | Expectunsampled | -| --- | --- | --- | --- | --- | --- | -| 1 | 0.900000 | 0, 1 | 100000 | 800000 | 100000 | -| 2 | 0.600000 | 0, 1 | 400000 | 200000 | 400000 | -| 3 | 0.330000 | 1, 2 | 170000 | 160000 | 670000 | -| 4 | 0.130000 | 2, 3 | 120000 | 10000 | 870000 | -| 5 | 0.100000 | 3, 4 | 25000 | 75000 | 900000 | -| 6 | 0.050000 | 4, 5 | 12500 | 37500 | 950000 | -| 7 | 0.017000 | 5, 6 | 14250 | 2750 | 983000 | -| 8 | 0.010000 | 6, 7 | 5625 | 4375 | 990000 | -| 9 | 0.005000 | 7, 8 | 2812.5 | 2187.5 | 995000 | -| 10 | 0.002900 | 8, 9 | 1006.25 | 1893.75 | 997100 | -| 11 | 0.001000 | 9, 10 | 953.125 | 46.875 | 999000 | -| 12 | 0.000500 | 10, 11 | 476.5625 | 23.4375 | 999500 | -| 13 | 0.000260 | 11, 12 | 228.28125 | 31.71875 | 999740 | -| 14 | 0.000230 | 12, 13 | 14.140625 | 215.859375 | 999770 | -| 15 | 0.000100 | 13, 14 | 22.0703125 | 77.9296875 | 999900 | +|-----------|----------------------|-----------------------------------|------------------------|------------------------|----------------------------| +| 1 | 0.900000 | 0, 1 | 10000 | 80000 | 10000 | +| 2 | 0.600000 | 0, 1 | 40000 | 20000 | 40000 | +| 3 | 0.330000 | 1, 2 | 17000 | 16000 | 67000 | +| 4 | 0.130000 | 2, 3 | 12000 | 1000 | 87000 | +| 5 | 0.100000 | 3, 4 | 2500 | 7500 | 90000 | +| 6 | 0.050000 | 4, 5 | 1250 | 3750 | 95000 | +| 7 | 0.017000 | 5, 6 | 1425 | 275 | 98300 | +| 8 | 0.010000 | 6, 7 | 562.5 | 437.5 | 99000 | +| 9 | 0.005000 | 7, 8 | 281.25 | 218.75 | 99500 | +| 10 | 0.002900 | 8, 9 | 100.625 | 189.375 | 99710 | +| 11 | 0.001000 | 9, 10 | 95.3125 | 4.6875 | 99900 | +| 12 | 0.000500 | 10, 11 | 47.65625 | 2.34375 | 99950 | The formula for computing Chi-Squared in this case is: @@ -928,7 +925,7 @@ than 0.102587. ##### Requirement: Pass 15 non-power-of-two statistical tests -For the test with 20 trials and 1 million spans each, the test MUST +For the test with 20 trials and 100,000 spans each, the test MUST demonstrate a random number generator seed such that the ChiSquared test statistic is below 0.102587 exactly 1 out of 20 times. @@ -937,13 +934,12 @@ test statistic is below 0.102587 exactly 1 out of 20 times. In this case there is one degree of freedom for the Chi-Squared test. The following table summarizes the test parameters. -| Test case | Sampling probability | P-value when sampled | Expectsampled | Expectunsampled | | -| --- | --- | --- | --- | --- | | -| 16 | 0x1p-01 (0.500000) | 1 | 500000 | n/a | 500000 | -| 17 | 0x1p-04 (0.062500) | 4 | 62500 | n/a | 937500 | -| 18 | 0x1p-07 (0.007812) | 7 | 7812.5 | n/a | 992187.5 | -| 19 | 0x1p-10 (0.000977) | 10 | 976.5625 | n/a | 999023.4375 | -| 20 | 0x1p-13 (0.000122) | 13 | 122.0703125 | n/a | 999877.9297 | +| Test case | Sampling probability | P-value when sampled | Expectsampled | Expectunsampled | | +|-----------|----------------------|----------------------|--------------------------|----------------------------|----------| +| 13 | 0x1p-01 (0.500000) | 1 | 50000 | n/a | 50000 | +| 14 | 0x1p-04 (0.062500) | 4 | 6250 | n/a | 93750 | +| 15 | 0x1p-07 (0.007812) | 7 | 781.25 | n/a | 99218.75 | + The formula for computing Chi-Squared in this case is: @@ -960,49 +956,44 @@ than 0.003932. ##### Requirement: Pass 5 power-of-two statistical tests -For the teset with 20 trials and 1 million spans each, the test MUST +For the teset with 20 trials and 100,000 spans each, the test MUST demonstrate a random number generator seed such that the ChiSquared test statistic is below 0.003932 exactly 1 out of 20 times. #### Test implementation -The recommended structure for this test uses a table listing the 20 +The recommended structure for this test uses a table listing the 15 probability values, the expected p-values, whether the ChiSquared statistic has one or two degrees of freedom, and the index into the predetermined list of seeds. ``` for _, test := range []testCase{ - // Non-powers of two - {0.90000, 1, twoDegrees, 5}, - {0.60000, 1, twoDegrees, 14}, - {0.33000, 2, twoDegrees, 3}, - {0.13000, 3, twoDegrees, 2}, - {0.10000, 4, twoDegrees, 0}, - {0.05000, 5, twoDegrees, 0}, - {0.01700, 6, twoDegrees, 2}, - {0.01000, 7, twoDegrees, 3}, - {0.00500, 8, twoDegrees, 1}, - {0.00290, 9, twoDegrees, 1}, - {0.00100, 10, twoDegrees, 5}, - {0.00050, 11, twoDegrees, 1}, - {0.00026, 12, twoDegrees, 3}, - {0.00023, 13, twoDegrees, 0}, - {0.00010, 14, twoDegrees, 2}, - - // Powers of two - {0x1p-1, 1, oneDegree, 0}, - {0x1p-4, 4, oneDegree, 2}, - {0x1p-7, 7, oneDegree, 3}, - {0x1p-10, 10, oneDegree, 0}, - {0x1p-13, 13, oneDegree, 1}, + // Non-powers of two + {0.90000, 1, twoDegrees, 3}, + {0.60000, 1, twoDegrees, 2}, + {0.33000, 2, twoDegrees, 2}, + {0.13000, 3, twoDegrees, 1}, + {0.10000, 4, twoDegrees, 0}, + {0.05000, 5, twoDegrees, 0}, + {0.01700, 6, twoDegrees, 2}, + {0.01000, 7, twoDegrees, 2}, + {0.00500, 8, twoDegrees, 2}, + {0.00290, 9, twoDegrees, 4}, + {0.00100, 10, twoDegrees, 6}, + {0.00050, 11, twoDegrees, 0}, + + // Powers of two + {0x1p-1, 1, oneDegree, 0}, + {0x1p-4, 4, oneDegree, 0}, + {0x1p-7, 7, oneDegree, 1}, } { ``` Note that seed indexes in the example above have what appears to be -the correct distribution. The five 0s, four 1s, four 2s, four 3s, and -two 5s demonstrate that it is relatively easy to find examples where -there is exactly one failure. Seed index 14, for probability 0.6 in +the correct distribution. The five 0s, two 1s, five 2s, one 3s, and +one 4 demonstrate that it is relatively easy to find examples where +there is exactly one failure. Probability 0.001, with seed index 6 in this case, is a reminder that outliers exist. Further significance testing of this distribution is not recommended. From f3cbbcc3cd097ddad7caa5fadba76490d81e625e Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Mon, 7 Feb 2022 09:59:04 -0800 Subject: [PATCH 15/41] Update specification/metrics/sdk.md Co-authored-by: Marc Pichler --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 5f1612e5729..adf6b1c2c6b 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -418,7 +418,7 @@ instruments that record negative measurements, e.g. `UpDownCounter` or `Observab ### Resolving duplicate instrument registration conflicts -Implementation are required by the API specification to tolerate +Implementations are required by the API specification to tolerate duplicate instrument registration conflicts. View configuration offers the user an opportunity to configure a workaround in cases where the conflict involves the instrument type. From ef7874798db442ce2c71556c1cab2e6a0c17e28e Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Mon, 7 Feb 2022 09:59:15 -0800 Subject: [PATCH 16/41] Update specification/metrics/datamodel.md Co-authored-by: Tyler Yahn --- specification/metrics/datamodel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 64d0418b8ed..26f3cd3949b 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -266,7 +266,7 @@ characteristics for a Timeseries. A metric stream is identified by: - The metric's associated Meter details (`name`, `version`, and `schema_url`) - The metric stream's `name`. - The attached `Attribute`s -- The metric stream's point kind (`Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) +- The metric stream's point kind (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) - Intrinsic properties of the point, including temporality and monotonicity It is possible (and likely) that more than one metric stream is created per From fc06d2e5a0a91c62c0e33e03af6425963bda1a6a Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Mon, 7 Feb 2022 12:43:19 -0800 Subject: [PATCH 17/41] add unit to identity for data points --- specification/metrics/datamodel.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 26f3cd3949b..4cca5a44737 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -266,7 +266,8 @@ characteristics for a Timeseries. A metric stream is identified by: - The metric's associated Meter details (`name`, `version`, and `schema_url`) - The metric stream's `name`. - The attached `Attribute`s -- The metric stream's point kind (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) +- The point kind (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) +- The metric stream's unit - Intrinsic properties of the point, including temporality and monotonicity It is possible (and likely) that more than one metric stream is created per From e9384f0f121fdac45481f88b2d966747e51d0ec5 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Wed, 9 Feb 2022 16:44:47 -0800 Subject: [PATCH 18/41] clarify data model uniqueness --- specification/metrics/datamodel.md | 102 ++++++++++++++++++----------- 1 file changed, 64 insertions(+), 38 deletions(-) diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 4a3e966d77d..6632b132197 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -233,7 +233,7 @@ consisting of several metadata properties: - Metric name - Attributes (dimensions) -- Kind of point (integer, floating point, etc) +- Value type of the point (integer, floating point, etc) - Unit of measurement The primary data of each timeseries are ordered (timestamp, value) points, with @@ -257,43 +257,69 @@ to map into, but is used as a reference throughout this document. ### OpenTelemetry Protocol data model -The OpenTelemetry protocol data model is composed of Metric data streams. These -streams are in turn composed of metric data points. Metric data streams -can be converted directly into Timeseries, and share the same identity -characteristics for a Timeseries. A metric stream is identified by: - -- The originating `Resource` -- The metric's associated Meter details (`name`, `version`, and `schema_url`) -- The metric stream's `name`. -- The attached `Attribute`s -- The point kind (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) -- The metric stream's unit -- Intrinsic properties of the point, including temporality and monotonicity - -It is possible (and likely) that more than one metric stream is created per -`Instrument` in the event model. - -Points with identical `name`, `Resource`, and Meter details but -different point kind or different point kind intrinsic properties -SHOULD be considered an "error state", due to semantic disagreement, -whether or not they use distinct `Attribute` values. - -Consumers of OpenTelemetry metrics data streams MAY reject data -containing the multiple point kinds for identical `name`, `Resource`, -and Meter details, even though the [Single-Writer](#single-writer) -rule is only broken when distinct `Attribute` values are involved. - -__Note: The OpenTelemetry SDK specification is written to explicitly -permit duplicate instrumentation registration, which means the -potential to produce the error state. This is considered to be in the -user's best interest. Although the SDK has permission to pass-through -duplicate instrument registration conflicts, consumers of -OpenTelemetry metrics are given equal permission to reject the data -because of semantic disagreement. - -**Note: The same `Resource`, `name` and `Attribute`s but differing point kind -coming out of an OpenTelemetry SDK is considered an "error state" that SHOULD -be handled by an SDK.** +The OpenTelemetry protocol (OTLP) data model is composed of Metric data +streams. These streams are in turn composed of metric data points. +Metric data streams can be converted directly into Timeseries. + +Metric streams are grouped into individual `Metric` objects, +identified by: + +- The originating `Resource` attributes +- The instrumentation `Scope` (e.g., instrumentation library name, version) +- The metric stream's `name` + +Aside from `name`, the `Metric` object is defined by the following +properties: + +- The data point type (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) +- The metric stream's `unit` +- The metric stream's `description` +- Intrinsic data point properties (e.g., `AggregationTemporality`, `Monotonic`) + +The data point type, `unit`, and intrinsic properties are considered +identifying, whereas the `description` field is explicitly not +identifying in nature. + +The `Metric` object contains individual streams, identified by the set +of `Attributes`. Within the individual streams, points are identified +by one or two timestamps, details vary by data point type. + +Within certain data point types (e.g., `Sum` and `Gauge`) there is +variation permitted in the numeric point value; in this case, the +associated variation (i.e., floating-point vs. integer) is not +considered identifying. + +#### Producer expectations + +The presence of multiple `Metric` identities for a given `name` with +the same `Resource` and `Scope` attributes is considered a conflict, a +"semantic error" is said to occur. + +Multiple `Metric` objects SHOULD be avoided by producers, if possible, +using one of several remedies: + +1. For identical definitions, the producer MUST aggregate the data + into a single `Metric` object to avoid the conflict +2. If the conflict involves a non-identifying property (i.e., + `description`), the producer SHOULD choose a single value. +3. For remaining types of conflict, if the OpenTelemetry Metrics SDK + Views configuration supports an action to correct the conflict, the + SDK SHOULD recommend a View configuration to mitigate the error. + +When no automatic remedy can be applied, producers SHOULD pass through +conflicting data containing semantic errors to be handled by the +consumer. + +#### Consumer recommendations + +Consumers MAY reject OpenTelemetry Metrics data containing semantic +errors (i.e., more than one `Metric` identity for a given `name`, +`Resource`, and `Scope`). + +OpenTelemetry does not specify any means for conveying such an outcome +to the end user. + +#### Point kinds A metric stream can use one of these basic point kinds, all of which satisfy the requirements above, meaning they define a decomposable From 40f8ed80b68107a8614fe63556917bd0e1fb3360 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Thu, 10 Feb 2022 00:29:30 -0800 Subject: [PATCH 19/41] clarify how to report semantic errors and when they can be remediated --- specification/metrics/datamodel.md | 82 ++++++++++++++++++++---------- specification/metrics/sdk.md | 62 ++++++++++------------ 2 files changed, 82 insertions(+), 62 deletions(-) diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 6632b132197..2d033032698 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -268,18 +268,24 @@ identified by: - The instrumentation `Scope` (e.g., instrumentation library name, version) - The metric stream's `name` -Aside from `name`, the `Metric` object is defined by the following +Including `name`, the `Metric` object is defined by the following properties: - The data point type (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`) - The metric stream's `unit` - The metric stream's `description` -- Intrinsic data point properties (e.g., `AggregationTemporality`, `Monotonic`) +- Intrinsic data point properties, where applicable: `AggregationTemporality`, `Monotonic` The data point type, `unit`, and intrinsic properties are considered identifying, whereas the `description` field is explicitly not identifying in nature. +Extrinsic properties of specific points are not considered +identifying; these include but are not limited to: + +- Bucket boundaries of a `Histogram` data point +- Scale or bucket count of a `ExponentialHistogram` data point. + The `Metric` object contains individual streams, identified by the set of `Attributes`. Within the individual streams, points are identified by one or two timestamps, details vary by data point type. @@ -289,35 +295,50 @@ variation permitted in the numeric point value; in this case, the associated variation (i.e., floating-point vs. integer) is not considered identifying. -#### Producer expectations - -The presence of multiple `Metric` identities for a given `name` with -the same `Resource` and `Scope` attributes is considered a conflict, a -"semantic error" is said to occur. - -Multiple `Metric` objects SHOULD be avoided by producers, if possible, -using one of several remedies: - -1. For identical definitions, the producer MUST aggregate the data - into a single `Metric` object to avoid the conflict -2. If the conflict involves a non-identifying property (i.e., - `description`), the producer SHOULD choose a single value. -3. For remaining types of conflict, if the OpenTelemetry Metrics SDK - Views configuration supports an action to correct the conflict, the - SDK SHOULD recommend a View configuration to mitigate the error. - -When no automatic remedy can be applied, producers SHOULD pass through -conflicting data containing semantic errors to be handled by the -consumer. - -#### Consumer recommendations +#### OpenTelemetry Protocol data model: Producer recomendations + +Producers SHOULD prevent the presence of multiple `Metric` identities +for a given `name` with the same `Resource` and `Scope` attributes. +Producers are expected to aggregate data for identical `Metric` +objects as a basic feature, so the appearance of multiple `Metric`, +considered a "semantic error", generally requires duplicate +conflicting instrument registration to have occurred somewhere. + +Producers MAY be able to remediate the problem, depending on whether +they are an SDK, a downstream processor, or a consumer at the +destination: + +1. If the potential conflict involves a non-identifying property (i.e., + `description`), the producer SHOULD choose the longer string. +2. If the potential conflict involves similar but disagreeing units + (e.g., "ms" and "s"), an implementation MAY convert units to avoid + semantic errors; otherwise an implementation SHOULD inform the user + of a semantic error and pass through both `Metric` objects. +3. If the potential conflict involves an `AggregationTemporality` + property, an implementation MAY convert temporality using a + Cumulative-to-Delta or a Delta-to-Cumulative transformation; + otherwise, an implementation SHOULD inform the user of a semantic + error and pass through both `Metric` objects. +4. Generally, for potential conflicts involving an identifying + property (i.e., all properties except `description`), the producer + SHOULD pass through the semantic error to the consumer and pass + through both `Metric` objects. + +When semantic errors such as these occur inside an implementation of +the OpenTelemetry API, there is an presumption of a fixed `Resource` +value. Consequently, SDKs implementing the OpenTelemetry API have +complete information about the origin of duplicate instrument +registration conflicts and are sometimes able to help users avoid +semantic errors. See the SDK specification for specific details. + +#### OpenTelemetry Protocol data model: Consumer recommendations Consumers MAY reject OpenTelemetry Metrics data containing semantic errors (i.e., more than one `Metric` identity for a given `name`, `Resource`, and `Scope`). OpenTelemetry does not specify any means for conveying such an outcome -to the end user. +to the end user, although this subject deserves attention. #### Point kinds @@ -700,7 +721,7 @@ For positive scales, lookup table methods have been demonstrated that are able to exactly compute the index in constant time from a lookup table with `O(2**scale)` entries. -##### Producer Recommendations +#### ExponentialHistogram: Producer Recommendations At the lowest or highest end of the 64 bit IEEE floating point, a bucket's range may only be partially representable by the floating @@ -721,7 +742,7 @@ perform an exact computation. As a result, ExponentialHistogram exemplars could map into buckets with zero count. We expect to find such values counted in the adjacent buckets. -#### Consumer Expectations +#### ExponentialHistogram: Consumer Recommendations ExponentialHistogram bucket indices are expected to map into buckets where both the upper and lower boundaries can be represented @@ -826,6 +847,13 @@ scenarios and take corrective actions. Additionally, it ensures that well-behaved systems can perform metric stream manipulation without undesired degradation or loss of visibility. +Note that violations of the Single-Writer principle are not semantic +errors, generally they result from misconfiguration. Whereas semantic +errors can sometimes be corrected by configuring Views, violations of +the Single-Writer principle can be corrected by differentiating the +`Resource` used or by ensuring that streams for a given `Resource` and +`Attribute` set do not overlap. + ## Temporality **Status**: [Stable](../document-status.md) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index d0d61f8af86..b46e17867cb 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -418,18 +418,33 @@ instruments that record negative measurements, e.g. `UpDownCounter` or `Observab ### Resolving duplicate instrument registration conflicts -Implementations are required by the API specification to tolerate -duplicate instrument registration conflicts. View configuration -offers the user an opportunity to configure a workaround in cases -where the conflict involves the instrument type. - -To help the user manage such conflicts, the implementation SHOULD warn -about dupliclate instrument registration conflicts after applying the -Views configuration. For conflicts that can be resolved through a -View configuration, which includes instrument-type conflicts but not -unit or description conflicts, the implementation's warning SHOULD -include an example suggesting a View configuration to resolve the -conflict. +Implementations are REQUIRED by the API specification to allow +duplicate instrument registration to occur, regardless of conflicting +identities. This ensures that user data is at least delivered to the +exporter, recognizing that such data MAY create a [semantic error +state](datamodel.md#opentelemetry-protocol-data-model-producer-expectations) +indicated by data having multiple `Metric` objects for a given +`Resource` and `Scope`. + +The SDK is required to issue warnings when instrumentation conflicts +arise, because they lead to semantic errors. To help the user manage +such conflicts, the implementation SHOULD warn about dupliclate +instrument registration conflicts after applying View configuration. + +View configuration cannot work around every conflict, and in some +cases the user will want to repair faulty instrumentation instead. +When a potential conflict arises between two `Metric` identities +having the same `name`: + +1. If the potential conflict involves a multiple `description` + properties, setting the `description` through a configured View + SHOULD avoid the warning. +2. If the potential conflict involves instruments that can be + distinguished by a supported View selector (e.g., instrument type) + a View recipe SHOULD be printed advising the user how to avoid the + warning by renaming one of the conflicting instruments. +3. Otherwise (e.g., use of multiple units), the implementation SHOULD + pass through the semantic error and report both `Metric` objects. ## Attribute limits @@ -969,26 +984,3 @@ called concurrently. **MetricExporter** - `ForceFlush` and `Shutdown` are safe to be called concurrently. - -## Data model requirements - -The [OpenTelemetry Metrics data -model](datamodel.md#opentelemetry-protocol-data-model) has a [Single -Writer](datamodel.md#single-writer) rule that the SDK is REQUIRED to -enforce, with one exception noted below. This rule is the origin of -the output-name uniqueness requirement for [Views](#view) and is the -reason the API specifies that implementations MUST aggregate data for -identical instruments in its pipeline. - -The implementation has permission to pass-through violations of the -semantic "error state" described in the [OpenTelemetry Metrics data -model](datamodel.md#opentelemetry-protocol-data-model). This is to -address the exceptional case of duplicate instrument registration -conflicts, which are tolerated with warnings in the API. - -Duplicate instrument registration conflicts can produce this semantic -error state, which may lead to violations of the single-writer rule. -SDKs are permitted to allow these conflicts to take place to avoid -user data loss. Since the semantic conflict may not be resolvable -downstream, Metrics data consumers MAY reject semantically conflicting -data and SHOULD notify the user of the error, somehow, in such cases. From faf19c14665c8e7beee90fbfd2ba55f47d5591ad Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 10:32:33 -0800 Subject: [PATCH 20/41] lint --- .../trace/tracestate-probability-sampling.md | 51 +++++++++---------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/specification/trace/tracestate-probability-sampling.md b/specification/trace/tracestate-probability-sampling.md index 700ec46c210..e531960fc3a 100644 --- a/specification/trace/tracestate-probability-sampling.md +++ b/specification/trace/tracestate-probability-sampling.md @@ -870,7 +870,7 @@ this a strict test for random behavior, we take the following approach: - Generate a pre-determined list of 20 random seeds - Use fixed values for significance level (5%) and trials (20) -- Use a population size of one hundred thousand spans +- Use a population size of 100,000 spans - For each trial, simulate the population and compute ChiSquared test statistic - Locate the first seed value in the ordered list such that the @@ -934,18 +934,17 @@ test statistic is below 0.102587 exactly 1 out of 20 times. In this case there is one degree of freedom for the Chi-Squared test. The following table summarizes the test parameters. -| Test case | Sampling probability | P-value when sampled | Expectsampled | Expectunsampled | | -|-----------|----------------------|----------------------|--------------------------|----------------------------|----------| -| 13 | 0x1p-01 (0.500000) | 1 | 50000 | n/a | 50000 | -| 14 | 0x1p-04 (0.062500) | 4 | 6250 | n/a | 93750 | -| 15 | 0x1p-07 (0.007812) | 7 | 781.25 | n/a | 99218.75 | - +| Test case | Sampling probability | P-value when sampled | Expectsampled | Expectunsampled | +|-----------|----------------------|----------------------|--------------------------|----------------------------| +| 13 | 0x1p-01 (0.500000) | 1 | 50000 | 50000 | +| 14 | 0x1p-04 (0.062500) | 4 | 6250 | 93750 | +| 15 | 0x1p-07 (0.007812) | 7 | 781.25 | 99218.75 | The formula for computing Chi-Squared in this case is: ``` ChiSquared = math.Pow(sampled - expect_sampled, 2) / expect_sampled + - math.Pow(1000000 - sampled - expect_unsampled, 2) / expect_unsampled + math.Pow(100000 - sampled - expect_unsampled, 2) / expect_unsampled ``` This should be compared with 0.003932, the value of the Chi-Squared @@ -969,24 +968,24 @@ predetermined list of seeds. ``` for _, test := range []testCase{ - // Non-powers of two - {0.90000, 1, twoDegrees, 3}, - {0.60000, 1, twoDegrees, 2}, - {0.33000, 2, twoDegrees, 2}, - {0.13000, 3, twoDegrees, 1}, - {0.10000, 4, twoDegrees, 0}, - {0.05000, 5, twoDegrees, 0}, - {0.01700, 6, twoDegrees, 2}, - {0.01000, 7, twoDegrees, 2}, - {0.00500, 8, twoDegrees, 2}, - {0.00290, 9, twoDegrees, 4}, - {0.00100, 10, twoDegrees, 6}, - {0.00050, 11, twoDegrees, 0}, - - // Powers of two - {0x1p-1, 1, oneDegree, 0}, - {0x1p-4, 4, oneDegree, 0}, - {0x1p-7, 7, oneDegree, 1}, + // Non-powers of two + {0.90000, 1, twoDegrees, 3}, + {0.60000, 1, twoDegrees, 2}, + {0.33000, 2, twoDegrees, 2}, + {0.13000, 3, twoDegrees, 1}, + {0.10000, 4, twoDegrees, 0}, + {0.05000, 5, twoDegrees, 0}, + {0.01700, 6, twoDegrees, 2}, + {0.01000, 7, twoDegrees, 2}, + {0.00500, 8, twoDegrees, 2}, + {0.00290, 9, twoDegrees, 4}, + {0.00100, 10, twoDegrees, 6}, + {0.00050, 11, twoDegrees, 0}, + + // Powers of two + {0x1p-1, 1, oneDegree, 0}, + {0x1p-4, 4, oneDegree, 0}, + {0x1p-7, 7, oneDegree, 1}, } { ``` From dfc5009e1cc84dbd55d6a51782a87ee1b1cc12ae Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 10:35:14 -0800 Subject: [PATCH 21/41] spelling --- specification/trace/tracestate-probability-sampling.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/trace/tracestate-probability-sampling.md b/specification/trace/tracestate-probability-sampling.md index e531960fc3a..816b289f0b3 100644 --- a/specification/trace/tracestate-probability-sampling.md +++ b/specification/trace/tracestate-probability-sampling.md @@ -923,7 +923,7 @@ For each probability in the table above, the test is required to demonstrate a seed that produces exactly one ChiSquared value less than 0.102587. -##### Requirement: Pass 15 non-power-of-two statistical tests +##### Requirement: Pass 12 non-power-of-two statistical tests For the test with 20 trials and 100,000 spans each, the test MUST demonstrate a random number generator seed such that the ChiSquared @@ -953,9 +953,9 @@ For each probability in the table above, the test is required to demonstrate a seed that produces exactly one ChiSquared value less than 0.003932. -##### Requirement: Pass 5 power-of-two statistical tests +##### Requirement: Pass 3 power-of-two statistical tests -For the teset with 20 trials and 100,000 spans each, the test MUST +For the test with 20 trials and 100,000 spans each, the test MUST demonstrate a random number generator seed such that the ChiSquared test statistic is below 0.003932 exactly 1 out of 20 times. From 841aa477e5cd752ac6e794b03b9ad0a42f08ddd2 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 10:35:49 -0800 Subject: [PATCH 22/41] spelling --- specification/trace/tracestate-probability-sampling.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/trace/tracestate-probability-sampling.md b/specification/trace/tracestate-probability-sampling.md index 816b289f0b3..83a1ecee205 100644 --- a/specification/trace/tracestate-probability-sampling.md +++ b/specification/trace/tracestate-probability-sampling.md @@ -72,9 +72,9 @@ - [Recommendation: Recognize inconsistent r-values](#recommendation-recognize-inconsistent-r-values) * [Appendix: Statistical test requirements](#appendix-statistical-test-requirements) + [Test procedure: non-powers of two](#test-procedure-non-powers-of-two) - - [Requirement: Pass 15 non-power-of-two statistical tests](#requirement-pass-15-non-power-of-two-statistical-tests) + - [Requirement: Pass 12 non-power-of-two statistical tests](#requirement-pass-12-non-power-of-two-statistical-tests) + [Test procedure: exact powers of two](#test-procedure-exact-powers-of-two) - - [Requirement: Pass 5 power-of-two statistical tests](#requirement-pass-5-power-of-two-statistical-tests) + - [Requirement: Pass 3 power-of-two statistical tests](#requirement-pass-3-power-of-two-statistical-tests) + [Test implementation](#test-implementation) - [Appendix](#appendix) * [Methods for generating R-values](#methods-for-generating-r-values) From 58e43358a4262f1cf057356acae8fb488d271c61 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 11:44:50 -0800 Subject: [PATCH 23/41] revert mistake unrelated --- .../trace/tracestate-probability-sampling.md | 92 ++++++++++--------- 1 file changed, 51 insertions(+), 41 deletions(-) diff --git a/specification/trace/tracestate-probability-sampling.md b/specification/trace/tracestate-probability-sampling.md index 83a1ecee205..6cad1c9ff82 100644 --- a/specification/trace/tracestate-probability-sampling.md +++ b/specification/trace/tracestate-probability-sampling.md @@ -72,9 +72,9 @@ - [Recommendation: Recognize inconsistent r-values](#recommendation-recognize-inconsistent-r-values) * [Appendix: Statistical test requirements](#appendix-statistical-test-requirements) + [Test procedure: non-powers of two](#test-procedure-non-powers-of-two) - - [Requirement: Pass 12 non-power-of-two statistical tests](#requirement-pass-12-non-power-of-two-statistical-tests) + - [Requirement: Pass 15 non-power-of-two statistical tests](#requirement-pass-15-non-power-of-two-statistical-tests) + [Test procedure: exact powers of two](#test-procedure-exact-powers-of-two) - - [Requirement: Pass 3 power-of-two statistical tests](#requirement-pass-3-power-of-two-statistical-tests) + - [Requirement: Pass 5 power-of-two statistical tests](#requirement-pass-5-power-of-two-statistical-tests) + [Test implementation](#test-implementation) - [Appendix](#appendix) * [Methods for generating R-values](#methods-for-generating-r-values) @@ -870,7 +870,7 @@ this a strict test for random behavior, we take the following approach: - Generate a pre-determined list of 20 random seeds - Use fixed values for significance level (5%) and trials (20) -- Use a population size of 100,000 spans +- Use a population size of one million spans - For each trial, simulate the population and compute ChiSquared test statistic - Locate the first seed value in the ordered list such that the @@ -895,19 +895,22 @@ In this case there are two degrees of freedom for the Chi-Squared test. The following table summarizes the test parameters. | Test case | Sampling probability | Lower, Upper p-value when sampled | Expectlower | Expectupper | Expectunsampled | -|-----------|----------------------|-----------------------------------|------------------------|------------------------|----------------------------| -| 1 | 0.900000 | 0, 1 | 10000 | 80000 | 10000 | -| 2 | 0.600000 | 0, 1 | 40000 | 20000 | 40000 | -| 3 | 0.330000 | 1, 2 | 17000 | 16000 | 67000 | -| 4 | 0.130000 | 2, 3 | 12000 | 1000 | 87000 | -| 5 | 0.100000 | 3, 4 | 2500 | 7500 | 90000 | -| 6 | 0.050000 | 4, 5 | 1250 | 3750 | 95000 | -| 7 | 0.017000 | 5, 6 | 1425 | 275 | 98300 | -| 8 | 0.010000 | 6, 7 | 562.5 | 437.5 | 99000 | -| 9 | 0.005000 | 7, 8 | 281.25 | 218.75 | 99500 | -| 10 | 0.002900 | 8, 9 | 100.625 | 189.375 | 99710 | -| 11 | 0.001000 | 9, 10 | 95.3125 | 4.6875 | 99900 | -| 12 | 0.000500 | 10, 11 | 47.65625 | 2.34375 | 99950 | +| --- | --- | --- | --- | --- | --- | +| 1 | 0.900000 | 0, 1 | 100000 | 800000 | 100000 | +| 2 | 0.600000 | 0, 1 | 400000 | 200000 | 400000 | +| 3 | 0.330000 | 1, 2 | 170000 | 160000 | 670000 | +| 4 | 0.130000 | 2, 3 | 120000 | 10000 | 870000 | +| 5 | 0.100000 | 3, 4 | 25000 | 75000 | 900000 | +| 6 | 0.050000 | 4, 5 | 12500 | 37500 | 950000 | +| 7 | 0.017000 | 5, 6 | 14250 | 2750 | 983000 | +| 8 | 0.010000 | 6, 7 | 5625 | 4375 | 990000 | +| 9 | 0.005000 | 7, 8 | 2812.5 | 2187.5 | 995000 | +| 10 | 0.002900 | 8, 9 | 1006.25 | 1893.75 | 997100 | +| 11 | 0.001000 | 9, 10 | 953.125 | 46.875 | 999000 | +| 12 | 0.000500 | 10, 11 | 476.5625 | 23.4375 | 999500 | +| 13 | 0.000260 | 11, 12 | 228.28125 | 31.71875 | 999740 | +| 14 | 0.000230 | 12, 13 | 14.140625 | 215.859375 | 999770 | +| 15 | 0.000100 | 13, 14 | 22.0703125 | 77.9296875 | 999900 | The formula for computing Chi-Squared in this case is: @@ -923,9 +926,9 @@ For each probability in the table above, the test is required to demonstrate a seed that produces exactly one ChiSquared value less than 0.102587. -##### Requirement: Pass 12 non-power-of-two statistical tests +##### Requirement: Pass 15 non-power-of-two statistical tests -For the test with 20 trials and 100,000 spans each, the test MUST +For the test with 20 trials and 1 million spans each, the test MUST demonstrate a random number generator seed such that the ChiSquared test statistic is below 0.102587 exactly 1 out of 20 times. @@ -934,17 +937,19 @@ test statistic is below 0.102587 exactly 1 out of 20 times. In this case there is one degree of freedom for the Chi-Squared test. The following table summarizes the test parameters. -| Test case | Sampling probability | P-value when sampled | Expectsampled | Expectunsampled | -|-----------|----------------------|----------------------|--------------------------|----------------------------| -| 13 | 0x1p-01 (0.500000) | 1 | 50000 | 50000 | -| 14 | 0x1p-04 (0.062500) | 4 | 6250 | 93750 | -| 15 | 0x1p-07 (0.007812) | 7 | 781.25 | 99218.75 | +| Test case | Sampling probability | P-value when sampled | Expectsampled | Expectunsampled | | +| --- | --- | --- | --- | --- | | +| 16 | 0x1p-01 (0.500000) | 1 | 500000 | n/a | 500000 | +| 17 | 0x1p-04 (0.062500) | 4 | 62500 | n/a | 937500 | +| 18 | 0x1p-07 (0.007812) | 7 | 7812.5 | n/a | 992187.5 | +| 19 | 0x1p-10 (0.000977) | 10 | 976.5625 | n/a | 999023.4375 | +| 20 | 0x1p-13 (0.000122) | 13 | 122.0703125 | n/a | 999877.9297 | The formula for computing Chi-Squared in this case is: ``` ChiSquared = math.Pow(sampled - expect_sampled, 2) / expect_sampled + - math.Pow(100000 - sampled - expect_unsampled, 2) / expect_unsampled + math.Pow(1000000 - sampled - expect_unsampled, 2) / expect_unsampled ``` This should be compared with 0.003932, the value of the Chi-Squared @@ -953,15 +958,15 @@ For each probability in the table above, the test is required to demonstrate a seed that produces exactly one ChiSquared value less than 0.003932. -##### Requirement: Pass 3 power-of-two statistical tests +##### Requirement: Pass 5 power-of-two statistical tests -For the test with 20 trials and 100,000 spans each, the test MUST +For the teset with 20 trials and 1 million spans each, the test MUST demonstrate a random number generator seed such that the ChiSquared test statistic is below 0.003932 exactly 1 out of 20 times. #### Test implementation -The recommended structure for this test uses a table listing the 15 +The recommended structure for this test uses a table listing the 20 probability values, the expected p-values, whether the ChiSquared statistic has one or two degrees of freedom, and the index into the predetermined list of seeds. @@ -969,30 +974,35 @@ predetermined list of seeds. ``` for _, test := range []testCase{ // Non-powers of two - {0.90000, 1, twoDegrees, 3}, - {0.60000, 1, twoDegrees, 2}, - {0.33000, 2, twoDegrees, 2}, - {0.13000, 3, twoDegrees, 1}, + {0.90000, 1, twoDegrees, 5}, + {0.60000, 1, twoDegrees, 14}, + {0.33000, 2, twoDegrees, 3}, + {0.13000, 3, twoDegrees, 2}, {0.10000, 4, twoDegrees, 0}, {0.05000, 5, twoDegrees, 0}, {0.01700, 6, twoDegrees, 2}, - {0.01000, 7, twoDegrees, 2}, - {0.00500, 8, twoDegrees, 2}, - {0.00290, 9, twoDegrees, 4}, - {0.00100, 10, twoDegrees, 6}, - {0.00050, 11, twoDegrees, 0}, + {0.01000, 7, twoDegrees, 3}, + {0.00500, 8, twoDegrees, 1}, + {0.00290, 9, twoDegrees, 1}, + {0.00100, 10, twoDegrees, 5}, + {0.00050, 11, twoDegrees, 1}, + {0.00026, 12, twoDegrees, 3}, + {0.00023, 13, twoDegrees, 0}, + {0.00010, 14, twoDegrees, 2}, // Powers of two {0x1p-1, 1, oneDegree, 0}, - {0x1p-4, 4, oneDegree, 0}, - {0x1p-7, 7, oneDegree, 1}, + {0x1p-4, 4, oneDegree, 2}, + {0x1p-7, 7, oneDegree, 3}, + {0x1p-10, 10, oneDegree, 0}, + {0x1p-13, 13, oneDegree, 1}, } { ``` Note that seed indexes in the example above have what appears to be -the correct distribution. The five 0s, two 1s, five 2s, one 3s, and -one 4 demonstrate that it is relatively easy to find examples where -there is exactly one failure. Probability 0.001, with seed index 6 in +the correct distribution. The five 0s, four 1s, four 2s, four 3s, and +two 5s demonstrate that it is relatively easy to find examples where +there is exactly one failure. Seed index 14, for probability 0.6 in this case, is a reminder that outliers exist. Further significance testing of this distribution is not recommended. From 8e93d28227d33ef2c29a7d87199f6c2d34de8301 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 13:38:53 -0800 Subject: [PATCH 24/41] identical and distinct, respectively --- specification/metrics/api.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index c1b76f71cdd..6db64c475fb 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -132,10 +132,10 @@ This API MUST accept the following parameters: Meters are identified by all of these fields. When more than one Meter of the same `name`, `version`, and `schema_url` is created, it is unspecified whether or under which conditions the same or different -`Meter` instances are returned. Meter identity, and the terms -_identical_ and _distinct_ applied to Meters in this specification -describes Meter instances that have identical and/or distinct values -for all three of their `name`, `version`, and `schema_url` attributes. +`Meter` instances are returned. The term _identical_ applied to +Meters describes instances where all identifying fields are equal. +The term _distinct_ applied to Meters describes instances where at +least one identifying field has a different value. Implementations MUST NOT require users to repeatedly obtain a `Meter` with the same identity to pick up configuration changes. This can be @@ -188,6 +188,9 @@ will have the following information: Instruments are associated with the Meter during creation. Instruments are identified by all of these fields. +Language-level features such as the distinction between integer and +floating point numbers SHOULD be considered as identifying. + When more than one Instrument of the same `name` is created for @@ -196,13 +199,10 @@ implementation MUST return a valid Instrument in every case. It is unspecified whether or under which conditions the same or different Instrument instance will be returned as a result of -duplicate instrument registration. Instrument identity, and the terms -_identical_ and _distinct_ applied to Instruments in this -specification describes Instrument instances that have identical -and/or distinct values for all four of their `name`, `kind`, `unit`, -and `description` attributes. Language-level features such as the -distinction between integer and floating point numbers SHOULD be -considered as identifying, as well. +duplicate instrument registration. The term _identical_ applied to +Instruments describes instances where all identifying fields are +equal. The term _distinct_ applied to Instruments describes instances +where at least one field value is different. The implementation MUST aggregate data from identical Instruments together in its export pipeline. From be56745003cce8861b8fdc6cace163401f142c02 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 15:02:47 -0800 Subject: [PATCH 25/41] refer to API/datamodel spec in SDK spec for conflict res --- specification/metrics/api.md | 8 +++++-- specification/metrics/datamodel.md | 28 ++++++++++++----------- specification/metrics/sdk.md | 36 +++++++++++++----------------- 3 files changed, 36 insertions(+), 36 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 6db64c475fb..d237ffd786d 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -177,7 +177,7 @@ Also see the respective sections below for more information on instrument creati ## Instrument Instruments are used to report [Measurements](#measurement). Each Instrument -will have the following information: +will have the following fields: * The `name` of the Instrument * The `kind` of the Instrument - whether it is a [Counter](#counter) or @@ -195,7 +195,11 @@ floating point numbers SHOULD be considered as identifying. When more than one Instrument of the same `name` is created for identical Meters, denoted _duplicate instrument registration_, the -implementation MUST return a valid Instrument in every case. +implementation MUST create a valid Instrument in every case. Here, +"valid" means an instrument that is functional and can expected to +export data, despite potentially creating a [semantic error in the +data +model](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations). It is unspecified whether or under which conditions the same or different Instrument instance will be returned as a result of diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 2d033032698..9f8ba847228 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -15,6 +15,9 @@ * [Event Model](#event-model) * [Timeseries Model](#timeseries-model) * [OpenTelemetry Protocol data model](#opentelemetry-protocol-data-model) + + [OpenTelemetry Protocol data model: Producer recommendations](#opentelemetry-protocol-data-model-producer-recommendations) + + [OpenTelemetry Protocol data model: Consumer recommendations](#opentelemetry-protocol-data-model-consumer-recommendations) + + [Point kinds](#point-kinds) - [Metric points](#metric-points) * [Sums](#sums) * [Gauge](#gauge) @@ -28,8 +31,8 @@ - [Negative Scale: Extract and Shift the Exponent](#negative-scale-extract-and-shift-the-exponent) - [All Scales: Use the Logarithm Function](#all-scales-use-the-logarithm-function) - [Positive Scale: Use a Lookup Table](#positive-scale-use-a-lookup-table) - - [Producer Recommendations](#producer-recommendations) - + [Consumer Expectations](#consumer-expectations) + + [ExponentialHistogram: Producer Recommendations](#exponentialhistogram-producer-recommendations) + + [ExponentialHistogram: Consumer Recommendations](#exponentialhistogram-consumer-recommendations) * [Summary (Legacy)](#summary-legacy) - [Exemplars](#exemplars) - [Single-Writer](#single-writer) @@ -295,7 +298,7 @@ variation permitted in the numeric point value; in this case, the associated variation (i.e., floating-point vs. integer) is not considered identifying. -#### OpenTelemetry Protocol data model: Producer recomendations +#### OpenTelemetry Protocol data model: Producer recommendations Producers SHOULD prevent the presence of multiple `Metric` identities for a given `name` with the same `Resource` and `Scope` attributes. @@ -305,24 +308,23 @@ considered a "semantic error", generally requires duplicate conflicting instrument registration to have occurred somewhere. Producers MAY be able to remediate the problem, depending on whether -they are an SDK, a downstream processor, or a consumer at the -destination: +they are an SDK or a downstream processor: 1. If the potential conflict involves a non-identifying property (i.e., `description`), the producer SHOULD choose the longer string. 2. If the potential conflict involves similar but disagreeing units (e.g., "ms" and "s"), an implementation MAY convert units to avoid semantic errors; otherwise an implementation SHOULD inform the user - of a semantic error and pass through both `Metric` objects. + of a semantic error and pass through conflicting data. 3. If the potential conflict involves an `AggregationTemporality` property, an implementation MAY convert temporality using a Cumulative-to-Delta or a Delta-to-Cumulative transformation; otherwise, an implementation SHOULD inform the user of a semantic - error and pass through both `Metric` objects. + error and pass through conflicting data. 4. Generally, for potential conflicts involving an identifying property (i.e., all properties except `description`), the producer - SHOULD pass through the semantic error to the consumer and pass - through both `Metric` objects. + SHOULD inform the user of a semantic error and pass through + conflicting data. When semantic errors such as these occur inside an implementation of the OpenTelemetry API, there is an presumption of a fixed `Resource` @@ -817,11 +819,11 @@ All metric data streams within OTLP MUST have one logical writer. This means, conceptually, that any Timeseries created from the Protocol MUST have one originating source of truth. In practical terms, this implies the following: -- All metric data streams produced by OTel SDKs MUST be globally uniquely - produced and free from duplicates. All metric data streams can be uniquely - identified in some way. +- All metric data streams produced by OTel SDKs SHOULD have globally + unique identity at any given point in time. [`Metric` identity is defined + above.](#opentelemetry-protocol-data-model-producer-recommendations) - Aggregations of metric streams MUST only be written from a single logical - source. + source at any given point time. **Note: This implies aggregated metric streams must reach one destination**. In systems, there is the possibility of multiple writers sending data for the diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index f3dcd88f254..f8ee3c17826 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -41,7 +41,6 @@ - [Numerical limits handling](#numerical-limits-handling) - [Compatibility requirements](#compatibility-requirements) - [Concurrency requirements](#concurrency-requirements) -- [Data model requirements](#data-model-requirements) @@ -421,25 +420,20 @@ instruments that record negative measurements, e.g. `UpDownCounter` or `Observab ### Resolving duplicate instrument registration conflicts -Implementations are REQUIRED by the API specification to allow -duplicate instrument registration to occur, regardless of conflicting -identities. This ensures that user data is at least delivered to the -exporter, recognizing that such data MAY create a [semantic error -state](datamodel.md#opentelemetry-protocol-data-model-producer-expectations) -indicated by data having multiple `Metric` objects for a given -`Resource` and `Scope`. - -The SDK is required to issue warnings when instrumentation conflicts -arise, because they lead to semantic errors. To help the user manage -such conflicts, the implementation SHOULD warn about dupliclate -instrument registration conflicts after applying View configuration. - -View configuration cannot work around every conflict, and in some -cases the user will want to repair faulty instrumentation instead. -When a potential conflict arises between two `Metric` identities -having the same `name`: - -1. If the potential conflict involves a multiple `description` +As [stated in the API +specification](api.md#instrument-type-conflict-detection), +implemetations are REQUIRED to create valid instruments in case of +duplicate instrument registration, and the [data model includes +RECOMMENDATIONS on how to treat the consequent duplicate +conflicting](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations) +`Metric` definitions. + +The implementation SHOULD assist the user in managing conflicts by +reporting each duplicate-conflicting instrument registration that was +not corrected by a View as follows. When a potential conflict arises +between two non-identical `Metric` instances having the same `name`: + +1. If the potential conflict involves multiple `description` properties, setting the `description` through a configured View SHOULD avoid the warning. 2. If the potential conflict involves instruments that can be @@ -447,7 +441,7 @@ having the same `name`: a View recipe SHOULD be printed advising the user how to avoid the warning by renaming one of the conflicting instruments. 3. Otherwise (e.g., use of multiple units), the implementation SHOULD - pass through the semantic error and report both `Metric` objects. + pass through the data by reporting both `Metric` objects. ## Attribute limits From 58d20d7e2cdc7b683127744c569a749e20cc84df Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 15:53:22 -0800 Subject: [PATCH 26/41] try to clarifyt async observation APIs --- specification/metrics/api.md | 40 +++++++++++++++--------------- specification/metrics/datamodel.md | 2 +- specification/metrics/sdk.md | 15 +++++++++++ 3 files changed, 36 insertions(+), 21 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index d237ffd786d..e9ba5f904e8 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -215,12 +215,12 @@ When more than one distinct Instrument is registered with the same `name` for identical Meters, the implementation SHOULD emit a warning to the user informing them of duplicate registration conflict(s). -__Note the warning about duplicate instrumentation registration -conflicts is meant to help avoid the semantic error state described in -the [OpenTelemetry Metrics data -model](datamodel.md#opentelemetry-data-model) when more than one kind -of point is written for a given instrument `name` and Meter identity -from the same process. +__Note the warning about duplicate Instrument registration conflicts +is meant to help avoid the semantic error state described in the +[OpenTelemetry Metrics data +model](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations) +when more than one `Metric` is written for a given instrument `name` +and Meter identity by the same MeterProvider. @@ -278,7 +278,7 @@ instrument. It MUST be treated as an opaque string from the API and SDK. * It MUST support at least 1023 characters. [OpenTelemetry API](../overview.md#api) authors MAY decide if they want to support more. -Instruments are categorized based on whether they are synchronous or +Instruments are categorized on whether they are synchronous or asynchronous: @@ -303,24 +303,24 @@ Please note that the term *synchronous* and *asynchronous* have nothing to do with the [asynchronous pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). -Considering duplicate registration of asynchronous instruments, the -implementation MUST support multiple callbacks registered per -identical instrument. +The API MUST support creation of asynchronous instruments by passing a +single callback function to be registered to the newly created +instrument. -The API MAY provide support for registration of a single callback -having multiple asynchronous instruments, where idiomatic. The -implementation is REQUIRED to maintain a mapping from every registered -callback to the associated instrument(s). +The API SHOULD support creation of asynchronous instruments +by passing a variable number of callback functions, greater than or +equal to zero, to be registered to the newly created instrument. -The implementation MUST provide a way to unregister callbacks -associated with asynchronous instruments. +The API SHOULD support registration of callbacks bound to +one or more instruments outside of instrument construcotrs, provided +the API declaratively states which instrument(s) will be be used. -The implementation SHOULD reject observations that are made outside of -a callback registered for the instrument. +The API SHOULD support unregistration of callbacks. Callers SHOULD avoid making duplicate observations from asynchronous -instrument callbacks, as the result of making duplicate observations -is not specified. +instrument callbacks. The resulting behavior when a callback observes +multiple values for identical instrument and attributes is explicitly +not specified. ### Counter diff --git a/specification/metrics/datamodel.md b/specification/metrics/datamodel.md index 9f8ba847228..a4da3e7dc04 100644 --- a/specification/metrics/datamodel.md +++ b/specification/metrics/datamodel.md @@ -854,7 +854,7 @@ errors, generally they result from misconfiguration. Whereas semantic errors can sometimes be corrected by configuring Views, violations of the Single-Writer principle can be corrected by differentiating the `Resource` used or by ensuring that streams for a given `Resource` and -`Attribute` set do not overlap. +`Attribute` set do not overlap in time. ## Temporality diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index f8ee3c17826..324994ed043 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -19,6 +19,7 @@ + [Last Value Aggregation](#last-value-aggregation) + [Histogram Aggregation](#histogram-aggregation) + [Explicit Bucket Histogram Aggregation](#explicit-bucket-histogram-aggregation) + * [Observations inside asynchronous callbacks](#observations-inside-asynchronous-callbacks) * [Resolving duplicate instrument registration conflicts](#resolving-duplicate-instrument-registration-conflicts) - [Attribute limits](#attribute-limits) - [Exemplar](#exemplar) @@ -418,6 +419,20 @@ instruments that record negative measurements, e.g. `UpDownCounter` or `Observab - Min (optional) `Measurement` value in population. - Max (optional) `Measurement` value in population. +### Observations inside asynchronous callbacks + +Callback functions MUST be invoked in the context of a specific +`MetricReader` performing collection, such that observations made or +produced by executing callbacks only apply to the intended +`MetricReader` during collection. + +The implementation SHOULD disregard the accidental use of APIs +appurtenant to asynchronous instruments outside of registered +callbacks in the context of a single `MetricReader` collection. + +The implementation MUST complete the execution of all callbacks for a +given instrument before starting a subsequent round of collection. + ### Resolving duplicate instrument registration conflicts As [stated in the API From 421ddbca9c0d4ad451c983ba83610f7c879ad4d5 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 11 Feb 2022 16:08:17 -0800 Subject: [PATCH 27/41] edits --- specification/metrics/api.md | 37 +++++++++--------------------------- specification/metrics/sdk.md | 3 +++ 2 files changed, 12 insertions(+), 28 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index e9ba5f904e8..c6f32591f2f 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -317,7 +317,9 @@ the API declaratively states which instrument(s) will be be used. The API SHOULD support unregistration of callbacks. -Callers SHOULD avoid making duplicate observations from asynchronous +Callback functions SHOULD NOT take an indefinite amount of time. + +Callback functions SHOULD NOT make duplicate observations from asynchronous instrument callbacks. The resulting behavior when a callback observes multiple values for identical instrument and attributes is explicitly not specified. @@ -454,12 +456,9 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). +* Zero or more `callback` functions. [See the general requirements](#asynchronous-instrument). -The API MUST support an idiomatic way to associate one or more -`callback` functions with each instrument, whether through the -instrument constructor or a separate registration method. - -The `callback` function is responsible for reporting the +The `callback` function is responsible for reporting [Measurement](#measurement)s. It will only be called when the Meter is being observed. [OpenTelemetry API](../overview.md#api) authors SHOULD define whether this callback function needs to be reentrant safe / thread safe or not. @@ -469,10 +468,6 @@ callback function reports the absolute value of the counter. To determine the reported rate the counter is changing, the difference between successive measurements is used. -The callback function SHOULD NOT take indefinite amount of time. If multiple -independent SDKs coexist in a running process, they MUST invoke the callback -function(s) independently. - [OpenTelemetry API](../overview.md#api) authors MAY decide what is the idiomatic approach. Here are some examples: @@ -674,20 +669,13 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). +* Zero or more `callback` functions. [See the general requirements](#asynchronous-instrument). -The API MUST support an idiomatic way to associate one or more -`callback` functions with each instrument, whether through the -instrument constructor or a separate registration method. - -The `callback` function is responsible for reporting the +The `callback` function is responsible for reporting [Measurement](#measurement)s. It will only be called when the Meter is being observed. [OpenTelemetry API](../overview.md#api) authors SHOULD define whether this callback function needs to be reentrant safe / thread safe or not. -The callback function SHOULD NOT take indefinite amount of time. If multiple -independent SDKs coexist in a running process, they MUST invoke the callback -function(s) independently. - [OpenTelemetry API](../overview.md#api) authors MAY decide what is the idiomatic approach. Here are some examples: @@ -949,12 +937,9 @@ The API MUST accept the following parameters: rule](#instrument-unit). * An optional `description`, following the [instrument description rule](#instrument-description). +* Zero or more `callback` functions. [See the general requirements](#asynchronous-instrument). -The API MUST support an idiomatic way to associate one or more -`callback` functions with each instrument, whether through the -instrument constructor or a separate registration method. - -The `callback` function is responsible for reporting the +The `callback` function is responsible for reporting [Measurement](#measurement)s. It will only be called when the Meter is being observed. [OpenTelemetry API](../overview.md#api) authors SHOULD define whether this callback function needs to be reentrant safe / thread safe or not. @@ -964,10 +949,6 @@ the callback function reports the absolute value of the Asynchronous UpDownCounter. To determine the reported rate the Asynchronous UpDownCounter is changing, the difference between successive measurements is used. -The callback function SHOULD NOT take indefinite amount of time. If multiple -independent SDKs coexist in a running process, they MUST invoke the callback -function(s) independently. - [OpenTelemetry API](../overview.md#api) authors MAY decide what is the idiomatic approach. Here are some examples: diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 324994ed043..ec573f010d2 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -430,6 +430,9 @@ The implementation SHOULD disregard the accidental use of APIs appurtenant to asynchronous instruments outside of registered callbacks in the context of a single `MetricReader` collection. +The implementation SHOULD use a timeout to prevent indefinite callback +execution. + The implementation MUST complete the execution of all callbacks for a given instrument before starting a subsequent round of collection. From 8e56ae61a4d850137be161a8c208d24939cf1de2 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Mon, 14 Feb 2022 11:51:17 -0800 Subject: [PATCH 28/41] Update specification/metrics/api.md Co-authored-by: Marc Pichler --- specification/metrics/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index c6f32591f2f..75a1725343f 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -312,7 +312,7 @@ by passing a variable number of callback functions, greater than or equal to zero, to be registered to the newly created instrument. The API SHOULD support registration of callbacks bound to -one or more instruments outside of instrument construcotrs, provided +one or more instruments outside of instrument constructors, provided the API declaratively states which instrument(s) will be be used. The API SHOULD support unregistration of callbacks. From 16b1be0d8552da161fb85d6b3df44d76bb9450b6 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Mon, 14 Feb 2022 14:18:32 -0800 Subject: [PATCH 29/41] example unregister in python --- specification/metrics/api.md | 86 ++++++++++++++++++++++++++++++++---- 1 file changed, 78 insertions(+), 8 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index c6f32591f2f..6fc08d19305 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -315,7 +315,11 @@ The API SHOULD support registration of callbacks bound to one or more instruments outside of instrument construcotrs, provided the API declaratively states which instrument(s) will be be used. -The API SHOULD support unregistration of callbacks. +The API SHOULD support unregistration of callbacks, in order to cease +making observations. Note that unregistration of callbacks does not +undo the effect of instrument registration itself; duplicate +instrument registration conflicts SHOULD still occur for instruments +with no registered callbacks. Callback functions SHOULD NOT take an indefinite amount of time. @@ -538,10 +542,32 @@ meter.CreateObservableCounter("caesium_oscillates", () => clock.GetCaesi #### Asynchronous Counter operations -Asynchronous Counter is only intended for an asynchronous scenario. The only -operation is provided by the `callback`, which is registered during the +Asynchronous Counter uses an idiomatic interface for reporting +measurements through a `callback`, which is registered during [Asynchronous Counter creation](#asynchronous-counter-creation). +The API SHOULD provide support for unregistering callbacks. For +example: + +```python +# Python +class Device: + """A device with one counter""" + + def __init__(self, meter, x): + self.x = x + self.cb = meter.create_observable_counter(name="usage", description="count of items used", self.counter_callback) + + def counter_callback(self, result): + result.Observe(self.read_counter(), {'x', self.x}) + + def read_counter(self): + return 100 # ... + + def stop(self): + self.cb.unregister() +``` + ### Histogram `Histogram` is a [synchronous Instrument](#synchronous-instrument) which can be @@ -749,10 +775,32 @@ meter.CreateObservableGauge("temperature", () => sensor.GetTemperature() #### Asynchronous Gauge operations -Asynchronous Gauge is only intended for an asynchronous scenario. The only -operation is provided by the `callback`, which is registered during the +Asynchronous Gauge uses an idiomatic interface for reporting +measurements through a `callback`, which is registered during [Asynchronous Gauge creation](#asynchronous-gauge-creation). +The API SHOULD provide support for unregistering callbacks. For +example: + +```python +# Python +class Device: + """A device with one gauge""" + + def __init__(self, meter, x): + self.x = x + self.cb = meter.create_observable_gauge(name="pressure", description="force/area", self.gauge_callback) + + def gauge_callback(self, result): + result.Observe(self.read_gauge(), {'x', self.x}) + + def read_gauge(self): + return 100 # ... + + def stop(self): + self.cb.unregister() +``` + ### UpDownCounter `UpDownCounter` is a [synchronous Instrument](#synchronous-instrument) which @@ -1021,9 +1069,31 @@ meter.CreateObservableUpDownCounter("memory.physical.free", () => WMI.Qu #### Asynchronous UpDownCounter operations -Asynchronous UpDownCounter is only intended for an asynchronous scenario. The -only operation is provided by the `callback`, which is registered during the -[Asynchronous UpDownCounter creation](#asynchronous-updowncounter-creation). +Asynchronous Updowncounter uses an idiomatic interface for reporting +measurements through a `callback`, which is registered during +[Asynchronous Updowncounter creation](#asynchronous-updowncounter-creation). + +The API SHOULD provide support for unregistering callbacks. For +example: + +```python +# Python +class Device: + """A device with one updowncounter""" + + def __init__(self, meter, x): + self.x = x + self.cb = meter.create_observable_updowncounter(name="queue_size", description="items in process", self.updowncounter_callback) + + def updowncounter_callback(self, result): + result.Observe(self.read_updowncounter(), {'x', self.x}) + + def read_updowncounter(self): + return 100 # ... + + def stop(self): + self.cb.unregister() +``` ## Measurement From 5e77317ba083c11fb2ff9908843e778d6fe0fcce Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Mon, 14 Feb 2022 14:47:50 -0800 Subject: [PATCH 30/41] let multi-instrument be MAY --- specification/metrics/api.md | 46 ++++++++++++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 174d03fef2c..e1ea16f0c49 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -311,15 +311,17 @@ The API SHOULD support creation of asynchronous instruments by passing a variable number of callback functions, greater than or equal to zero, to be registered to the newly created instrument. -The API SHOULD support registration of callbacks bound to +The API MAY support registration of callbacks bound to one or more instruments outside of instrument constructors, provided -the API declaratively states which instrument(s) will be be used. +the API declaratively states which instrument(s) will be be used from +each callback. The API SHOULD support unregistration of callbacks, in order to cease making observations. Note that unregistration of callbacks does not undo the effect of instrument registration itself; duplicate instrument registration conflicts SHOULD still occur for instruments -with no registered callbacks. +with no registered callbacks. The effect of unregistering a callback +more than once is unspecified. Callback functions SHOULD NOT take an indefinite amount of time. @@ -1106,10 +1108,44 @@ for the interaction between the API and SDK. * A value * [`Attributes`](../common/common.md#attributes) +## Multi-instrument asynchronous callback example + +Where idiomatic, [the Metrics API supports the use of multiple +instruments from a single registered +callback](#asynchronous-instrument). Provided there is runtime +support for dynamic scope (e.g., thread-local state, context +variables), this can be achieved by an API similar to the synchronous +instruments. For example, + +```Python +# Python +class Device: + """A device with one counter""" + + def __init__(self, meter, x): + self.x = x + self.usage = meter.create_observable_counter(name="usage", description="count of items used") + self.pressure = meter.create_observable_gauge(name="pressure", description="force per unit area") + self.cb = meter.register_callback([self.usage, self.pressure], self.observe) + + def observe(self): + self.usage.observe(self.read_usage(), {'x', self.x}) + self.pressure.observe(self.read_pressure(), {'x', self.x}) + + def read_usage(self): + return 100 # ... + + def read_pressure(self): + return 10 # ... + + def stop(self): + self.cb.unregister() +``` + ## Compatibility requirements -All the metrics components SHOULD allow new APIs to be added to existing -components without introducing breaking changes. +All the metrics components SHOULD allow new APIs to be added to +existing components without introducing breaking changes. All the metrics APIs SHOULD allow optional parameter(s) to be added to existing APIs without introducing breaking changes, if possible. From 33d6c47f50e33ffddcdee9b65405c0826e89c8e6 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 10:56:27 -0800 Subject: [PATCH 31/41] Update specification/metrics/api.md Co-authored-by: Aaron Abbott --- specification/metrics/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index e1ea16f0c49..c25e145aaae 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -1071,7 +1071,7 @@ meter.CreateObservableUpDownCounter("memory.physical.free", () => WMI.Qu #### Asynchronous UpDownCounter operations -Asynchronous Updowncounter uses an idiomatic interface for reporting +Asynchronous UpDownCounter uses an idiomatic interface for reporting measurements through a `callback`, which is registered during [Asynchronous Updowncounter creation](#asynchronous-updowncounter-creation). From 3b13c12c99349d6d53e0dc44a65725d38ce1b646 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 10:57:09 -0800 Subject: [PATCH 32/41] Update specification/metrics/api.md Co-authored-by: Aaron Abbott --- specification/metrics/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index c25e145aaae..0ca94335ad9 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -196,7 +196,7 @@ floating point numbers SHOULD be considered as identifying. When more than one Instrument of the same `name` is created for identical Meters, denoted _duplicate instrument registration_, the implementation MUST create a valid Instrument in every case. Here, -"valid" means an instrument that is functional and can expected to +"valid" means an instrument that is functional and can be expected to export data, despite potentially creating a [semantic error in the data model](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations). From 1f4243b8804d046cfedad3e54a88721578ccffd7 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 11:00:30 -0800 Subject: [PATCH 33/41] Update specification/metrics/sdk.md Co-authored-by: Aaron Abbott --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index ec573f010d2..e911f718b1e 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -440,7 +440,7 @@ given instrument before starting a subsequent round of collection. As [stated in the API specification](api.md#instrument-type-conflict-detection), -implemetations are REQUIRED to create valid instruments in case of +implementations are REQUIRED to create valid instruments in case of duplicate instrument registration, and the [data model includes RECOMMENDATIONS on how to treat the consequent duplicate conflicting](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations) From 478b3cd84699a1289b631acbadc5004b30134a9d Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 11:46:42 -0800 Subject: [PATCH 34/41] remove multi-instrument callbacks for now --- specification/metrics/api.md | 58 +++++------------------------------- 1 file changed, 8 insertions(+), 50 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 45995e03613..1e59ae7684f 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -305,25 +305,17 @@ Please note that the term *synchronous* and *asynchronous* have nothing to do with the [asynchronous pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). -The API MUST support creation of asynchronous instruments by passing a -single callback function to be registered to the newly created +The API MUST support creation of asynchronous instruments by passing +one or more callback functions to be registered to the newly created instrument. -The API SHOULD support creation of asynchronous instruments -by passing a variable number of callback functions, greater than or -equal to zero, to be registered to the newly created instrument. +The API SHOULD support registration of callback functions to +asynchronous instruments after they are created. -The API MAY support registration of callbacks bound to -one or more instruments outside of instrument constructors, provided -the API declaratively states which instrument(s) will be be used from -each callback. - -The API SHOULD support unregistration of callbacks, in order to cease -making observations. Note that unregistration of callbacks does not -undo the effect of instrument registration itself; duplicate -instrument registration conflicts SHOULD still occur for instruments -with no registered callbacks. The effect of unregistering a callback -more than once is unspecified. +Where the API supports registration of callback functions after +asynchronous instrumentation creation, it MUST return something (e.g., +a registration handle, receipt or token) to the user that supports +undoing the effect of callback registation. Callback functions SHOULD NOT take an indefinite amount of time. @@ -1110,40 +1102,6 @@ for the interaction between the API and SDK. * A value * [`Attributes`](../common/common.md#attributes) -## Multi-instrument asynchronous callback example - -Where idiomatic, [the Metrics API supports the use of multiple -instruments from a single registered -callback](#asynchronous-instrument). Provided there is runtime -support for dynamic scope (e.g., thread-local state, context -variables), this can be achieved by an API similar to the synchronous -instruments. For example, - -```Python -# Python -class Device: - """A device with one counter""" - - def __init__(self, meter, x): - self.x = x - self.usage = meter.create_observable_counter(name="usage", description="count of items used") - self.pressure = meter.create_observable_gauge(name="pressure", description="force per unit area") - self.cb = meter.register_callback([self.usage, self.pressure], self.observe) - - def observe(self): - self.usage.observe(self.read_usage(), {'x', self.x}) - self.pressure.observe(self.read_pressure(), {'x', self.x}) - - def read_usage(self): - return 100 # ... - - def read_pressure(self): - return 10 # ... - - def stop(self): - self.cb.unregister() -``` - ## Compatibility requirements All the metrics components SHOULD allow new APIs to be added to From 519e8541d0f643946c5d76b6efebc72b83055861 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 11:56:14 -0800 Subject: [PATCH 35/41] update examples --- specification/metrics/api.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 1e59ae7684f..11b6cfa0782 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -552,7 +552,8 @@ class Device: def __init__(self, meter, x): self.x = x - self.cb = meter.create_observable_counter(name="usage", description="count of items used", self.counter_callback) + counter = meter.create_observable_counter(name="usage", description="count of items used") + self.cb = counter.register_callback(self.counter_callback) def counter_callback(self, result): result.Observe(self.read_counter(), {'x', self.x}) @@ -785,7 +786,8 @@ class Device: def __init__(self, meter, x): self.x = x - self.cb = meter.create_observable_gauge(name="pressure", description="force/area", self.gauge_callback) + gauge = meter.create_observable_gauge(name="pressure", description="force/area") + self.cb = gauge.register_callback(self.gauge_callback) def gauge_callback(self, result): result.Observe(self.read_gauge(), {'x', self.x}) @@ -1079,7 +1081,8 @@ class Device: def __init__(self, meter, x): self.x = x - self.cb = meter.create_observable_updowncounter(name="queue_size", description="items in process", self.updowncounter_callback) + updowncounter = meter.create_observable_updowncounter(name="queue_size", description="items in process") + self.cb = updowncounter.register_callback(self.updowncounter_callback) def updowncounter_callback(self, result): result.Observe(self.read_updowncounter(), {'x', self.x}) From dba0a00bb8f2ce1d0c122e14bbe0f2fdcba4893d Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 12:10:39 -0800 Subject: [PATCH 36/41] example text --- specification/metrics/api.md | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 11b6cfa0782..72e21a85d1d 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -306,8 +306,9 @@ with the [asynchronous pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). The API MUST support creation of asynchronous instruments by passing -one or more callback functions to be registered to the newly created -instrument. +zero or more callback functions to be permanently registered to the +newly created instrument. Callbacks functions passed to the +instrument constructor cannot be unregistered. The API SHOULD support registration of callback functions to asynchronous instruments after they are created. @@ -542,8 +543,11 @@ Asynchronous Counter uses an idiomatic interface for reporting measurements through a `callback`, which is registered during [Asynchronous Counter creation](#asynchronous-counter-creation). -The API SHOULD provide support for unregistering callbacks. For -example: +For callback functions registered after an asynchronous instrument is +created, the API is required to support a mechanism for +unregistration. For example, the object returned from `register_callback` +can suport an `unregister()` method directly. + ```python # Python @@ -776,8 +780,10 @@ Asynchronous Gauge uses an idiomatic interface for reporting measurements through a `callback`, which is registered during [Asynchronous Gauge creation](#asynchronous-gauge-creation). -The API SHOULD provide support for unregistering callbacks. For -example: +For callback functions registered after an asynchronous instrument is +created, the API is required to support a mechanism for +unregistration. For example, the object returned from `register_callback` +can suport an `unregister()` method directly. ```python # Python @@ -1071,8 +1077,10 @@ Asynchronous UpDownCounter uses an idiomatic interface for reporting measurements through a `callback`, which is registered during [Asynchronous Updowncounter creation](#asynchronous-updowncounter-creation). -The API SHOULD provide support for unregistering callbacks. For -example: +For callback functions registered after an asynchronous instrument is +created, the API is required to support a mechanism for +unregistration. For example, the object returned from `register_callback` +can suport an `unregister()` method directly. ```python # Python From 351eea6b4d3a0493a461becf83cb10b6ffb4c6fd Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 12:13:41 -0800 Subject: [PATCH 37/41] remove use of 'context' --- specification/metrics/sdk.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index c6d7708deaf..704c7521257 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -420,10 +420,10 @@ instruments that record negative measurements, e.g. `UpDownCounter` or `Observab ### Observations inside asynchronous callbacks -Callback functions MUST be invoked in the context of a specific -`MetricReader` performing collection, such that observations made or -produced by executing callbacks only apply to the intended -`MetricReader` during collection. +Callback functions MUST be invoked for the specific `MetricReader` +performing collection, such that observations made or produced by +executing callbacks only apply to the intended `MetricReader` during +collection. The implementation SHOULD disregard the accidental use of APIs appurtenant to asynchronous instruments outside of registered From 32c001bb7cd4126ed46e4604f09d662e9efb5f01 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 12:20:36 -0800 Subject: [PATCH 38/41] move an SDK req --- specification/metrics/api.md | 3 --- specification/metrics/sdk.md | 3 +++ 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index 72e21a85d1d..f6cc784f060 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -210,9 +210,6 @@ Instruments describes instances where all identifying fields are equal. The term _distinct_ applied to Instruments describes instances where at least one field value is different. -The implementation MUST aggregate data from identical Instruments -together in its export pipeline. - When more than one distinct Instrument is registered with the same `name` for identical Meters, the implementation SHOULD emit a warning to the user informing them of duplicate registration conflict(s). diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 704c7521257..d7d0139f6d0 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -445,6 +445,9 @@ RECOMMENDATIONS on how to treat the consequent duplicate conflicting](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations) `Metric` definitions. +The implementation MUST aggregate data from identical Instruments +together in its export pipeline. + The implementation SHOULD assist the user in managing conflicts by reporting each duplicate-conflicting instrument registration that was not corrected by a View as follows. When a potential conflict arises From 06cbf7de11f2bfd27ee0589499450804ffa024ab Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Tue, 15 Feb 2022 12:43:52 -0800 Subject: [PATCH 39/41] lint --- specification/metrics/api.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index f6cc784f060..c8d41636a5c 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -542,9 +542,8 @@ measurements through a `callback`, which is registered during For callback functions registered after an asynchronous instrument is created, the API is required to support a mechanism for -unregistration. For example, the object returned from `register_callback` -can suport an `unregister()` method directly. - +unregistration. For example, the object returned from `register_callback` +can support an `unregister()` method directly. ```python # Python @@ -561,7 +560,7 @@ class Device: def read_counter(self): return 100 # ... - + def stop(self): self.cb.unregister() ``` @@ -779,8 +778,8 @@ measurements through a `callback`, which is registered during For callback functions registered after an asynchronous instrument is created, the API is required to support a mechanism for -unregistration. For example, the object returned from `register_callback` -can suport an `unregister()` method directly. +unregistration. For example, the object returned from `register_callback` +can support an `unregister()` method directly. ```python # Python @@ -797,7 +796,7 @@ class Device: def read_gauge(self): return 100 # ... - + def stop(self): self.cb.unregister() ``` @@ -1076,8 +1075,8 @@ measurements through a `callback`, which is registered during For callback functions registered after an asynchronous instrument is created, the API is required to support a mechanism for -unregistration. For example, the object returned from `register_callback` -can suport an `unregister()` method directly. +unregistration. For example, the object returned from `register_callback` +can support an `unregister()` method directly. ```python # Python @@ -1094,7 +1093,7 @@ class Device: def read_updowncounter(self): return 100 # ... - + def stop(self): self.cb.unregister() ``` From 8a566084bbb09dcebf1f5af6be808bbb98069761 Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Thu, 17 Feb 2022 09:30:44 -0800 Subject: [PATCH 40/41] Fix lint --- specification/metrics/api.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index c8d41636a5c..f19b2697a6a 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -134,9 +134,9 @@ This API MUST accept the following parameters: Meters are identified by all of these fields. When more than one Meter of the same `name`, `version`, and `schema_url` is created, it is unspecified whether or under which conditions the same or different -`Meter` instances are returned. The term _identical_ applied to +`Meter` instances are returned. The term *identical* applied to Meters describes instances where all identifying fields are equal. -The term _distinct_ applied to Meters describes instances where at +The term *distinct* applied to Meters describes instances where at least one identifying field has a different value. Implementations MUST NOT require users to repeatedly obtain a `Meter` with @@ -196,7 +196,7 @@ floating point numbers SHOULD be considered as identifying. When more than one Instrument of the same `name` is created for -identical Meters, denoted _duplicate instrument registration_, the +identical Meters, denoted *duplicate instrument registration*, the implementation MUST create a valid Instrument in every case. Here, "valid" means an instrument that is functional and can be expected to export data, despite potentially creating a [semantic error in the @@ -205,9 +205,9 @@ model](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations). It is unspecified whether or under which conditions the same or different Instrument instance will be returned as a result of -duplicate instrument registration. The term _identical_ applied to +duplicate instrument registration. The term *identical* applied to Instruments describes instances where all identifying fields are -equal. The term _distinct_ applied to Instruments describes instances +equal. The term *distinct* applied to Instruments describes instances where at least one field value is different. When more than one distinct Instrument is registered with the same From 9e1a2d63a63bb5952117f0fbc4225793c992b304 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Thu, 17 Feb 2022 09:57:01 -0800 Subject: [PATCH 41/41] Update specification/metrics/api.md Co-authored-by: Bogdan Drutu --- specification/metrics/api.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specification/metrics/api.md b/specification/metrics/api.md index f19b2697a6a..4af98ad46b7 100644 --- a/specification/metrics/api.md +++ b/specification/metrics/api.md @@ -304,8 +304,7 @@ pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation). The API MUST support creation of asynchronous instruments by passing zero or more callback functions to be permanently registered to the -newly created instrument. Callbacks functions passed to the -instrument constructor cannot be unregistered. +newly created instrument. The API SHOULD support registration of callback functions to asynchronous instruments after they are created.