Merge pull request #1286 from bosch-io/doc/ditto-230-release-notes

prepare Ditto 2.3.0 release and release notes, blogpost, etc.

SECURITY.md

@@ -5,10 +5,10 @@
 These versions of Eclipse Ditto are currently being supported with security updates.
 
 | Version | Supported          |
-| ------- | ------------------ |
+|---------| ------------------ |
+| 2.3.x   | :white_check_mark: |
 | 2.2.x   | :white_check_mark: |
-| 2.1.x   | :white_check_mark: |
-| < 2.1.0 | :x:                |
+| < 2.2.0 | :x:                |
 
 ## Reporting a Vulnerability
 

documentation/src/main/resources/_config.yml

@@ -114,6 +114,8 @@ plugins:
 docVersions:
   - label: "development"
     basePath: ""
+  - label: "2.3"
+    basePath: "2.3"
   - label: "2.2"
     basePath: "2.2"
   - label: "2.1"

documentation/src/main/resources/_data/sidebars/ditto_sidebar.yml

@@ -23,6 +23,9 @@ entries:
       - title: Release Notes
         output: web
         folderitems:
+          - title: 2.3.0
+            url: /release_notes_230.html
+            output: web
           - title: 2.2.2
             url: /release_notes_222.html
             output: web

documentation/src/main/resources/_posts/2022-01-21-release-announcement-230.md

@@ -0,0 +1,89 @@
+---
+title: "Announcing Eclipse Ditto Release 2.3.0"
+published: true
+permalink: 2022-01-21-release-announcement-230.html
+layout: post
+author: thomas_jaeckle
+tags: [blog]
+hide_sidebar: true
+sidebar: false
+toc: false
+---
+
+The Eclipse Ditto teams announces availability of Eclipse Ditto [2.3.0](https://projects.eclipse.org/projects/iot.ditto/releases/2.3.0).
+
+It contains mainly new features around the Ditto ["live" channel](protocol-twinlive.html#live) which can be used to 
+directly interact with devices.
+
+Such live commands may now be easily [created via the HTTP API](2021-12-20-http-live-channel.html) - and in addition
+a conventional API call targeting the persisted [twin](protocol-twinlive.html#twin) may now be automatically converted
+to a live command, based on a passed [live channel condition](2021-12-22-live-channel-condition.html).
+
+With that, we are proud that we now can provide a really powerful addition to Ditto's "Digital Twin pattern":
+* requesting data from an actual device e.g. when it is currently connected/online (based on a [live channel condition](basic-conditional-requests.html#live-channel-condition) targeting the online status)
+* fall back to its last known persisted state
+  * if it is currently not online (the condition did not match)
+  * or also as a fallback, if the device does not answer within e.g. 10 seconds
+* all that within one single API call, including information from which source the data was reported back
+  * the "live" device or the persisted "twin" 
+
+## Adoption
+
+Companies are willing to show their adoption of Eclipse Ditto publicly: [https://iot.eclipse.org/adopters/?#iot.ditto](https://iot.eclipse.org/adopters/?#iot.ditto)
+
+From our various [feedback channels](feedback.html) we however know of more adoption.  
+If you are making use of Eclipse Ditto, it would be great to show this by adding your company name to that list of 
+known adopters.  
+In the end, that's one main way of measuring the success of the project.
+
+
+## Changelog
+
+The main improvements and additions of Ditto 2.3.0 are:
+
+* HTTP API for "live" commands
+* Smart channel strategy for live/twin read access
+* Configurable allowing creation of entities (policies/things) based on namespace and authenticated subjects
+* Allow using `*` as a placeholder for the feature id in selected fields
+
+The following notable fixes are included:
+
+* Fix potential concurrent modification errors when using JavaScript payload mapping and global variables
+* Fix reconnect backoff for Kafka connections with authentication failures
+* Fix caching of JWTs in HTTP push connections
+* Fix potentially unreachable client actors in connections with `clientCount > 1`
+* Fix search inconsistencies for very active things during shard relocation (e.g. on rolling updates)
+* Fix that a Kafka connection with only targets remains "open" even if Kafka broker is not available
+* Allow usage of absolute domain paths ending with a "." as Kafka bootstrap servers
+* Ensure that Ditto pub/sub state is eventually consistent with a guaranteed upper time limit
+
+The following non-functional work is also included:
+
+* Update of several used dependencies
+
+Please have a look at the [2.3.0 release notes](release_notes_230.html) for a more detailed information on the release.
+
+
+## Artifacts
+
+The new Java artifacts have been published at the [Eclipse Maven repository](https://repo.eclipse.org/content/repositories/ditto/)
+as well as [Maven central](https://repo1.maven.org/maven2/org/eclipse/ditto/).
+
+The Ditto JavaScript client release was published on [npmjs.com](https://www.npmjs.com/~eclipse_ditto):
+* [@eclipse-ditto/ditto-javascript-client-dom](https://www.npmjs.com/package/@eclipse-ditto/ditto-javascript-client-dom)
+* [@eclipse-ditto/ditto-javascript-client-node](https://www.npmjs.com/package/@eclipse-ditto/ditto-javascript-client-node)
+
+
+The Docker images have been pushed to Docker Hub:
+* [eclipse/ditto-policies](https://hub.docker.com/r/eclipse/ditto-policies/)
+* [eclipse/ditto-things](https://hub.docker.com/r/eclipse/ditto-things/)
+* [eclipse/ditto-things-search](https://hub.docker.com/r/eclipse/ditto-things-search/)
+* [eclipse/ditto-gateway](https://hub.docker.com/r/eclipse/ditto-gateway/)
+* [eclipse/ditto-connectivity](https://hub.docker.com/r/eclipse/ditto-connectivity/)
+* [eclipse/ditto-concierge](https://hub.docker.com/r/eclipse/ditto-concierge/)
+
+<br/>
+<br/>
+{% include image.html file="ditto.svg" alt="Ditto" max-width=500 %}
+--<br/>
+The Eclipse Ditto team

documentation/src/main/resources/pages/ditto/basic-placeholders.md

@@ -41,7 +41,7 @@ or subscriptions for live messages, the following placeholders are available in
 
 | Placeholder                       | Description                                                                                                                                                                                     |
 |-----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `topic:full` | full [Ditto Protocol topic path](protocol-specification-topic.html)<br/>in the form `{namespace}/{entityId}/{group}/`<br/>`{channel}/{criterion}/{action-subject}`                              |
+| `topic:full` | full [Ditto Protocol topic path](protocol-specification-topic.html)<br/>in the form `{namespace}/{entityName}/{group}/`<br/>`{channel}/{criterion}/{action-subject}`                            |
 | `topic:namespace` | Ditto Protocol [topic namespace](protocol-specification-topic.html#namespace)                                                                                                                   |
 | `topic:entityName` | Ditto Protocol [topic entity name](protocol-specification-topic.html#entity-name)                                                                                                               |
 | `topic:group` | Ditto Protocol [topic group](protocol-specification-topic.html#group)                                                                                                                           |
@@ -70,7 +70,7 @@ In [connections](basic-connections.html), the following placeholders are availab
 | `{%raw%}{{ feature:id }}{%endraw%}` | the ID of the feature (only available if the processed signal was related to a feature)                                                                                                         |
 | `{%raw%}{{ header:<header-name> }}{%endraw%}` | external header value for connection sources, or Ditto protocol header value for targets and reply-targets (both case-insensitive)                                                              |
 | `{%raw%}{{ request:subjectId }}{%endraw%}` | primary authorization subject of a command, or primary authorization subject that caused an event                                                                                               |
-| `{%raw%}{{ topic:full }}{%endraw%}` | full [Ditto Protocol topic path](protocol-specification-topic.html)<br/>in the form `{namespace}/{entityId}/{group}/`<br/>`{channel}/{criterion}/{action-subject}`                              |
+| `{%raw%}{{ topic:full }}{%endraw%}` | full [Ditto Protocol topic path](protocol-specification-topic.html)<br/>in the form `{namespace}/{entityName}/{group}/`<br/>`{channel}/{criterion}/{action-subject}`                            |
 | `{%raw%}{{ topic:namespace }}{%endraw%}` | Ditto Protocol [topic namespace](protocol-specification-topic.html#namespace)                                                                                                                   |
 | `{%raw%}{{ topic:entityName }}{%endraw%}` | Ditto Protocol [topic entity name](protocol-specification-topic.html#entity-name)                                                                                                               |
 | `{%raw%}{{ topic:group }}{%endraw%}` | Ditto Protocol [topic group](protocol-specification-topic.html#group)                                                                                                                           |
@@ -81,7 +81,7 @@ In [connections](basic-connections.html), the following placeholders are availab
 | `{%raw%}{{ topic:action-subject }}{%endraw%}` | either Ditto Protocol [topic action](protocol-specification-topic.html#action-optional) or [topic subject](protocol-specification-topic.html#messages-criterion-actions) (for message commands) |
 | `{%raw%}{{ resource:type }}{%endraw%}` | the type of the Ditto Protocol [path](protocol-specification.html#path) , one of: `thing`, `policy`, `message` or `connection`                                                                  |
 | `{%raw%}{{ resource:path }}{%endraw%}` | the affected resource's path being the Ditto Protocol [path](protocol-specification.html#path) in JsonPointer notation, e.g. `/` when a complete thing was created/modified/deleted             |
-| `{%raw%}{{ time:now }}{%endraw%}` | the current timestamp in ISO-8601 format as string in UTC timezone                                                                                                                                             | 
+| `{%raw%}{{ time:now }}{%endraw%}` | the current timestamp in ISO-8601 format as string in UTC timezone                                                                                                                              | 
 | `{%raw%}{{ time:now_epoch_millis }}{%endraw%}` | the current timestamp in "milliseconds since epoch" formatted as string                                                                                                                         | 
 
 

documentation/src/main/resources/pages/ditto/basic-thing.md

@@ -76,8 +76,8 @@ semantics of the data or adding other useful information about the data points o
 
 ```json
 {
-  "thingId": "the.namespace:theId",
-  "policyId": "the.namespace:thePolicyId",
+  "thingId": "the.namespace:theName",
+  "policyId": "the.namespace:thePolicyName",
   "definition": "org.eclipse.ditto:HeatingDevice:2.1.0",
   "attributes": {
       "someAttr": 32,

documentation/src/main/resources/pages/ditto/release_notes_220.md

@@ -33,7 +33,7 @@ The following non-functional work is also included:
 <br/>
 
 For a complete list of all merged PRs, inspect the following milestones:
-* [merged pull requests for milestone 2.1.0](https://github.com/eclipse/ditto/pulls?q=is:pr+milestone:2.2.0)
+* [merged pull requests for milestone 2.2.0](https://github.com/eclipse/ditto/pulls?q=is:pr+milestone:2.2.0)
 
 <br/>
 <br/>

documentation/src/main/resources/pages/ditto/release_notes_230.md

@@ -0,0 +1,127 @@
+---
+title: Release notes 2.3.0
+tags: [release_notes]
+published: true
+keywords: release notes, announcements, changelog
+summary: "Version 2.3.0 of Eclipse Ditto, released on 21.01.2022"
+permalink: release_notes_230.html
+---
+
+Ditto **2.3.0** is API and [binary compatible](https://github.com/eclipse/ditto/blob/master/documentation/src/main/resources/architecture/DADR-0005-semantic-versioning.md)
+to prior Eclipse Ditto 2.x versions.
+
+## Changelog
+
+Eclipse Ditto 2.3.0 includes the following topics/enhancements:
+
+* HTTP API for "live" commands
+* Smart channel strategy for live/twin read access
+* Configurable allowing creation of entities (policies/things) based on namespace and authenticated subjects
+* Allow using `*` as a placeholder for the feature id in selected fields
+
+The following notable fixes are included:
+
+* Fix potential concurrent modification errors when using JavaScript payload mapping and global variables
+* Fix reconnect backoff for Kafka connections with authentication failures
+* Fix caching of JWTs in HTTP push connections
+* Fix potentially unreachable client actors in connections with `clientCount > 1`
+* Fix search inconsistencies for very active things during shard relocation (e.g. on rolling updates)
+* Fix that a Kafka connection with only targets remains "open" even if Kafka broker is not available
+* Allow usage of absolute domain paths ending with a "." as Kafka bootstrap servers
+* Ensure that Ditto pub/sub state is eventually consistent with a guaranteed upper time limit
+
+The following non-functional work is also included:
+
+* Update of several used dependencies:
+  * Akka: `2.6.18`
+  * Akka Management: `1.1.2`
+  * Caffeine: `3.0.5`
+  * Classindex: `3.11`
+  * Cloudevents: `2.3.0`
+  * HiveMQ MQTT client: `1.3.0`
+  * Jackson: `2.12.6`
+  * Logback: `1.2.10`
+  * SLF4J: `1.7.32`
+  * SSL config core: `0.4.3`
+  * Typesafe config: `1.4.1`
+  * Rhino JS engine: `1.7.14`
+
+<br/>
+
+For a complete list of all merged PRs, inspect the following milestones:
+* [merged pull requests for milestone 2.3.0](https://github.com/eclipse/ditto/pulls?q=is:pr+milestone:2.3.0)
+
+<br/>
+<br/>
+
+Compared to the latest release [2.2.0](release_notes_220.html), the following most notable changes, new features and
+bugfixes were added.
+
+
+### New features
+
+#### [HTTP API for "live" commands](https://github.com/eclipse/ditto/issues/106)
+
+Ditto's ["live" channel](protocol-twinlive.html#live) is now also available for commands invoked via HTTP API.  
+See also the [blogpost covering that topic](2021-12-20-http-live-channel.html).
+
+To qualify a command (e.g. "modify thing" or "retrieve feature property") as a "live" command, the header or 
+query parameter `channel=live` has to be specified.
+
+Live commands bypass the twin and go directly to the devices.  
+However, an existing twin is still required for policy enforcement.
+
+
+#### [Smart channel strategy for live/twin read access](https://github.com/eclipse/ditto/issues/1228)
+
+Ditto adds support for selecting the "twin" or the "live" channel for thing query commands based on an 
+[RQL condition](basic-rql.html) of a newly added parameter 
+[live channel condition](basic-conditional-requests.html#live-channel-condition).  
+See also the [blogpost covering that topic](2021-12-22-live-channel-condition.html).
+
+In addition, a new [payload mapper](connectivity-mapping.html#updatetwinwithliveresponse-mapper) automatically updating 
+the twin based on received live data from devices was added.
+
+#### [Configurable allowing creation of entities based on namespace and authenticated subjects](https://github.com/eclipse/ditto/pull/1251)
+
+This added feature allows configuring restrictions, which [authenticated subjects](basic-auth.html#authenticated-subjects)
+may create new entities (things / policies) in which namespaces.
+
+#### [Allow using `*` as a placeholder for the feature id in selected fields](https://github.com/eclipse/ditto/pull/1277)
+
+When selecting for certain [`fields` of a thing](httpapi-concepts.html#field-selector-with-wildcard) or when using 
+[signal enrichment (extraFields)](basic-enrichment.html) in order to add more (unchanged) data from a twin to e.g. events 
+the wildcard `*` can now be used in order to select all features of a thing without the need to know their feature names.
+
+
+### Bugfixes
+
+Several bugs in Ditto 2.2.x were fixed for 2.3.0.  
+This is a complete list of the
+[merged pull requests](https://github.com/eclipse/ditto/pulls?q=is%3Apr+milestone%3A2.3.0), including the fixed bugs.
+
+
+## Migration notes
+
+No migrations required updating from Ditto 2.2.x
+
+## Ditto clients
+
+For a complete list of all merged client PRs, inspect the following milestones:
+* [merged pull requests for milestone 2.3.0](https://github.com/eclipse/ditto-clients/pulls?q=is:pr+milestone:2.3.0)
+
+### Ditto Java SDK
+
+No mentionable changes/enhancements/bugfixes.
+
+### Ditto JavaScript SDK
+
+See separate [Changelog](https://github.com/eclipse/ditto-clients/blob/master/javascript/CHANGELOG.md) of JS client.
+
+
+## Roadmap
+
+Looking forward, the current plans for Ditto 2.4.0 are:
+
+* Update service code to Java 17 (APIs stay at Java 8) + run Ditto containers with Java 17 runtime
+* Continue work on the started [WoT (Web of Things) integration](https://github.com/eclipse/ditto/pull/1270)

legal/3rd-party-dependencies/compile.txt

@@ -29,11 +29,12 @@ com.google.j2objc:j2objc-annotations:jar:1.3:compile
 com.google.protobuf:protobuf-java:jar:3.14.0:compile
 com.hierynomus:asn-one:jar:0.5.0:compile
 com.hivemq:hivemq-mqtt-client:jar:1.3.0:compile
-com.lightbend.akka.discovery:akka-discovery-kubernetes-api_2.13:jar:1.0.10:compile
-com.lightbend.akka.management:akka-lease-kubernetes_2.13:jar:1.0.10:compile
-com.lightbend.akka.management:akka-management_2.13:jar:1.0.10:compile
-com.lightbend.akka.management:akka-management-cluster-bootstrap_2.13:jar:1.0.10:compile
-com.lightbend.akka.management:akka-management-cluster-http_2.13:jar:1.0.10:compile
+com.lightbend.akka.discovery:akka-discovery-kubernetes-api_2.13:jar:1.1.2:compile
+com.lightbend.akka.management:akka-lease-kubernetes_2.13:jar:1.1.2:compile
+com.lightbend.akka.management:akka-management_2.13:jar:1.1.2:compile
+com.lightbend.akka.management:akka-management-cluster-bootstrap_2.13:jar:1.1.2:compile
+com.lightbend.akka.management:akka-management-cluster-http_2.13:jar:1.1.2:compile
+com.lightbend.akka.management:akka-management-pki_2.13:jar:1.1.2:compile
 commons-net:commons-net:jar:3.8.0:compile
 com.newmotion:akka-rabbitmq_2.13:jar:5.1.2:compile
 com.rabbitmq:amqp-client:jar:5.7.3:compile
@@ -114,7 +115,7 @@ org.mongodb:mongodb-driver-core:jar:4.1.2:compile
 org.mongodb:mongodb-driver-reactivestreams:jar:4.1.2:compile
 org.mongodb.scala:mongo-scala-bson_2.13:jar:4.1.1:compile
 org.mongodb.scala:mongo-scala-driver_2.13:jar:4.1.1:compile
-org.mozilla:rhino-runtime:jar:1.7.13:compile
+org.mozilla:rhino-runtime:jar:1.7.14:compile
 org.ow2.asm:asm-analysis:jar:7.2:compile
 org.ow2.asm:asm-commons:jar:7.2:compile
 org.ow2.asm:asm:jar:7.2:compile