[Workspace] feat: update workspace server to support data connection type #8200

raintygao · 2024-09-14T05:51:30Z

Description

Update workspace server to support data connection type

This is a PR picked from #8013, as we want to separate frontend and server changes and merge server changes first for testing, so we extract server changes from #8013 and create this PR.

Testing the changes

Create workspace

post http://localhost:6601/api/workspaces

{
    "attributes": {
        "name": "workspace-name1",
        "color": "#54B399",
        "features": [
            "use-case-observability"
        ]
    },
    "settings": {
        "dataSources": [],
        "dataConnections": [
            "12345" // data connection object id
        ],
        "permissions": {
            "library_write": {
                "users": [
                    "%me%"
                ]
            },
            "write": {
                "users": [
                    "%me%"
                ]
            }
        }
    }
}

Result

{
    "success": true,
    "result": {
        "id": "_IlxNR"
    }
}

Update workspace

put http://localhost:6601/api/_IlxNR

{
    "attributes": {
        "name": "workspace-name1",
        "color": "#54B399",
        "features": [
            "use-case-observability"
        ]
    },
    "settings": {
        "dataSources": [],
        "dataConnections": [
            "12346" // new data connection object ids that you want to associate
        ],
        "permissions": {
            "library_write": {
                "users": [
                    "%me%"
                ]
            },
            "write": {
                "users": [
                    "%me%"
                ]
            }
        }
    }
}

Result

{
    "success": true,
    "result": true
}

Verify

get data-connection objects with workspace field
http://localhost:6601/api/saved_objects/_find?fields=id&fields=title&fields=workspaces&per_page=10000&type=data-connection&workspaces=*

Result

Changelog

feat: update workspace server to support data connection type

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: tygao <tygao@amazon.com>

codecov · 2024-09-14T06:08:26Z

Codecov Report

Attention: Patch coverage is 97.50000% with 1 line in your changes missing coverage. Please review.

Project coverage is 64.05%. Comparing base (21f4218) to head (35996f2).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...ed_objects_wrapper_for_check_workspace_conflict.ts	50.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8200   +/-   ##
=======================================
  Coverage   64.04%   64.05%           
=======================================
  Files        3740     3741    +1     
  Lines       88608    88635   +27     
  Branches    13799    13803    +4     
=======================================
+ Hits        56746    56772   +26     
- Misses      31264    31265    +1     
  Partials      598      598

Flag	Coverage Δ
Linux_1	`30.06% <97.50%> (+0.02%)`	⬆️
Linux_2	`58.83% <ø> (ø)`
Linux_3	`40.37% <ø> (+<0.01%)`	⬆️
Linux_4	`31.46% <ø> (-0.01%)`	⬇️
Windows_1	`30.08% <97.50%> (+0.02%)`	⬆️
Windows_2	`58.78% <ø> (ø)`
Windows_3	`40.37% <ø> (+<0.01%)`	⬆️
Windows_4	`31.46% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: tygao <tygao@amazon.com>

SuZhou-Joe · 2024-09-14T07:43:40Z

src/plugins/data_source/opensearch_dashboards.json

@@ -6,5 +6,5 @@
  "ui": true,
  "requiredPlugins": ["opensearchDashboardsUtils"],
  "optionalPlugins": [],
-  "extraPublicDirs": ["common/data_sources", "common/util"]
+  "extraPublicDirs": ["common","common/data_sources", "common/util"]


Seems like a new pattern to me, so it will bypass the lint check?

I suppose would, I checked some other usage before modified.
https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/opensearch_dashboards.json
https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/expressions/opensearch_dashboards.json

SuZhou-Joe

LGTM, so the permission on the data connections will be raised in another PR?

raintygao · 2024-09-14T08:04:30Z

LGTM, so the permission on the data connections will be raised in another PR?

Yes, this will be addressed and included in later PRs.

SuZhou-Joe · 2024-09-14T08:07:38Z

src/plugins/workspace/server/workspace_client.ts

+      if (newDataConnections) {
+        const originalSelectedDataConnectionIds = originalSelectedDataSourcesAndConnections
+          .filter((item) => item.type === DATA_CONNECTION_SAVED_OBJECT_TYPE)
+          .map((ds) => ds.id);
+        const dataConnectionsToBeRemoved = originalSelectedDataConnectionIds.filter(
+          (ds) => !newDataConnections.find((item) => item === ds)
+        );
+        const dataConnectionsToBeAdded = newDataConnections.filter(
+          (ds) => !originalSelectedDataConnectionIds.find((item) => item === ds)
+        );
+        if (dataConnectionsToBeRemoved.length > 0) {
+          for (const dataConnectionId of dataConnectionsToBeRemoved) {
+            promises.push(
+              client.deleteFromWorkspaces(DATA_CONNECTION_SAVED_OBJECT_TYPE, dataConnectionId, [id])
+            );
+          }
+        }
+        if (dataConnectionsToBeAdded.length > 0) {
+          for (const dataConnectionId of dataConnectionsToBeAdded) {
+            promises.push(
+              client.addToWorkspaces(DATA_CONNECTION_SAVED_OBJECT_TYPE, dataConnectionId, [id])
+            );
+          }
        }
      }


Nit: seems like the logic to handle the data connection are the same as data source, would it be possible to make it into a function like addOperationsToPromisesList(type: string)?

So the code here will be like

const promises = []; const addOperationsToPromisesList = (type: string) => { xxx } // Add data sources operations into promise addOperationsToPromisesList(DATA_SOURCE_SAVED_OBJECT_TYPE); // Add data connections operations into promise addOperationsToPromisesList(DATA_CONNECTION_SAVED_OBJECT_TYPE);

Sounds good, will optimize this part in #8013.

ruanyl · 2024-09-14T08:26:00Z

src/plugins/workspace/public/workspace_client.ts

@@ -204,6 +204,7 @@ export class WorkspaceClient implements IWorkspaceClient {
    settings: {
      dataSources?: string[];
      permissions?: SavedObjectPermissions;
+      dataConnections?: string[];


Why not reuse dataSources? It doesn't seem necessary to add an extra parameter dataConnections. Because for workspace, they are all data sources, I personally don't want it to be implemented in a way that every time adding new type of data source, it requires to change the API interface.

what's passed in here is just saved object ids, we could handle different type of data source internally.

Yes, if we reuse dataSources field, we need to update this field from [‘id1’,‘id2’] to [{type:‘data-source’,id:‘id1’},{type:‘data-connection’,id:‘id2’], which would make a breaking change to existing API.

…type (#8200) * feat: update worksapce server to support data connection type Signed-off-by: tygao <tygao@amazon.com> * Changeset file for PR #8200 created/updated * test: add tests for wrapper Signed-off-by: tygao <tygao@amazon.com> --------- Signed-off-by: tygao <tygao@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit f338aff) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…type (#8200) (#8203) * feat: update worksapce server to support data connection type * Changeset file for PR #8200 created/updated * test: add tests for wrapper --------- (cherry picked from commit f338aff) Signed-off-by: tygao <tygao@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>

github-actions bot added the all-star-contributor label Sep 14, 2024

opensearch-changeset-bot bot added a commit to raintygao/OpenSearch-Dashboards that referenced this pull request Sep 14, 2024

Changeset file for PR opensearch-project#8200 created/updated

5421ae6

raintygao and others added 2 commits September 14, 2024 13:53

feat: update worksapce server to support data connection type

4dbfbdc

Signed-off-by: tygao <tygao@amazon.com>

Changeset file for PR opensearch-project#8200 created/updated

aec3dec

raintygao force-pushed the connection-server branch from 5421ae6 to aec3dec Compare September 14, 2024 05:53

raintygao mentioned this pull request Sep 14, 2024

[Workspace] Integrate workspace front end with data connection type saved object #8013

Merged

7 tasks

test: add tests for wrapper

35996f2

Signed-off-by: tygao <tygao@amazon.com>

raintygao changed the title ~~feat: update workspace server to support data connection type~~ [Workspace] feat: update workspace server to support data connection type Sep 14, 2024

SuZhou-Joe reviewed Sep 14, 2024

View reviewed changes

SuZhou-Joe approved these changes Sep 14, 2024

View reviewed changes

SuZhou-Joe reviewed Sep 14, 2024

View reviewed changes

ruanyl reviewed Sep 14, 2024

View reviewed changes

ruanyl approved these changes Sep 14, 2024

View reviewed changes

SuZhou-Joe added backport 2.x v2.18.0 and removed backport 2.x labels Sep 14, 2024

SuZhou-Joe merged commit f338aff into opensearch-project:main Sep 14, 2024
73 checks passed

amsiglan added the backport 2.x label Sep 16, 2024

opensearch-trigger-bot bot mentioned this pull request Sep 16, 2024

[Backport 2.x] [Workspace] feat: update workspace server to support data connection type #8203

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Workspace] feat: update workspace server to support data connection type #8200

[Workspace] feat: update workspace server to support data connection type #8200

raintygao commented Sep 14, 2024 •

edited

Loading

codecov bot commented Sep 14, 2024 •

edited

Loading

SuZhou-Joe Sep 14, 2024

raintygao Sep 14, 2024

SuZhou-Joe left a comment

raintygao commented Sep 14, 2024

SuZhou-Joe Sep 14, 2024 •

edited

Loading

raintygao Sep 14, 2024

ruanyl Sep 14, 2024

ruanyl Sep 14, 2024

raintygao Sep 14, 2024

[Workspace] feat: update workspace server to support data connection type #8200

[Workspace] feat: update workspace server to support data connection type #8200

Conversation

raintygao commented Sep 14, 2024 • edited Loading

Description

Testing the changes

Create workspace

Update workspace

Verify

Changelog

Check List

codecov bot commented Sep 14, 2024 • edited Loading

Codecov Report

SuZhou-Joe Sep 14, 2024

Choose a reason for hiding this comment

raintygao Sep 14, 2024

Choose a reason for hiding this comment

SuZhou-Joe left a comment

Choose a reason for hiding this comment

raintygao commented Sep 14, 2024

SuZhou-Joe Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

raintygao Sep 14, 2024

Choose a reason for hiding this comment

ruanyl Sep 14, 2024

Choose a reason for hiding this comment

ruanyl Sep 14, 2024

Choose a reason for hiding this comment

raintygao Sep 14, 2024

Choose a reason for hiding this comment

raintygao commented Sep 14, 2024 •

edited

Loading

codecov bot commented Sep 14, 2024 •

edited

Loading

SuZhou-Joe Sep 14, 2024 •

edited

Loading