Set the mapped security roles of the user so these can be used by the… #1588

ch-govau · 2022-01-27T04:50:40Z

… DLS privileges evaluator. Allow security roles to be used for DLS parameter substitution. Fixes opensearch-project/security/#1568

Signed-off-by: Caitlin Harper caitlin.harper@defence.gov.au

Description

Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation)
Bug Fix /Enhancement
Why these changes are required?
The changes are required so that we can map the backend roles to more appropriately named roles via the roles.mapping and these security roles can be used in the DLS privileges evaluation.
What is the old behavior before changes and new behavior after changes?
Old behaviour:
security roles were not added to the user object prior to the DLS privileges evaluation

New behaviour:
mapped roles are added to the user object for DLS privileges evaluation
a new option for parameter substitution in the DLS query has been added. This is ${user.securityRoles} and is the mapped roles.

Note: there is no change to the existing ${user.roles} substitution. It remains the same.

Issues Resolved

Resolves [BUG] Unexpected role set used for document level security. #1568

Is this a backport? If so, please add backport PR # and/or commits #

Testing

Manual test steps:
Testing was completed for both basic auth and OIDC/jwt auth. Note that both users had read access to the index.

Push 2 documents into the index, one with DLS required role = ROLE1 and the other with ROLE2.
example:

curl https://localhost:9200/myindex/_doc?pretty -u admin:$PASS -XPOST -H "Content-Type: application/json" -d @doc1.json

doc1.json

{
	"security": {
                        "exclusive": [
                            "ROLE1",
                        ],
                        "num_exclusive": 1
          },
	  "info":{
		"generalinfo": "this document should only be available to those with ROLE1"
	  }
}

Add the roles ROLE1, ROLE2 and the read to the roles.yml
example:

ROLE1:
  reserved: false
  hidden: false
  cluster_permissions: []
  index_permissions: []
  tenant_permissions: []
  static: false

ROLE2:
  reserved: false
  hidden: false
  cluster_permissions: []
  index_permissions: []
  tenant_permissions: []
  static: false

myindex_read:
  reserved: false
  hidden: false
  cluster_permissions:
  - "indices:data/read/scroll/clear"
  - "indices:data/read/msearch"
  - "indices:data/write/bulk"
  - "indices:data/read/scroll"
  - "indices:data/read/mget"
  index_permissions:
  - index_patterns:
    - "myindex"
    dls: "{ \"bool\": { \"filter\": [\n   {\"terms_set\": {\"security.exclusive.keyword\": {\n\
      \ \"terms\": [${user.securityRoles}],\n \"minimum_should_match_field\": \"security.num_exclusive\"\
      \n }}}\n ]}}\n"
    fls: []
    masked_fields: []
    allowed_actions:
    - "read"

Add the role mappings to the backend roles in roles_mapping.yml

ROLE1:
  hosts: []
  users: []
  reserved: false
  hidden: false
  backend_roles:
  - "client-id/role-1"
  and_backend_roles: []

ROLE2:
  hosts: []
  users: []
  reserved: false
  hidden: false
  backend_roles:
  - "client-id/role-2"
  and_backend_roles: []

Also add the following user for the internal/basic auth testing

user1:
  hash: "<password-hash>"
  reserved: false
  backend_roles:
    - "client-id/role-1"
    - "myindex_read"
  description: "user1"

Note that we also have the test server set up to use keycloak and user2 has the following backend roles: "myindex_read", "client-id/role-1", "client-id/role-2".
Run the security config reload script

./plugins/opensearch-security/tools/securityadmin.sh" -cd "./plugins/opensearch-security/securityconfig" -icl -key "./config/kirk-key.pem" -cert "./config/kirk.pem" -cacert "./config/root-ca.pem" -nhnv

Test basic auth for user 1

curl https://localhost:9200/myindex/_search?pretty --insecure -u user1:user1

Expected result: This should return 1 document, doc1
8. Test jwt for user2

curl -H "Authorization: Bearer $TOKEN" https://localhost:9200/myindex/_search?pretty --insecure

Expected result: This should return 2 documents.
9. Also tested with another user that has neither ROLE1 or ROLE2 and confirm they have access to zero documents.

Documentation

This will require an update to the DLS parameter substitution documentation to include the new ${user.securityRoles}.
The update to the documentation has been submitted here: opensearch-project/documentation-website#420

Check List

New functionality includes testing
New functionality has been documented
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

ae-govau · 2022-02-21T05:15:25Z

Gentle ping - our stakeholders are still keen on this. Would appreciate feedback (positive or negative) on the approach and whether anything additional is needed.

I note the failing "CodeQL" check - but I note that what it is complaining about is not relevant to (ie is not changed by) this PR.

cliu123 · 2022-02-21T17:16:36Z

Thanks for contributing! Can you add UTs for all possible use cases?

ch-govau · 2022-02-21T20:49:58Z

Thanks for contributing! Can you add UTs for all possible use cases?

Can do.

ch-govau · 2022-02-22T01:39:53Z

Tests added, will rebase and squash when everyone is happy.

ch-govau · 2022-02-25T03:38:28Z

Any thoughts/comments stopping this from being merged. I'm happy to make changes if required? Also the documentation update has been completed by my colleague and is linked to the PR.

peternied · 2022-03-04T20:51:46Z

@opensearch-project/security can we review this PR?

ae-govau · 2022-03-16T23:24:21Z

Hey all, this has been sitting stagnant for a while. @ch-govau has added the tests, and @mm-govau has added the documentation in opensearch-project/documentation-website#420 which is already approved.

I think all we are waiting on is an approval here? Any attention for review for would be much appreciated. Outside of the tests, it's 10 lines of new code, so hoping it's not a hard review task.

Thanks again.

peternied · 2022-03-17T23:22:48Z

@cliu123 can you review this?

src/test/resources/dlsfls/roles.yml

src/main/java/org/opensearch/security/securityconf/ConfigModelV7.java

src/test/java/org/opensearch/security/dlic/dlsfls/DlsPropsReplaceTest.java

cliu123 · 2022-03-20T21:12:29Z

Could you please sign all the commits?

ch-govau · 2022-03-20T21:21:59Z

Could you please sign all the commits?

sorry about that. have rebased and the commit is signed. Thanks so much for reviewing this. It is very much appreciated :)

codecov-commenter · 2022-03-20T22:27:33Z

Codecov Report

Merging #1588 (0efcaa5) into main (e638c76) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 0efcaa5 differs from pull request most recent head f90dc0e. Consider uploading reports for the commit f90dc0e to get more accurate results

@@             Coverage Diff              @@
##               main    #1588      +/-   ##
============================================
- Coverage     62.94%   62.94%   -0.01%     
  Complexity     3261     3261              
============================================
  Files           253      253              
  Lines         18126    18133       +7     
  Branches       3258     3259       +1     
============================================
+ Hits          11410    11413       +3     
- Misses         5067     5071       +4     
  Partials       1649     1649

Impacted Files	Coverage Δ
...earch/security/privileges/PrivilegesEvaluator.java	`71.84% <100.00%> (+0.09%)`	⬆️
...pensearch/security/securityconf/ConfigModelV7.java	`62.63% <100.00%> (+0.42%)`	⬆️
...ecurity/configuration/ConfigurationRepository.java	`73.07% <0.00%> (-2.20%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e638c76...f90dc0e. Read the comment docs.

… DLS privileges evaluator. Allow security roles to be used for DLS parameter substitution. Fixes opensearch-project/security/opensearch-project#1568 Signed-off-by: Caitlin Harper <caitlin.harper@defence.gov.au>

ae-govau · 2022-03-28T22:38:52Z

@cliu123 - is anything else needed on this PR? I think @ch-govau has addressed all comments and this can be merged after a 2nd approval. Thanks for your help.

cliu123 · 2022-03-28T22:53:44Z

@cliu123 - is anything else needed on this PR? I think @ch-govau has addressed all comments and this can be merged after a 2nd approval. Thanks for your help.

LGTM! Thanks for the contribution!

… DLS privileges evaluator. Allow security roles to be used for DLS parameter substitution. Fixes opensearch-project/security/opensearch-project#1568 (opensearch-project#1588) Signed-off-by: Caitlin Harper <caitlin.harper@defence.gov.au>

ch-govau requested a review from a team January 27, 2022 04:50

mm-govau mentioned this pull request Feb 22, 2022

Documentation for security roles to be used for DLS parameter substit… opensearch-project/documentation-website#420

Merged

1 task

ch-govau force-pushed the security-roles-in-dls branch from 132369f to 45ff94c Compare February 25, 2022 04:02

peternied requested a review from cliu123 March 17, 2022 23:23

cliu123 reviewed Mar 18, 2022

View reviewed changes

src/test/resources/dlsfls/roles.yml Outdated Show resolved Hide resolved

cliu123 reviewed Mar 18, 2022

View reviewed changes

src/main/java/org/opensearch/security/securityconf/ConfigModelV7.java Show resolved Hide resolved

cliu123 reviewed Mar 18, 2022

View reviewed changes

src/test/java/org/opensearch/security/dlic/dlsfls/DlsPropsReplaceTest.java Show resolved Hide resolved

ch-govau force-pushed the security-roles-in-dls branch from 0efcaa5 to f90dc0e Compare March 20, 2022 21:15

peternied previously approved these changes Mar 21, 2022

View reviewed changes

Set the mapped security roles of the user so these can be used by the…

f778b38

… DLS privileges evaluator. Allow security roles to be used for DLS parameter substitution. Fixes opensearch-project/security/opensearch-project#1568 Signed-off-by: Caitlin Harper <caitlin.harper@defence.gov.au>

ch-govau dismissed peternied’s stale review via f778b38 March 23, 2022 00:56

ch-govau force-pushed the security-roles-in-dls branch from f90dc0e to f778b38 Compare March 23, 2022 00:56

peternied approved these changes Mar 23, 2022

View reviewed changes

cliu123 approved these changes Mar 28, 2022

View reviewed changes

cliu123 merged commit 51e492c into opensearch-project:main Mar 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set the mapped security roles of the user so these can be used by the… #1588

Set the mapped security roles of the user so these can be used by the… #1588

ch-govau commented Jan 27, 2022 •

edited by peternied

Loading

ae-govau commented Feb 21, 2022

cliu123 commented Feb 21, 2022

ch-govau commented Feb 21, 2022

ch-govau commented Feb 22, 2022

ch-govau commented Feb 25, 2022

peternied commented Mar 4, 2022

ae-govau commented Mar 16, 2022

peternied commented Mar 17, 2022

cliu123 commented Mar 20, 2022

ch-govau commented Mar 20, 2022

codecov-commenter commented Mar 20, 2022 •

edited

Loading

ae-govau commented Mar 28, 2022

cliu123 commented Mar 28, 2022

Set the mapped security roles of the user so these can be used by the… #1588

Set the mapped security roles of the user so these can be used by the… #1588

Conversation

ch-govau commented Jan 27, 2022 • edited by peternied Loading

Description

Issues Resolved

Testing

Documentation

Check List

ae-govau commented Feb 21, 2022

cliu123 commented Feb 21, 2022

ch-govau commented Feb 21, 2022

ch-govau commented Feb 22, 2022

ch-govau commented Feb 25, 2022

peternied commented Mar 4, 2022

ae-govau commented Mar 16, 2022

peternied commented Mar 17, 2022

cliu123 commented Mar 20, 2022

ch-govau commented Mar 20, 2022

codecov-commenter commented Mar 20, 2022 • edited Loading

Codecov Report

ae-govau commented Mar 28, 2022

cliu123 commented Mar 28, 2022

ch-govau commented Jan 27, 2022 •

edited by peternied

Loading

codecov-commenter commented Mar 20, 2022 •

edited

Loading