Auto-convert V6 configuration instances into V7 configuration instances

[nibix]

Description

This relates to the alternative proposal "Behind the scenes conversion from V6 to V7" from #4493.

For now, this is a draft. It is to illustrate what could be done to convert the V6 config instances to V7 config instances "behind the scenes". This allows to save a bit of duplicate implementation regarding to privilege evaluation and possibly also further logic.

As an additional benefit, this makes it possible to use SecurityDynamicConfiguration<> instances in a generically type-safe manner. So far, one usually needs to use SecurityDynamicConfiguration<?> instances and do unsafe casts.

This only touches the runtime representation of the configuration. If there's a V6 configuration in the configuration index, it remains untouched. The possibility to update configuration in V6 format will remain.

For now, this serves for illustrating what can be done. It is mostly complete, but might require some more attention to some details and polishing to merge it.

Behind the scenes conversion

Some more detail on what is changed by this PR:

The configuration architecture consists of these levels:

1. JSON documents in the configuration index
2. ConfigurationRepository which manages SecurityDynamicConfiguration<> instances which contain as entry instances of org.opensearch.security.securityconf.impl.v6.RoleV6, org.opensearch.security.securityconf.impl.v6.RoleMappingsV6, org.opensearch.security.securityconf.impl.v7.RoleV7, org.opensearch.security.securityconf.impl.v7.RoleMappingsV7, etc.
3. DynamicConfigFactory which consumes SecurityDynamicConfiguration<> instances and creates ConfigModel and DynamicConfigModel instances. If it finds V6 classes in SecurityDynamicConfiguration<>, it creates ConfigModelV6 instances, if it finds V7 classes, it creates ConfigModelV7 instances.
4. The ConfigModelV6 and ConfigModelV7 classes contain business logic which evaluates roles and permissions.
5. This is used by PrivilegesEvaluator.

This PR changes the architecture at level 2:

1. The JSON documents in the index do not change
2. ConfigurationRepository is changed in this way: If it finds V6 instances like org.opensearch.security.securityconf.impl.v6.RoleV6, it auto-converts these to V7 instances like org.opensearch.security.securityconf.impl.v7.RoleV7. The V7 classes already have functionality for that which is already used for the securityadmin migration command.
3. This makes the distinction of V6/V7 on all layers below unnecessary. DynamicConfigFactory does no longer need a distinction between two config versions.
4. ConfigModelV6 becomes redundant.
5. As PrivilegeEvaluator consumes the super interface ConfigModel, it does not notice anything about this change.

• Category: Refactoring
• Why these changes are required? - Minimize duplicate logic for privilege evaluation
• What is the old behavior before changes and new behavior after changes? - none

Issues Resolved

• Partially solves [RFC] Retire support for V6 configuration #4493

Testing

[Please provide details of testing done: unit testing, integration testing and manual testing]

Check List

• New functionality includes testing
• New functionality has been documented
• New Roles/Permissions have a corresponding security dashboards plugin PR
• API changes companion pull request created
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[DarshitChanpura]

Changes look good to me @nibix. Could you please check and resolve the conflict?

[DarshitChanpura]

👏 Much cleaner!

[DarshitChanpura]

why delete v6 test?

[nibix]

Now we convert "behind the scenes" any V6 config into a V7 config. Thus, the business logic which operates on the V6 config is no longer in use - that's especially the classes ConfigModelV6 and DynamicConfigModelV6. I actually forgot to delete these classes in the first commit - in a second commit, I have now deleted these classes.

As the SecurityRolesPermissionsV6Test tests the business logic in the ConfigModelV6 class, it has now nothing to test anymore.

[nibix]

BTW, I felt that I should describe the idea of the change a bit more in detail. See the updated description of this PR.

[cwperks]

With this change, would a user still be able to use the Migrate API to change the documents stored in the security index to the V7 format?

If a user uses securityadmin to export the security index, what format will the generated yml files be in?

[nibix]

@cwperks

With this change, would a user still be able to use the Migrate API to change the documents stored in the security index to the V7 format?
If a user uses securityadmin to export the security index, what format will the generated yml files be in?

In the end, it is up to us to define these behaviors. I would expect that both functionalities remain unaffected. Thus, exporting the configuratin will yield v6/v1 configuration files. Also, the migrate API and the migrate command continue to work as before.

[cwperks]

In the end, it is up to us to define these behaviors. I would expect that both functionalities remain unaffected. Thus, exporting the configuratin will yield v6/v1 configuration files. Also, the migrate API and the migrate command continue to work as before.

Sounds good. What work is remaining on this PR in order to flip it from Draft?

[nibix]

@cwperks

What work is remaining on this PR in order to flip it from Draft?

Mainly reviewing test failures and reviewing the special cases, like migration and API - as mentioned above.

So, I would then focus on this again?

[cwperks]

If its an enabler for #4380, then IMO I think we should move this forward