{Core} Vendor track 2 subscription SDK in azure-cli-core #15711

jiasli · 2020-10-29T05:09:53Z

Description

Fix #15496: azure-cli-core has hard dependency on previous version of azure-mgmt-resource
Workaround #15530: Bump azure-mgmt-resource to Track 2 15.0.0
Replace PR #15669: {Core} Migrate azure-cli-core to use azure-mgmt-resource Track 2 SDK

Vendor track 2 subscription SDK in azure-cli-core so that it doesn't have dependency on azure-mgmt-resource, which is required by resource command module.

In this way, azure-cli-core can start using Track 2 azure-mgmt-resource SDK before resource finishes the migration.

To switch back to using public SDK, simply change _USE_VENDEROED_SUBSCRIPTION_SDK to False.

yonzhan · 2020-10-29T10:31:02Z

Core

# Conflicts: # src/azure-cli-core/azure/cli/core/_profile.py # src/azure-cli-core/azure/cli/core/adal_authentication.py # src/azure-cli-core/azure/cli/core/commands/client_factory.py # src/azure-cli-core/setup.py

jiasli · 2020-11-16T02:16:30Z

src/azure-cli-core/azure/cli/core/_profile.py

        s = SubscriptionType()
-        s.state = StateType.enabled
+        s.state = 'Enabled'


Track 2 uses str instead of Enum SubscriptionState as the value when deserializing Subscription objects, so we do the same.

Track 1:

https://github.com/Azure/azure-sdk-for-python/blob/azure-mgmt-resource_10.2.0/sdk/resources/azure-mgmt-resource/azure/mgmt/resource/subscriptions/v2019_11_01/models/_models_py3.py#L390

'state': {'key': 'state', 'type': 'SubscriptionState'},

Track 2:

https://github.com/Azure/azure-sdk-for-python/blob/azure-mgmt-resource_15.0.0/sdk/resources/azure-mgmt-resource/azure/mgmt/resource/subscriptions/v2019_11_01/models/_models_py3.py#L453

'state': {'key': 'state', 'type': 'str'},

See email: A breaking change in Enum property's deserialization

jiasli · 2020-11-16T02:17:13Z

src/azure-cli-core/azure/cli/core/_profile.py

@@ -449,8 +458,7 @@ def _match_account(account, subscription_id, secondary_key_name, secondary_key_v

    @staticmethod
    def _pick_working_subscription(subscriptions):
-        from azure.mgmt.resource.subscriptions.models import SubscriptionState
-        s = next((x for x in subscriptions if x.get(_STATE) == SubscriptionState.enabled.value), None)
+        s = next((x for x in subscriptions if x.get(_STATE) == 'Enabled'), None)


Same as https://github.com/Azure/azure-cli/pull/15711/files#r523861234

jiasli · 2020-11-16T02:34:28Z

src/azure-cli-core/setup.py

-        'azure.cli.core.extension',
-        'azure.cli.core.profiles',
-    ],
+    packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests", "azure", "azure.cli"]),


Change the package finding logic so that vendored_sdks/ are included. Thank @arrownj for the help.

arrownj · 2020-11-16T06:37:10Z

src/azure-cli-core/azure/cli/core/_profile.py

@@ -77,6 +77,8 @@

 _AZ_LOGIN_MESSAGE = "Please run 'az login' to setup account."

+_USE_VENDORED_SUBSCRIPTION_SDK = True


Do we need to change this to False in the future ?

Probably not. azure-cli-core uses only very limited functionalities from this SDK, so we can always use the fixed version.

zhoxing-ms · 2020-11-17T06:14:00Z

src/azure-cli-core/azure/cli/core/_profile.py

            api_version = get_api_version(cli_ctx, ResourceType.MGMT_RESOURCE_SUBSCRIPTIONS)
+            client_kwargs = _prepare_client_kwargs_track2(cli_ctx)
+            # We don't need to change credential_scopes as 'scopes' is ignored by BasicTokenCredential anyway
            client = client_type(credentials, api_version=api_version,
-                                 base_url=self.cli_ctx.cloud.endpoints.resource_manager)


What api-version of Subscription is used by azure-cli, the corresponding version of vendor SDK is required for azure-cli-core, right?

Only 2019-11-01 and 2016-06-01 are used and they are included under vendored_sdks.

azure-cli/src/azure-cli-core/azure/cli/core/profiles/_shared.py

Line 155 in 0952571

ResourceType.MGMT_RESOURCE_SUBSCRIPTIONS: '2019-11-01',

azure-cli/src/azure-cli-core/azure/cli/core/profiles/_shared.py

Line 225 in 0952571

ResourceType.MGMT_RESOURCE_SUBSCRIPTIONS: '2016-06-01',

Yes, I know it~ I mean if a new api-version of Subscription is added to azure-cli in the feture, does the vendor SDK in azure-cli-core also need to change to this new api-version, right?

MGMT_RESOURCE_SUBSCRIPTIONS is now only used by azure-cli-core. We can use it for now, assuming the corresponding api-version is vendored correctly.

jsntcy · 2020-11-18T03:19:59Z

src/azure-cli-core/azure/cli/core/adal_authentication.py

+        self.access_token = access_token
+
+    def get_token(self, *scopes, **kwargs):  # pylint:disable=unused-argument
+        return AccessToken(self.access_token, int(time.time() + 3600))


Can we use a meaningful constant instead of a magic number (3600) so that reviewers can understand it easier?@jiasli@microsoft.com

As self.access_token is a fixed value, get_token cannot refresh the token, thus defeating the purpose of returning a valid expires_on. Using time.time() + 3600 will force SDK to use the provided token, regardless of whether the token is expired or not.

Also, Subscriptions - List and Tenants - List APIs are not long-running operations, so the time of expires_on will not be reached anyway.

jsntcy

jiasli requested review from arrownj, fengzhou-msft, haroldrandom, jsntcy, Juliehzl and mmyyrroonn as code owners October 29, 2020 05:09

Vendor subscription SDK

748e1ff

jiasli force-pushed the vendor-subscription-sdk branch from 83f9c82 to 748e1ff Compare October 29, 2020 06:29

jiasli added 3 commits October 29, 2020 15:19

Add switch _USE_VENDEROED_SUBSCRIPTION_SDK

3147c99

Add __init__

a35910c

Use find_packages

2e7d18e

yonzhan assigned jiasli Oct 29, 2020

yonzhan requested review from evelyn-ys and zhoxing-ms October 29, 2020 10:30

yonzhan added this to the S178 milestone Oct 29, 2020

jiasli added 2 commits November 4, 2020 10:34

Tab to space

f4cec0a

Typo

2bdd5f2

jiasli changed the title ~~{Core} Vendor subscription SDK in azure-cli-core~~ {Core} Vendor track 2 subscription SDK in azure-cli-core Nov 12, 2020

jiasli mentioned this pull request Nov 12, 2020

Bump azure-mgmt-resource to Track 2 15.0.0 #15530

Closed

yonzhan modified the milestones: S178, S179 Nov 14, 2020

jiasli added 2 commits November 16, 2020 09:54

Merge branch 'dev' into vendor-subscription-sdk

96a823c

# Conflicts: # src/azure-cli-core/azure/cli/core/_profile.py # src/azure-cli-core/azure/cli/core/adal_authentication.py # src/azure-cli-core/azure/cli/core/commands/client_factory.py # src/azure-cli-core/setup.py

Refine _new_account

a216f4e

jiasli commented Nov 16, 2020

View reviewed changes

jiasli added 3 commits November 16, 2020 11:04

Style

83fe6a1

Dependency

5ecb89a

Fix test_subscription_finder_fail_on_arm_client_factory

8ff4b54

arrownj reviewed Nov 16, 2020

View reviewed changes

zhoxing-ms reviewed Nov 17, 2020

View reviewed changes

jiasli removed the request for review from mmyyrroonn November 17, 2020 07:02

arrownj approved these changes Nov 17, 2020

View reviewed changes

zhoxing-ms approved these changes Nov 17, 2020

View reviewed changes

jsntcy reviewed Nov 18, 2020

View reviewed changes

jsntcy approved these changes Nov 18, 2020

View reviewed changes

Add comments

081d233

jiasli added the Track2 label Nov 19, 2020

jiasli merged commit 3ade968 into Azure:dev Nov 19, 2020

jiasli mentioned this pull request Dec 7, 2020

azure-cli-core has hard dependency on previous version of azure-mgmt-resource #15496

Closed

jiasli deleted the vendor-subscription-sdk branch February 19, 2021 07:37

jiasli mentioned this pull request Mar 31, 2021

{Core} Quick patch for managed_by_tenants missing from response #17509

Closed

jiasli mentioned this pull request Apr 22, 2021

{Core} Add __init__.py to azure.cli.core.sdk #17817

Merged

jiasli mentioned this pull request Sep 14, 2021

{Core} Drop azure-common from azure-cli-core #19551

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

{Core} Vendor track 2 subscription SDK in azure-cli-core #15711

{Core} Vendor track 2 subscription SDK in azure-cli-core #15711

jiasli commented Oct 29, 2020 •

edited

Loading

yonzhan commented Oct 29, 2020

jiasli Nov 16, 2020 •

edited

Loading

jiasli Nov 16, 2020

jiasli Nov 16, 2020

arrownj Nov 16, 2020

jiasli Nov 16, 2020

zhoxing-ms Nov 17, 2020

jiasli Nov 17, 2020

zhoxing-ms Nov 17, 2020 •

edited

Loading

jiasli Nov 17, 2020

jsntcy Nov 18, 2020 •

edited

Loading

jiasli Nov 18, 2020

jsntcy left a comment

		@@ -77,6 +77,8 @@

		_AZ_LOGIN_MESSAGE = "Please run 'az login' to setup account."

		_USE_VENDORED_SUBSCRIPTION_SDK = True

{Core} Vendor track 2 subscription SDK in azure-cli-core #15711

{Core} Vendor track 2 subscription SDK in azure-cli-core #15711

Conversation

jiasli commented Oct 29, 2020 • edited Loading

yonzhan commented Oct 29, 2020

jiasli Nov 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhoxing-ms Nov 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsntcy Nov 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsntcy left a comment

Choose a reason for hiding this comment

jiasli commented Oct 29, 2020 •

edited

Loading

jiasli Nov 16, 2020 •

edited

Loading

zhoxing-ms Nov 17, 2020 •

edited

Loading

jsntcy Nov 18, 2020 •

edited

Loading