Remove resources post test (#13379)

* pruning tests.yml

* potential fix

* updated environment variables

* another change to tests.yml

* change

* edits to testcase

* setting storage_name and rg_kwargs to None to help identify problem

* forcing i_need_to_create_rg to be True

* flirting with version change

* reverting back to normal for both testcase and version

* Packaging update of azure-data-tables

* adding azure identity to dev reqs

* adding azure-identity to dev_reqs

* mirror updates from personal branch

* adding continue on error so we can see the env

* update shared test case rg_kwargs resource to mapping instead of NoneType

* updating dev_reqs

* random issues gone?

* build-test changes

* undoing all the bot changes from AutorestCI

* edits thanks to Kristas suggestion, hopefully stops AutorestCI from sabotage

* import directly from file, python2 can't find the module

* pylint fixes

* debugging why python 2.7 can't find azure data tables

* undoing last change

* reverting setup.py to master branch, adding space in inits just incase

* add comprehension of nspkg to azure-data-tables

* adding nspkg to shared_reqs

* updating shared_req to freeze nspkg at the same version

Co-authored-by: Azure SDK Bot <adxpysdk@microsoft.com>
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>

sdk/tables/azure-data-tables/MANIFEST.in

@@ -1,6 +1,7 @@
+recursive-include tests *.py *.yaml
 include *.md
 include azure/__init__.py
 include azure/data/__init__.py
 include LICENSE.txt
 recursive-include tests *.py
-recursive-include samples *.py *.md
+recursive-include samples *.py *.md

sdk/tables/azure-data-tables/README.md

@@ -1,16 +1,17 @@
 # Azure Data Tables client library for Python
 
-Azure Data Tables is a NoSQL data storing service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. 
-Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing. 
-Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing. 
+Azure Data Tables is a NoSQL data storing service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS.
+Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing.
 The Azure Data Tables client can be used to access Azure Storage or Cosmos accounts.
 
-Common uses of Azure Data Tables include:
 
-* Storing structured data in the form of tables
+# Usage
+* Storing structured data in the form of tables	# Usage
 * Quickly querying data using a clustered index
 
 [Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) | [Package (PyPI)](https://pypi.org) | [API reference documentation](https://aka.ms/azsdk/python/tables/docs) | [Product documentation](https://docs.microsoft.com/azure/storage/) | [Samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk)
+For code examples, see [MyService Management](https://docs.microsoft.com/python/api/overview/azure/)
+on docs.microsoft.com.
 
 ## Getting started
 
@@ -38,7 +39,6 @@ or [Azure CLI](https://docs.microsoft.com/azure/storage/common/storage-quickstar
 # Create a new resource group to hold the storage account -
 # if using an existing resource group, skip this step
 az group create --name MyResourceGroup --location westus2
-
 # Create the storage account
 az storage account create -n mystorageaccount -g MyResourceGroup
 ```
@@ -51,12 +51,11 @@ you to access the account:
 
 ```python
 from azure.data.tables import TableServiceClient
-
 service = TableServiceClient(account_url="https://<myaccount>.table.core.windows.net/", credential=credential)
 ```
 
 #### Looking up the account URL
-You can find the account's table service URL using the 
+You can find the account's table service URL using the
 [Azure Portal](https://docs.microsoft.com/azure/storage/common/storage-account-overview#storage-account-endpoints),
 [Azure PowerShell](https://docs.microsoft.com/powershell/module/az.storage/get-azstorageaccount),
 or [Azure CLI](https://docs.microsoft.com/cli/azure/storage/account?view=azure-cli-latest#az-storage-account-show):
@@ -77,24 +76,23 @@ The `credential` parameter may be provided in a number of different forms, depen
 ```python
     from datetime import datetime, timedelta
     from azure.data.tables import TableServiceClient, generate_account_sas, ResourceTypes, AccountSasPermissions
-    
+
     sas_token = generate_account_sas(
         account_name="<account-name>",
         account_key="<account-access-key>",
         resource_types=ResourceTypes(service=True),
         permission=AccountSasPermissions(read=True),
         expiry=datetime.utcnow() + timedelta(hours=1)
     )
-    
+
     table_service_client = TableServiceClient(account_url="https://<my_account_name>.table.core.windows.net", credential=sas_token)
 ```
 
 2. To use an account [shared key](https://docs.microsoft.com/rest/api/storageservices/authenticate-with-shared-key/)
-   (aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys" 
+   (aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys"
    section or by running the following Azure CLI command:
 
     ```az storage account keys list -g MyResourceGroup -n mystorageaccount```
-
     Use the key as the credential parameter to authenticate the client:
     ```python
     from azure.data.tables import TableServiceClient
@@ -103,12 +101,11 @@ The `credential` parameter may be provided in a number of different forms, depen
 
 #### Creating the client from a connection string
 Depending on your use case and authorization method, you may prefer to initialize a client instance with a
-connection string instead of providing the account URL and credential separately. To do this, pass the 
+connection string instead of providing the account URL and credential separately. To do this, pass the
 connection string to the client's `from_connection_string` class method:
 
 ```python
 from azure.data.tables import TableServiceClient
-
 connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net"
 service = TableServiceClient.from_connection_string(conn_str=connection_string)
 ```
@@ -139,14 +136,14 @@ Two different clients are provided to to interact with the various components of
     this client represents interaction with a specific table (which need not exist yet). It provides operations to
     create, delete, or update a table and includes operations to query, get, and upsert entities
     within it.
-    
+
 ### Entities
 * **Create** - Adds an entity to the table.
 * **Delete** - Deletes an entity from the table.
 * **Update** - Updates an entities information by either merging or replacing the existing entity.
 * **Query** - Queries existing entities in a table based off of the QueryOptions (OData).
 * **Get** - Gets a specific entity from a table by partition and row key.
-* **Upsert** - Merges or replaces an entity in a table, or if the entity does not exist, inserts the entity. 
+* **Upsert** - Merges or replaces an entity in a table, or if the entity does not exist, inserts the entity.
 
 ## Examples
 
@@ -162,7 +159,6 @@ Create a table in your account
 
 ```python
 from azure.data.tables import TableServiceClient
-
 table_service_client = TableServiceClient.from_connection_string(conn_str="<connection_string>")
 table_service_client.create_table(table_name="myTable")
 ```
@@ -172,9 +168,7 @@ Create entities in the table
 
 ```python
 from azure.data.tables import TableClient
-
 my_entity = {'PartitionKey':'part','RowKey':'row'}
-
 table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="myTable")
 entity = table_client.create_entity(entity=my_entity)
 ```
@@ -184,9 +178,7 @@ Querying entities in the table
 
 ```python
 from azure.data.tables import TableClient
-
 my_filter = "text eq Marker"
-
 table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="mytable")
 entity = table_client.query_entities(filter=my_filter)
 ```
@@ -246,15 +238,12 @@ headers, can be enabled on a client with the `logging_enable` argument:
 import sys
 import logging
 from azure.data.tables import TableServiceClient
-
 # Create a logger for the 'azure.data.tables' SDK
 logger = logging.getLogger('azure.data.tables')
 logger.setLevel(logging.DEBUG)
-
 # Configure a console output
 handler = logging.StreamHandler(stream=sys.stdout)
 logger.addHandler(handler)
-
 # This client will log detailed information about its HTTP sessions, at DEBUG level
 service_client = TableServiceClient.from_connection_string("your_connection_string", logging_enable=True)
 ```
@@ -287,7 +276,7 @@ Several Azure Data Tables Python SDK samples are available to you in the SDK's G
     * Query entities
     * Update entities
     * Upsert entities
-    
+
 ### Additional documentation
 For more extensive documentation on Azure Data Tables, see the [Azure Data Tables documentation](https://docs.microsoft.com/azure/storage/tables/) on docs.microsoft.com.
 
@@ -296,4 +285,4 @@ This project welcomes contributions and suggestions.  Most contributions require
 
 When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
 
-This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
+This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

sdk/tables/azure-data-tables/dev_requirements.txt

@@ -3,4 +3,5 @@
 ../../core/azure-core
 cryptography>=2.1.4
 aiohttp>=3.0; python_version >= '3.5'
-
+azure-identity
+../azure-data-nspkg

sdk/tables/azure-data-tables/sdk_packaging.toml

@@ -0,0 +1,2 @@
+[packaging]
+auto_update = false

sdk/tables/azure-data-tables/setup.py

@@ -70,15 +70,16 @@
         # Exclude packages that will be covered by PEP420 or nspkg
         'azure',
         'tests',
+        'azure.data',
     ]),
     install_requires=[
         "azure-core<2.0.0,>=1.2.2",
         "msrest>=0.6.10"
         # azure-data-tables
     ],
     extras_require={
-        ":python_version<'3.0'": ['futures'],
+        ":python_version<'3.0'": ['futures', 'azure-data-nspkg<2.0.0,>=1.0.0'],
         ":python_version<'3.4'": ['enum34>=1.0.4'],
         ":python_version<'3.5'": ["typing"]
     },
-)
+)

sdk/tables/azure-data-tables/tests/_shared/testcase.py

@@ -13,9 +13,6 @@
 import time
 from datetime import datetime, timedelta
 
-from azure.data.tables import ResourceTypes, AccountSasPermissions
-from azure.data.tables._table_shared_access_signature import generate_account_sas
-
 try:
     import unittest.mock as mock
 except ImportError:
@@ -42,8 +39,8 @@
     from io import StringIO
 
 from azure.core.credentials import AccessToken
-#from azure.data.tabless import generate_account_sas, AccountSasPermissions, ResourceTypes
 from azure.mgmt.storage.models import StorageAccount, Endpoints
+from azure.data.tables import generate_account_sas, AccountSasPermissions, ResourceTypes
 
 try:
     from devtools_testutils import mgmt_settings_real as settings
@@ -351,6 +348,9 @@ def storage_account():
     i_need_to_create_rg = not (existing_rg_name or existing_storage_name or storage_connection_string)
     got_storage_info_from_env = existing_storage_name or storage_connection_string
 
+    storage_name = None
+    rg_kwargs = {}
+
     try:
         if i_need_to_create_rg:
             rg_name, rg_kwargs = rg_preparer._prepare_create_resource(test_case)
@@ -431,11 +431,12 @@ def build_service_endpoint(service):
             TableTestCase._STORAGE_CONNECTION_STRING = storage_connection_string
             yield
         finally:
-            if not got_storage_info_from_env:
-                storage_preparer.remove_resource(
-                    storage_name,
-                    resource_group=rg
-                )
+            if storage_name is not None:
+                if not got_storage_info_from_env:
+                    storage_preparer.remove_resource(
+                        storage_name,
+                        resource_group=rg
+                    )
     finally:
         if i_need_to_create_rg:
             rg_preparer.remove_resource(rg_name)

sdk/tables/tests.yml

@@ -5,29 +5,12 @@ jobs:
     parameters:
       BuildTargetingString: azure-data-tables
       ServiceDirectory: tables
+      AllocateResourceGroup: 'false'
       EnvVars:
-        STORAGE_ACCOUNT_NAME: $(python-storage-storage-account-name)
-        STORAGE_ACCOUNT_KEY: $(python-storage-storage-account-key)
-        STORAGE_DATA_LAKE_ACCOUNT_NAME: $(python-storage-data-lake-account-name)
-        STORAGE_DATA_LAKE_ACCOUNT_KEY: $(python-storage-data-lake-account-key)
-        BLOB_STORAGE_ACCOUNT_NAME: $(python-storage-blob-storage-account-name)
-        BLOB_STORAGE_ACCOUNT_KEY: $(python-storage-blob-storage-account-key)
-        REMOTE_STORAGE_ACCOUNT_NAME: $(python-storage-remote-storage-account-name)
-        REMOTE_STORAGE_ACCOUNT_KEY: $(python-storage-remote-storage-account-key)
-        PREMIUM_STORAGE_ACCOUNT_NAME: $(python-storage-premium-storage-account-name)
-        PREMIUM_STORAGE_ACCOUNT_KEY: $(python-storage-premium-storage-account-key)
-        OAUTH_STORAGE_ACCOUNT_NAME: $(python-storage-oauth-storage-account-name)
-        OAUTH_STORAGE_ACCOUNT_KEY: $(python-storage-oauth-storage-account-key)
-        ACTIVE_DIRECTORY_APPLICATION_ID: $(aad-azure-sdk-test-client-id)
-        ACTIVE_DIRECTORY_APPLICATION_SECRET: $(aad-azure-sdk-test-client-secret)
-        ACTIVE_DIRECTORY_TENANT_ID: $(aad-azure-sdk-test-tenant-id)
-        CONNECTION_STRING: $(python-storage-blob-connection-string)
-        BLOB_CONNECTION_STRING: $(python-storage-blob-connection-string)
-        PREMIUM_CONNECTION_STRING: $(python-storage-premium-connection-string)
-        TEST_MODE: 'RunLiveNoRecord'
-        AZURE_SKIP_LIVE_RECORDING: 'True'
-        AZURE_TEST_RUN_LIVE: 'true'
         AZURE_TENANT_ID: $(aad-azure-sdk-test-tenant-id)
         AZURE_SUBSCRIPTION_ID: $(azure-subscription-id)
-        AZURE_CLIENT_SECRET: $(aad-azure-sdk-test-client-secret)
         AZURE_CLIENT_ID: $(aad-azure-sdk-test-client-id)
+        AZURE_CLIENT_SECRET: $(aad-azure-sdk-test-client-secret)
+        TEST_MODE: 'RunLiveNoRecord'
+        AZURE_SKIP_LIVE_RECORDING: 'True'
+        AZURE_TEST_RUN_LIVE: 'true'

shared_requirements.txt

@@ -10,6 +10,7 @@ azure-common~=1.1
 azure-core<2.0.0,>=1.2.2
 azure-cosmosdb-table~=1.0
 azure-datalake-store~=0.0.18
+azure-data-nspkg<2.0.0,>=1.0.0
 azure-eventhub<6.0.0,>=5.0.0
 azure-eventgrid~=1.1
 azure-graphrbac~=0.40.0