Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history
commit 7832c5a
Author: Azure CLI Bot <azclibot@microsoft.com>
Date:   Sat Sep 18 09:34:11 2021 +0800

    [AutoRelease] t2-eventhub-2021-09-17-55263(Do not merge) (Azure#20739)

    * CodeGen from PR 15979 in Azure/azure-rest-api-specs
    EventHub: Added missing Cluster API in 2021-06-01-preview (Azure#15979)

    * Added missing Cluster APIs to 2021-01-01-preview

    * added cluster API in 2021-06-01-preview

    * updated lint errors

    * update 1

    * added listbysubscription

    * fixed ModelValidation

    Co-authored-by: v-ajnava <v-ajnava@microsoft.com>

    * version,CHANGELOG

    Co-authored-by: SDKAuto <sdkautomation@microsoft.com>
    Co-authored-by: v-ajnava <v-ajnava@microsoft.com>
    Co-authored-by: PythonSdkPipelines <PythonSdkPipelines>

commit 29d98e8
Author: Chidozie Ononiwu (His Righteousness) <31145988+chidozieononiwu@users.noreply.github.com>
Date:   Fri Sep 17 17:27:57 2021 -0700

    Fix broken link in python repo (Azure#20746)

commit cbfacbd
Author: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>
Date:   Fri Sep 17 14:04:28 2021 -0700

    [Storage]Unify service version and update changelog (Azure#20723)

    * [Storage]Unify service version and update changelog

    * fix test

commit ea5ecea
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Fri Sep 17 10:19:31 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 2010 (Azure#20729)

    * Remove empty sections in changelog entry as part of prepare release

    * Update SECTIONS_HEADER_REGEX to accomodate various header levels

    Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>

commit 9b63801
Author: Azure CLI Bot <azclibot@microsoft.com>
Date:   Fri Sep 17 15:00:44 2021 +0800

    [AutoRelease] t2-azurearcdata-2021-09-15-74995 (Azure#20714)

    * CodeGen from PR 15874 in Azure/azure-rest-api-specs
    Adding 202 Accepted for deletion call (Azure#15874)

    * Adding 202 for deletion call

    * Adding headers to suppress warnings

    * version,CHANGELOG

    Co-authored-by: SDKAuto <sdkautomation@microsoft.com>
    Co-authored-by: PythonSdkPipelines <PythonSdkPipelines>

commit 0645f49
Author: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
Date:   Thu Sep 16 15:58:10 2021 -0700

    [Test Proxy] Make add_sanitizer a module-level method (Azure#20701)

commit ddf49b1
Author: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
Date:   Thu Sep 16 15:27:08 2021 -0700

    [Test proxy] Add migration guide (Azure#20469)

commit e116660
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Thu Sep 16 12:33:48 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 2011 (Azure#20702)

    * Common pipeline template and script to detect API changes

    * Cahnges as per review comments

    * Review comment changes

    * CCahgnes as per review comments

    * Show warning for list of failed packages to detect API changes

    * Apply suggestions from code review

    Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

    Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
    Co-authored-by: praveenkuttappan <55455725+praveenkuttappan@users.noreply.github.com>
    Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

commit 44e2acf
Author: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Date:   Thu Sep 16 10:57:23 2021 -0700

    protobuf to handle python 2.7 issues (Azure#20725)

commit 81c6aa0
Author: Xiang Yan <xiangsjtu@gmail.com>
Date:   Thu Sep 16 08:33:08 2021 -0700

    add troubleshoot doc (Azure#20684)

    * add troubleshoot doc

    * fix typo

    * update

    * update

    * update

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update sdk/identity/azure-identity/Troubleshoot.md

    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

    * Update Troubleshoot.md

    * update error log

    Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
    Co-authored-by: Charles Lowell <chlowe@microsoft.com>

commit 87e0f99
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Wed Sep 15 15:32:41 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 2016 (Azure#20706)

    * Make ServiceName optional for verify-changelog

    In cases like go where we don't pass a service name having this default to "not-specified" breaks things so we should allow for ServiceName and/or ServiceDirectory to be empty.

    * Put quotes around the strings to allow for empty

    Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

commit c80e26e
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Wed Sep 15 12:45:42 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 2009 (Azure#20672)

    * Skip PSModule caching in container jobs

    * Add succeeded check to condition

    Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

    Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
    Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
    Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

commit 1efff7f
Author: Travis Prescott <tjprescott@users.noreply.github.com>
Date:   Wed Sep 15 10:53:42 2021 -0700

    [Search] Add skillset validation (Azure#20669)

    * Add client-side validation and test.

    * Add skillset validation test.

    * Make code more Pythonic.

commit 6a06b0e
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Tue Sep 14 16:56:19 2021 -0700

    Add for multiple levels of Atx Headers in the CHANGELOG.md (Azure#20694)

    Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>

commit 3fdbaa7
Author: swathipil <76007337+swathipil@users.noreply.github.com>
Date:   Tue Sep 14 16:27:06 2021 -0700

    [EventHub] update arm template with storage conn str (Azure#20376)

    * update test resources

    * fix

    * fix failing tests

    * adams comments

    * pylint

    * remove dateutil

    * adams comments

    * nit

commit 279e7eb
Author: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Date:   Tue Sep 14 15:22:05 2021 -0700

    Add a new row type in query (Azure#20685)

    * Add row type

    * Add a new row type

    * add test

    * lint

    * Apply suggestions from code review

    * changes

commit 5016742
Author: luc <44377201+LuChen-Microsoft@users.noreply.github.com>
Date:   Tue Sep 14 13:09:43 2021 -0700

    Update changelog date (Azure#20686)

    * update swagger

    * update release date

commit 080f88d
Author: swathipil <76007337+swathipil@users.noreply.github.com>
Date:   Tue Sep 14 13:01:32 2021 -0700

    [SchemaRegistry] update type of `schema` in `serialize` (Azure#20683)

    * update type

    * update sample naming

commit 599a099
Author: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Date:   Tue Sep 14 09:41:19 2021 -0700

    Handle Errors and Tables in Query (Azure#20658)

    * initial commit

    * batch set

    * batch

    * tests + changes

    * more tests

    * lint

    * changelog

    * Apply suggestions from code review

    * comment

    * lint

    * querry

    * comments

commit 3252969
Author: swathipil <76007337+swathipil@users.noreply.github.com>
Date:   Tue Sep 14 09:17:18 2021 -0700

    [SchemaRegistry] remove codec param in serializer (Azure#20671)

    * remove codec

    * changelog

commit 27b7e75
Author: msyyc <70930885+msyyc@users.noreply.github.com>
Date:   Tue Sep 14 16:52:11 2021 +0800

    Update change_log.py (Azure#20680)

commit fd80cb7
Author: Azure CLI Bot <azclibot@microsoft.com>
Date:   Tue Sep 14 13:52:13 2021 +0800

    [AutoRelease] t2-storage-2021-09-14-45016(Do not merge) (Azure#20678)

    * CodeGen from PR 15627 in Azure/azure-rest-api-specs
    [SRP] 2021-06-01 Swagger Api (Azure#15627)

    * Add June21 Swagger Api version, Updated Readme files

    * adding abort and hnson migration swagger API

    * swagger: marking requesttype as required parameter for hns onmigration

    * Added new PublicNetworkAccess property to swagger spec

    * Add enableNfsV3RootSquash and enableNfsV3AllSquash to June21 swagger

    * Add Account Level VLW Swagger changes and example

    * Update Blob Inventory Api comment to include AccessTierInferred and Tags

    * Rename HnsOn to hierarchical namespace

    * [Swagger] [June21] Added defaultToOAuthAuthentication to swagger spec

    * Update Spell check custom words list. Correct incorrect spellings

    * Add missing refrences to PublicNetworkAccess Examples

    * Add required type:object, Add default return type for hns migration apis

    * Prettier tool update to storage.json

    * Add update account with immutability policy example

    * Removed StorageFileDataSmbShareOwner as Server side does not support it

    * Add AllowProtectedAppendWritesAll feature changes with example

    * Updated enum values and description for PublicNetworkAccess

    * Add type:object to ProtectedAppendWritesHistory ; Spellcheck

    * version,CHANGELOG

    Co-authored-by: SDKAuto <sdkautomation@microsoft.com>
    Co-authored-by: PythonSdkPipelines <PythonSdkPipelines>

commit 3eb0af9
Author: William Harding <william.jeffrey.harding@gmail.com>
Date:   Mon Sep 13 16:06:06 2021 -0700

    Updated notebooks to make them more "general" (Azure#15660)

    * Updated notebooks to make them more "general"

    * updated gitignore, added models, updated path

    * reverted gitignore

    * reverting gitignore

    * Renamed some files, changed model id to be more generic

commit 69be40d
Author: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
Date:   Mon Sep 13 15:35:26 2021 -0700

     [Key Vault] Update new test resource script (Azure#20663)

commit ce307fe
Author: Scott Addie <10702007+scottaddie@users.noreply.github.com>
Date:   Mon Sep 13 17:23:16 2021 -0500

    Final consistency review of Monitor Query README (Azure#20667)

    * Final consistency review of Monitor Query README

    * Add rate limits section

commit c4d64b2
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Mon Sep 13 13:50:44 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 1979 (Azure#20666)

    * Move logic for removing empty sections to ChangeLog-Operations.ps1

    * Refactor sections regex

    * Add SanitizeEntry parameter

    Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>

commit 3a537c9
Author: Travis Prescott <tjprescott@users.noreply.github.com>
Date:   Mon Sep 13 13:45:21 2021 -0700

    Update docstrings for custom models. (Azure#20592)

commit 90ce516
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Mon Sep 13 13:06:02 2021 -0700

    Fix SemVer.ToString to handle v0 case (Azure#20665)

    Since we are treating v0 versions as prerelease we need to make sure
    we don't accidently start to add the bogus prerelease label in cases where
    we call ToString() on the version.

    Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

commit dd67039
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Mon Sep 13 13:04:52 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 1946 (Azure#20656)

    * Add ability to exit gracefully when all files in the diff are excluded

    * Address case where cspell exits with an error when all files from the 'files' config list are excluded by the expressions in 'ignorePaths'

    * Add tests

    * Review feedback: impl goes at the bottom and should be treated as a script, logic for testing should happen above that and exit appropriately if running tests

    * Import common instead of logging

    * Enable strict mode

    Co-authored-by: Daniel Jurek <djurek@microsoft.com>

commit 7b2aeaf
Author: swathipil <76007337+swathipil@users.noreply.github.com>
Date:   Mon Sep 13 10:47:42 2021 -0700

    [EventHubs] update test to test async producer (Azure#19892)

    - call async Producer for testing/improving code coverage
    - remove passing in fake kwarg to PartitionContext; untested lines in PartitionContext can only be tested with a user implemented CheckpointStore class, so not worrying about this

commit 8f28e2a
Author: Leighton Chen <lechen@microsoft.com>
Date:   Mon Sep 13 10:37:18 2021 -0700

    [Monitor exporter] Add OTLP and dual exporter scenario to samples (Azure#20634)

    * rpc

    * samples

commit 5450368
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Mon Sep 13 09:21:54 2021 -0700

    Sync eng/common directory with azure-sdk-tools for PR 1983 (Azure#20618)

    * Doc Updates and Revisions for External Use

    The focus of these changes is to revise the script to better support use
    by external contributors and others outside of the Azure SDK ecosystem and
    without access to the Microsoft AAD Tenant.

    Changes include:

    - Creation of a new Test Application service principal is now possible
      from a non-Microsoft AAD tenant.

    - When a new Test Application principal is created, the principle of least
      privilege is now applied; the new Test Application is granted ownership
      of the resource group associated with the test resources and no longer
      has access to any other resources in the subscription.

    - If an existing Test Application principal is specified, it will be
      assigned ownership of the resource group created.  This supports using
      a Test Application principal without privileges at the subscription-level.

    - When no provisioner is specified, the script is now executed in the
      context of the caller rather than the Test Application principal.
      This supports using a Test Application principal that has restricted
      privileges and better aligns to the purpose of the Test Application
      principal.

    - The `$TestApplicationOid` is now explicitly bound at the time a new Test
      Application principal is created rather than having to query for it later.

    - Common error scenarios resulting from lack of permissions now provide
      messaging with more context of why the failure occurred and suggest
      remediation.

    - Added new examples to illustrate the common call patterns needed by
      external contributors running the script, outside of the Microsoft tenant
      and Azure SDK ecosystem.

    - Documentation has been enhanced with additional context to detail the
      permissions and roles assigned by the script.

    - Added documentation details for Bicep template use.

    * Add the provisioner OID to the deployment params

    Key Vault needs this to deploy Managed HSMs. There's a corresponding change necessary in test-resources.json I'll roll out across languages.

    * Fixing typos and spelling mistakes

    Co-authored-by: Jesse Squire <jesse.squire@gmail.com>
    Co-authored-by: Heath Stewart <heaths@microsoft.com>

commit b7b7e36
Author: Travis Prescott <tjprescott@users.noreply.github.com>
Date:   Mon Sep 13 09:10:38 2021 -0700

    [Search] Update SearchClient.search API (Azure#20602)

    * Update SearchClient.search API.

    * Update CHANGELOG and async SearchClient.

    * Update parameter parsing for omitted values.

    * Code review comments and linter fixes.

commit 4048684
Author: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Date:   Sun Sep 12 15:49:22 2021 -0700

    Increment version for monitor releases (Azure#20635)

    Increment package version after release of azure-monitor-query
  • Loading branch information
zihzhan-msft committed Sep 20, 2021
1 parent e8cf9b2 commit 4ec3539
Show file tree
Hide file tree
Showing 160 changed files with 22,011 additions and 187 deletions.
2 changes: 1 addition & 1 deletion doc/dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Overview of the documents:
- [Developer Set-Up](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/dev_setup.md) : How to create a development environment for this repo
- [Release](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/release.md) : How to release a package when ready
- [Packaging](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/packaging.md) : How to organize packaging information for packages under `azure`
- [Testing](./tests.md): How to write unit and functional tests for a library
- [Testing](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/tests.md): How to write unit and functional tests for a library
- [Docstrings and Type hints](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/docstring_typehint.md): How to document an SDK for compatability with internal tools (API View) and our documentation at [MS Docs][ms_docs] and the [azure.github.io][azure_github_io] site.

The [mgmt](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/mgmt) folder contains information specific to management packages (i.e. packages prefixed by `azure-mgmt`)
Expand Down
2 changes: 1 addition & 1 deletion doc/dev/docstring_typehint.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ All public methods should have docstrings to document the parameters, keywords,

* [Docstrings](#docstrings)
* [Method Docstrings](#method_docstrings)
* [Model and Client Docstrings](#Model_and_Client_Docstrings)
* [Model and Client Docstrings](#model_and_Client_Docstrings)
* [Type Hints](#type_hints)
* [Type Hints for Python 2.7 and 3.5+](#type_hints_for_python_2.7_and_3.5+)
* [Type Hints for Python 3.5+](#type_hints_for_python_3.5+)
Expand Down
176 changes: 176 additions & 0 deletions doc/dev/test_proxy_migration_guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
# Guide for migrating to the test proxy from vcrpy

This guide describes the changes that service SDKs should make to their test frameworks in order to take advantage of
the Azure SDK test proxy.

Documentation of the motivations and goals of the test proxy can be found [here][general_docs] in the azure-sdk-tools
GitHub repository, and documentation of how to set up and use the proxy can be found [here][detailed_docs].

## Update existing tests

### Current test structure

Test classes currently inherit from AzureTestCase, and test methods can optionally use decorators:

```py
from devtools_testutils import AzureTestCase

class TestExample(AzureTestCase):

def test_example(self):
...

@ExamplePreparer()
def test_example_with_preparer(self):
...
```

### New test structure

To use the proxy, test classes should inherit from AzureRecordedTestCase and recorded test methods should use a
RecordedByProxy decorator:

```py
from devtools_testutils import AzureRecordedTestCase, RecordedByProxy

class TestExample(AzureRecordedTestCase):

@RecordedByProxy
def test_example(self):
...

@ExamplePreparer()
@RecordedByProxy
def test_example_with_preparer(self):
...
```

For async tests, import the RecordedByProxyAsync decorator from `devtools_testutils.aio` and use it in the same
way as RecordedByProxy.

> **Note:** since AzureRecordedTestCase doesn't inherit from `unittest.TestCase`, test class names need to start
> with "Test" in order to be properly collected by pytest by default. For more information, please refer to
> [pytest's documentation][pytest_collection].
## Run the tests

### Perform one-time setup

The test proxy is made available for your tests via a Docker container. Some tests require an SSL connection to work, so
the Docker image used for the container has a certificate imported that you need to trust on your machine. Instructions
on how to do so can be found [here][proxy_cert_docs].

### Start the proxy server

There is a [PowerShell script][docker_start_proxy] in `eng/common/testproxy` that will fetch the proxy Docker image if
you don't already have it, and will start or stop a container running the image for you. You can run the following
command from the root of the `azure-sdk-for-python` directory to start the container whenever you want to make the test
proxy available for running tests:

```powershell
.\eng\common\testproxy\docker-start-proxy.ps1 "start"
```

Note that the proxy is available as long as the container is running. In other words, you don't need to start and
stop the container for each test run or between tests for different SDKs. You can run the above command in the morning
and just stop the container whenever you'd like. To stop the container, run the same command but with `"stop"` in place
of `"start"`. In the future, the proxy container will be set up and started automatically when tests are run, and
starting it manually will be optional.

For more details on proxy startup, please refer to the [proxy documentation][detailed_docs].

### Record or play back tests

Configuring live and playback tests is done with the `AZURE_TEST_RUN_LIVE` environment variable. When this variable is
set to "true" or "yes", live tests will run and produce recordings. When this variable is set to "false" or "no", or
not set at all, tests will run in playback mode and attempt to match existing recordings.

Recordings for a given package will end up in that package's `/tests/recordings` directory, just like they currently
do.

> **Note:** at this time, support for configuring live or playback tests with a `testsettings_local.cfg` file has been
> deprecated in favor of using just `AZURE_TEST_RUN_LIVE`.
### Register sanitizers

Since the test proxy doesn't use [`vcrpy`][vcrpy], tests don't use a scrubber to sanitize values in recordings.
Instead, sanitizers (as well as matchers and transforms) can be registered on the proxy as detailed in
[this][sanitizers] section of the proxy documentation. At the time of writing, sanitizers can be registered via the
`add_sanitizer` method in `devtools_testutils`.

Sanitizers, matchers, and transforms remain registered until the proxy container is stopped, so for any sanitizers that
are shared by different tests, using a session fixture declared in a `conftest.py` file is recommended. Please refer to
[pytest's scoped fixture documentation][pytest_fixtures] for more details.

For example, to sanitize URIs in recordings, you can set up a URI sanitizer for all tests in the pytest session by
adding something like the following in the package's `conftest.py` file:

```python
from devtools_testutils import add_sanitizer

@pytest.fixture(scope="session")
def sanitize_uris():
add_sanitizer(ProxyRecordingSanitizer.URI, value="fakeendpoint")
```

`add_sanitizer` accepts a sanitizer, matcher, or transform type from the ProxyRecordingSanitizer enum as a required
parameter. Keyword-only arguments can be provided to customize the sanitizer; for example, in the snippet above, any
request URIs that match the default URI regular expression will have their domain name replaced with "fakeendpoint". A
request made to `https://tableaccount.table.core.windows.net` will be recorded as being made to
`https://fakeendpoint.table.core.windows.net`.

## Implementation details

### What does the test proxy do?

The gist of the test proxy is that it stands in between your tests and the service. What this means is that test
requests which would usually go straight to the service should instead point to the locally-hosted test proxy.

For example, if an operation would typically make a GET request to
`https://fakeazsdktestaccount.table.core.windows.net/Tables`, that operation should now be sent to
`https://localhost:5001/Tables` instead. The original endpoint should be stored in an `x-recording-upstream-base-uri` --
the proxy will send the original request and record the result.

The RecordedByProxy and RecordedByProxyAsync decorators patch test requests to do this for you.

### How does the test proxy know when and what to record or play back?

This is achieved by making POST requests to the proxy server that say whether to start or stop recording or playing
back, as well as what test is being run.

To start recording a test, the server should be primed with a POST request:

```
URL: https://localhost:5001/record/start
headers {
"x-recording-file": "<path-to-test>/recordings/<testfile>.<testname>"
}
```

This will return a recording ID in an `x-recording-id` header. This ID should be sent as an `x-recording-id` header in
all further requests during the test.

After the test has finished, a POST request should be sent to indicate that recording is complete:

```
URL: https://localhost:5001/record/stop
headers {
"x-recording-id": "<x-recording-id>"
}
```

Running tests in playback follows the same pattern, except that requests will be sent to `/playback/start` and
`/playback/stop` instead. A header, `x-recording-mode`, should be set to `record` for all requests when recording and
`playback` when playing recordings back. More details can be found [here][detailed_docs].

The RecordedByProxy and RecordedByProxyAsync decorators send the appropriate requests at the start and end of each test
case.

[detailed_docs]: https://github.com/Azure/azure-sdk-tools/tree/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
[docker_start_proxy]: https://github.com/Azure/azure-sdk-for-python/blob/main/eng/common/testproxy/docker-start-proxy.ps1
[general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/README.md
[proxy_cert_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/documentation/trusting-cert-per-language.md
[pytest_collection]: https://docs.pytest.org/latest/goodpractices.html#test-discovery
[pytest_fixtures]: https://docs.pytest.org/latest/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session
[sanitizers]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md#session-and-test-level-transforms-sanitiziers-and-matchers
[vcrpy]: https://vcrpy.readthedocs.io
19 changes: 19 additions & 0 deletions eng/common/pipelines/templates/steps/detect-api-changes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
parameters:
ArtifactPath: $(Build.ArtifactStagingDirectory)
Artifacts: []

steps:
- task: Powershell@2
inputs:
filePath: $(Build.SourcesDirectory)/eng/common/scripts/Detect-Api-Changes.ps1
arguments: >
-ArtifactList ('${{ convertToJson(parameters.Artifacts) }}' | ConvertFrom-Json | Select-Object Name)
-ArtifactPath ${{parameters.ArtifactPath}}
-CommitSha '$(Build.SourceVersion)'
-BuildId $(Build.BuildId)
-PullRequestNumber $(System.PullRequest.PullRequestNumber)
-RepoFullName $(Build.Repository.Name)
pwsh: true
workingDirectory: $(Pipeline.Workspace)
displayName: Detect API changes
condition: and(succeededOrFailed(), eq(variables['Build.Reason'],'PullRequest'))
28 changes: 19 additions & 9 deletions eng/common/scripts/ChangeLog-Operations.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
. "${PSScriptRoot}\SemVer.ps1"

$RELEASE_TITLE_REGEX = "(?<releaseNoteTitle>^\#+\s+(?<version>$([AzureEngSemanticVersion]::SEMVER_REGEX))(\s+(?<releaseStatus>\(.+\))))"
$SECTIONS_HEADER_REGEX = "^###+\s(?<sectionName>.*)"
$SECTION_HEADER_REGEX_SUFFIX = "##\s(?<sectionName>.*)"
$CHANGELOG_UNRELEASED_STATUS = "(Unreleased)"
$CHANGELOG_DATE_FORMAT = "yyyy-MM-dd"
$RecommendedSectionHeaders = @("Features Added", "Breaking Changes", "Bugs Fixed", "Other Changes")
Expand Down Expand Up @@ -49,6 +49,7 @@ function Get-ChangeLogEntriesFromContent {
$initialAtxHeader = $matches["HeaderLevel"]
}

$sectionHeaderRegex = "^${initialAtxHeader}${SECTION_HEADER_REGEX_SUFFIX}"
$changeLogEntries | Add-Member -NotePropertyName "InitialAtxHeader" -NotePropertyValue $initialAtxHeader
$releaseTitleAtxHeader = $initialAtxHeader + "#"

Expand All @@ -67,7 +68,7 @@ function Get-ChangeLogEntriesFromContent {
}
else {
if ($changeLogEntry) {
if ($line.Trim() -match $SECTIONS_HEADER_REGEX)
if ($line.Trim() -match $sectionHeaderRegex)
{
$sectionName = $matches["sectionName"].Trim()
$changeLogEntry.Sections[$sectionName] = @()
Expand Down Expand Up @@ -136,16 +137,24 @@ function Confirm-ChangeLogEntry {
[String]$ChangeLogLocation,
[Parameter(Mandatory = $true)]
[String]$VersionString,
[boolean]$ForRelease = $false
[boolean]$ForRelease = $false,
[Switch]$SantizeEntry
)

$changeLogEntry = Get-ChangeLogEntry -ChangeLogLocation $ChangeLogLocation -VersionString $VersionString
$changeLogEntries = Get-ChangeLogEntries -ChangeLogLocation $ChangeLogLocation
$changeLogEntry = $changeLogEntries[$VersionString]

if (!$changeLogEntry) {
LogError "ChangeLog[${ChangeLogLocation}] does not have an entry for version ${VersionString}."
return $false
}

if ($SantizeEntry)
{
Remove-EmptySections -ChangeLogEntry $changeLogEntry -InitialAtxHeader $changeLogEntries.InitialAtxHeader
Set-ChangeLogContent -ChangeLogLocation $ChangeLogLocation -ChangeLogEntries $changeLogEntries
}

Write-Host "Found the following change log entry for version '${VersionString}' in [${ChangeLogLocation}]."
Write-Host "-----"
Write-Host (ChangeLogEntryAsString $changeLogEntry)
Expand Down Expand Up @@ -308,24 +317,26 @@ function Set-ChangeLogContent {
function Remove-EmptySections {
param (
[Parameter(Mandatory = $true)]
$ChangeLogEntry
$ChangeLogEntry,
$InitialAtxHeader = "#"
)

$sectionHeaderRegex = "^${InitialAtxHeader}${SECTION_HEADER_REGEX_SUFFIX}"
$releaseContent = $ChangeLogEntry.ReleaseContent

if ($releaseContent.Count -gt 0)
{
$parsedSections = $ChangeLogEntry.Sections
$sanitizedReleaseContent = New-Object System.Collections.ArrayList(,$releaseContent)

foreach ($key in @($parsedSections.Key))
foreach ($key in @($parsedSections.Keys))
{
if ([System.String]::IsNullOrWhiteSpace($parsedSections[$key]))
{
for ($i = 0; $i -lt $sanitizedReleaseContent.Count; $i++)
{
$line = $sanitizedReleaseContent[$i]
if ($line -match $SECTIONS_HEADER_REGEX -and $matches["sectionName"].Trim() -eq $key)
if ($line -match $sectionHeaderRegex -and $matches["sectionName"].Trim() -eq $key)
{
$sanitizedReleaseContent.RemoveAt($i)
while($i -lt $sanitizedReleaseContent.Count -and [System.String]::IsNullOrWhiteSpace($sanitizedReleaseContent[$i]))
Expand All @@ -340,5 +351,4 @@ function Remove-EmptySections {
}
$ChangeLogEntry.ReleaseContent = $sanitizedReleaseContent.ToArray()
}
return $changeLogEntry
}
}
Loading

0 comments on commit 4ec3539

Please sign in to comment.