Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrub batch shared keys #17030

Merged
merged 4 commits into from
Mar 3, 2021
Merged

Conversation

tasherif-msft
Copy link
Contributor

@tasherif-msft tasherif-msft commented Mar 2, 2021

Resolves #17054. Currently shared keys are leaking in batch operations since we have no way of replacing these values. This PR adds a new method to detect a batch operation and filter out the storage account name and shared keys from its payload using a passed regex expression.

@ghost ghost added the Storage Storage Service (Queues, Blobs, Files) label Mar 2, 2021
@tasherif-msft tasherif-msft changed the title Scrub batch Scrub batch shared keys Mar 2, 2021
@tasherif-msft tasherif-msft marked this pull request as draft March 2, 2021 22:07
@tasherif-msft tasherif-msft marked this pull request as ready for review March 3, 2021 01:25
if request.body and request.uri and is_batch_payload(request):
import re
body = six.ensure_str(request.body)
matched_objects = set(re.findall(old, body))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use set to ensure that we don't iterate over the already replaced substring (no duplicates)

Copy link
Member

@seankane-msft seankane-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :shipit:

@tasherif-msft tasherif-msft merged commit 8c726ac into Azure:master Mar 3, 2021
tasherif-msft added a commit that referenced this pull request Mar 3, 2021
iscai-msft added a commit to iscai-msft/azure-sdk-for-python that referenced this pull request Mar 3, 2021
…into http_request_json

* 'master' of https://github.com/Azure/azure-sdk-for-python: (147 commits)
  [text analytics] add perf tests (Azure#17060)
  Add cloud event to core (Azure#16800)
  [Perf] Small fixes to storage-blob (Azure#17055)
  [EG] Regenerate Code (Azure#17053)
  Scrub batch shared keys (Azure#17030)
  [Tables] Add SAS to tables (Azure#16717)
  T2 containerservice 2021 03 03 (Azure#17050)
  Addressing issues with CredScan (Azure#16944)
  Communication chat preview4 (Azure#16905) (Azure#17037)
  remove first query section (Azure#17033)
  [formrecognizer] temp disable sample tests until service bug fixed (Azure#17036)
  [device update] allow device update pylint failures (Azure#17034)
  fix build (Azure#17029)
  update artifact names for ALL packages to align with the actual package name
  Create azure-iot-nspkg (Azure#17026)
  [Communication]: SMS 1:N Messages, Custom Tags, and Idempotence (Azure#16836)
  Fixing credentials to use AAD (Azure#16885)
  T2 deviceupdate 2021 03 02 (Azure#17016)
  T2 cosmosdb 2021 02 23 (Azure#16875)
  T2 datadog 2021 03 02 (Azure#17004)
  ...
iscai-msft added a commit that referenced this pull request Mar 3, 2021
…into add_sample_check

* 'master' of https://github.com/Azure/azure-sdk-for-python: (388 commits)
  [text analytics] add normalized_text (#17074)
  Renaming with_token identity function (#17066)
  Adapt to azure core's cloud event (#17063)
  align perf tests with js (#17069)
  [Perfstress][Storage] Added FileShare perf tests (#15834)
  [formrecognizer] Adding custom forms perf test (#16969)
  Fix LanguageShort typo (#17068)
  sas creds updates (#17065)
  [eventgrid] Fix Sample eh (#17064)
  [Perfstress][Storage] Added Datalake perf tests (#15861)
  [text analytics] Healthcare n-ary relations (#16997)
  ServiceBus dict-representation acceptance and kwarg-update functionality  (#14807)
  [text analytics] add perf tests (#17060)
  Add cloud event to core (#16800)
  [Perf] Small fixes to storage-blob (#17055)
  [EG] Regenerate Code (#17053)
  Scrub batch shared keys (#17030)
  [Tables] Add SAS to tables (#16717)
  T2 containerservice 2021 03 03 (#17050)
  Addressing issues with CredScan (#16944)
  ...
openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-python that referenced this pull request Dec 13, 2021
CosmosDB: Fix resource IDs in SqlContainer examples (Azure#17030)
openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-python that referenced this pull request Jan 5, 2022
CosmosDB: Fix resource IDs in SqlContainer examples (Azure#17030)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Scrub batch operation shared keys from yaml recording
3 participants