Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DBMON-3271] add new obfuscator options to customize SQL obfuscation and normalization #16429

Merged
merged 3 commits into from
Dec 19, 2023

Conversation

lu-zhengda
Copy link
Contributor

@lu-zhengda lu-zhengda commented Dec 14, 2023

What does this PR do?

This PR adds new SQL obfuscation options to postgres, mysql and sqlserver integration. The new options includes

  • obfuscation_mode: enables/disables SQL obfuscation with new obfuscator from go-sqllexer
  • remove_space_between_parentheses: remove spaces between parentheses in your normalized SQL statements.
  • keep_null: keep the keyword NULL in your obfuscated SQL statements.
  • keep_boolean: keep the keywords TRUE and FALSE in your obfuscated SQL statements.
  • keep_positional_parameter: keep positional parameters (e.g. $1) in your obfuscated SQL statements.
  • keep_trailing_semicolon: keep trailing semicolons in your normalized SQL statements.
  • keep_identifier_quotation: keep identifier quotations (e.g. "my_table") in your normalized SQL statements.

The new options are depend on DataDog/datadog-agent#21568

Motivation

Additional Notes

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Changelog entries must be created for modifications to shipped code
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.

@lu-zhengda lu-zhengda changed the title add new obfuscator options to customize SQL obfuscation and normaliza… [DBMON-3271] add new obfuscator options to customize SQL obfuscation and normalization Dec 14, 2023
Copy link

codecov bot commented Dec 14, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (64c2fee) 90.53% compared to head (b308150) 90.63%.
Report is 26 commits behind head on master.

Additional details and impacted files
Flag Coverage Δ
activemq ?
cassandra ?
confluent_platform ?
hive ?
hivemq ?
hudi ?
ignite ?
jboss_wildfly ?
kafka ?
mysql 87.33% <ø> (-0.04%) ⬇️
postgres 92.27% <ø> (+0.01%) ⬆️
presto ?
solr ?
sqlserver 89.83% <ø> (ø)
tomcat ?
weblogic ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Copy link

github-actions bot commented Dec 14, 2023

Test Results

     48 files       48 suites   1h 20m 56s ⏱️
   676 tests    662 ✔️   14 💤 0
5 990 runs  5 648 ✔️ 342 💤 0

Results for commit b308150.

♻️ This comment has been updated with latest results.

@lu-zhengda lu-zhengda merged commit 4272b9a into master Dec 19, 2023
45 checks passed
@lu-zhengda lu-zhengda deleted the zhengda.lu/obfuscation-options branch December 19, 2023 19:22
github-actions bot pushed a commit to wolf31o2/integrations-core that referenced this pull request Dec 19, 2023
…and normalization (DataDog#16429)

* add new obfuscator options to customize SQL obfuscation and normalization

* add changelogs

* fix lint 4272b9a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants