DeepTau v2p5 in nanoAOD #38726

mbluj · 2022-07-13T14:59:39Z

PR description:

This PR adds working point definitions for a newly integrated DeepTau v2p5.
A corresponding study with details about WP threshold derivation and tau efficiency/mis-ID rate plots for both Run2 UL and Run 3 samples can be found here.

In particular, the changes in this PR are:

WP thresholds values for DeepTau v2p5 are added to runTauIdMVA.py and thus to miniAOD;
For nanoAOD, add a dedicated set of variables _deepTauVars2018v2p5 to taus_cff.py,
Unify WP masking interface to a single function _tauIdWPMask();
Option to compute WP flags from raw scores (from_raw argument in _tauIdWPMask()) given the threshold values, instead of reading them directly from MINIAOD;
Change the format of storing WPs from bitmask to integer values for user-friendliness [see note 1].

Backport to 12_4_X foreseen.

[Note 1]: Change of the format of storing WPs affects also WPs of already present tauIDs and thus is not backward compatible. For instance if old versions of nano (with old eras) were produced with this PR integrated they would have format which is not backward compatible with the same versions without this PR. We think, however, that there is not realistic use-case, i.e. old versions of nano are not produced with new releases. Keeping backward compatibility is possible, but implementation will be painful. Comments from experts are welcome.

PR validation:

Successfully tested with the "limited" set of matrix tests and a custom nanoAOD production.

cmsbuild · 2022-07-13T15:08:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38726/31027

This PR adds an extra 104KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nanoDQM_cfi.py modified in PR(s): Adding HZZ electron MVA ID #38355

cmsbuild · 2022-07-13T15:09:16Z

A new Pull Request was created by @mbluj for master.

It involves the following packages:

PhysicsTools/NanoAOD (xpog)
RecoTauTag/RecoTau (reconstruction)

@gouskos, @clacaputo, @cmsbuild, @fgolf, @jpata, @mariadalfonso can you please review it and eventually sign? Thanks.
@mbluj, @gpetruc, @azotz, @swertz this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2022-07-13T15:26:18Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38726/31029

This PR adds an extra 108KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nanoDQM_cfi.py modified in PR(s): Adding HZZ electron MVA ID #38355

cmsbuild · 2022-07-13T15:26:38Z

Pull request #38726 was updated. @gouskos, @clacaputo, @cmsbuild, @fgolf, @jpata, @mariadalfonso can you please check and sign again.

gouskos · 2022-07-13T15:38:12Z

please test

cmsbuild · 2022-07-13T20:11:44Z

-1

Failed Tests: UnitTests RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f206e7/26211/summary.html
COMMIT: d09f58d
CMSSW: CMSSW_12_5_X_2022-07-13-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38726/26211/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals-INPUT

The relvals timed out after 4 hours.

1325.61325.6_TTbar_13_94Xv1NanoAODINPUT+TTbar_13_94Xv1NanoAODINPUT+NANOAODMC2017_94XMiniAODv1/step2_TTbar_13_94Xv1NanoAODINPUT+TTbar_13_94Xv1NanoAODINPUT+NANOAODMC2017_94XMiniAODv1.log
1325.81325.8_TTbar_13_94Xv1NanoAODINPUT+TTbar_13_94Xv1NanoAODINPUT+NANOEDMMC2017_94XMiniAODv1+HARVESTNANOAODMC2017_94XMiniAODv1/step2_TTbar_13_94Xv1NanoAODINPUT+TTbar_13_94Xv1NanoAODINPUT+NANOEDMMC2017_94XMiniAODv1+HARVESTNANOAODMC2017_94XMiniAODv1.log

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 197 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3653966
DQMHistoTests: Total failures: 235
DQMHistoTests: Total nulls: 76
DQMHistoTests: Total successes: 3653633
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -30.842000000000002 KiB( 49 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -3.398 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 1325.81 ): -7.797 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 136.8523 ): -2.661 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

mbluj · 2022-07-14T08:45:21Z

Investigating problem with the failed unit tests.

mbluj · 2022-07-14T09:36:23Z

Unit tests should be fixed now. Also 1325.6 and 1325.8 run at lxplus are successfully passed.

mbluj · 2022-08-02T10:10:53Z

In d0122b1 suffixes are added to tauID modules created for MiniAOD and for NanoAOD. This avoids that modules from one workflow are overriding ones from another workflow when both workflows are run together as in 136.72412.
This commit does not solve possible issue that deepTauId is run twice, however I assume that it will not happen in a regular production, e.g. Run-2 UL miniAOD will not be produced together with NanoAOD v10 using CMSSW_12_4_X (or newer).

cmsbuild · 2022-08-02T10:11:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38726/31381

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/boostedTaus_cff.py modified in PR(s): [124X backport] Add DY option to Embedding HepMC filter #38937
- File PhysicsTools/NanoAOD/python/nanoDQM_cff.py modified in PR(s): [124X backport] Add DY option to Embedding HepMC filter #38937
- File PhysicsTools/NanoAOD/python/nanoDQM_cfi.py modified in PR(s): [124X backport] Add DY option to Embedding HepMC filter #38937
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167, [124X backport] Add DY option to Embedding HepMC filter #38937, fix dump of nano + add modifier for 122 mini #38749
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): [124X backport] Add DY option to Embedding HepMC filter #38937
- File RecoTauTag/RecoTau/python/tools/runTauIdMVA.py modified in PR(s): [124X backport] Add DY option to Embedding HepMC filter #38937

cmsbuild · 2022-08-02T10:11:44Z

Pull request #38726 was updated. @gouskos, @swertz, @vlimant, @clacaputo, @cmsbuild, @jpata, @mariadalfonso can you please check and sign again.

mariadalfonso · 2022-08-02T10:16:06Z

please test

cmsbuild · 2022-08-02T14:12:38Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f206e7/26594/summary.html
COMMIT: d0122b1
CMSSW: CMSSW_12_5_X_2022-08-02-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38726/26594/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 221 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3677948
DQMHistoTests: Total failures: 213
DQMHistoTests: Total nulls: 71
DQMHistoTests: Total successes: 3677642
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -30.846000000000004 KiB( 50 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -3.398 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 1325.81 ): -7.797 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 136.8523 ): -2.661 KiB Physics/NanoAODDQM
Checked 210 log files, 47 edm output root files, 51 DQM output files
TriggerResults: no differences found

mariadalfonso · 2022-08-03T14:39:36Z

verified the following:

change of the mask to int in the storage
add the new ID in miniRun3
ID changed in the nano v2.1 to v2.5
runs on old mini (winter 12_2 and run2)

--> 3) and 4) verified mode directly with the backport PR

mariadalfonso · 2022-08-03T14:42:57Z

+xpog

clacaputo · 2022-08-04T13:23:45Z

+reconstruction

reco differences are present only in tau.tauIDs and are in line with the content of this PR

cmsbuild · 2022-08-04T13:24:13Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-08-05T00:57:57Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Jul 13, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending xpog-pending labels Jul 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 13, 2022

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 13, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 13, 2022

cmsbuild added tests-rejected and removed tests-started labels Jul 13, 2022

mariadalfonso mentioned this pull request Jul 14, 2022

new features for V10 nanoAOD cms-nanoAOD/cmssw#580

Closed

39 tasks

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Jul 14, 2022

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Aug 2, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 2, 2022

cmsbuild added tests-started and removed tests-pending labels Aug 2, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 2, 2022

cmsbuild added xpog-approved and removed xpog-pending labels Aug 3, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Aug 4, 2022

cmsbuild added orp-approved and removed orp-pending labels Aug 5, 2022

cmsbuild merged commit 92e4e3f into cms-sw:master Aug 5, 2022

mbluj deleted the CMSSW_12_5_X_tau-pog_deepTau-v2p5-WPs-nano branch October 10, 2023 10:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DeepTau v2p5 in nanoAOD #38726

DeepTau v2p5 in nanoAOD #38726

mbluj commented Jul 13, 2022 •

edited

Loading

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

gouskos commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

mbluj commented Jul 14, 2022

mbluj commented Jul 14, 2022

mbluj commented Aug 2, 2022

cmsbuild commented Aug 2, 2022

cmsbuild commented Aug 2, 2022

mariadalfonso commented Aug 2, 2022

cmsbuild commented Aug 2, 2022

mariadalfonso commented Aug 3, 2022 •

edited

Loading

mariadalfonso commented Aug 3, 2022

clacaputo commented Aug 4, 2022

cmsbuild commented Aug 4, 2022

qliphy commented Aug 5, 2022

DeepTau v2p5 in nanoAOD #38726

DeepTau v2p5 in nanoAOD #38726

Conversation

mbluj commented Jul 13, 2022 • edited Loading

PR description:

PR validation:

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

gouskos commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

Unit Tests

RelVals-INPUT

Comparison Summary

mbluj commented Jul 14, 2022

mbluj commented Jul 14, 2022

mbluj commented Aug 2, 2022

cmsbuild commented Aug 2, 2022

cmsbuild commented Aug 2, 2022

mariadalfonso commented Aug 2, 2022

cmsbuild commented Aug 2, 2022

Comparison Summary

mariadalfonso commented Aug 3, 2022 • edited Loading

mariadalfonso commented Aug 3, 2022

clacaputo commented Aug 4, 2022

cmsbuild commented Aug 4, 2022

qliphy commented Aug 5, 2022

mbluj commented Jul 13, 2022 •

edited

Loading

mariadalfonso commented Aug 3, 2022 •

edited

Loading