HeavyIon Reaction Plane MiniAOD #31129

BetterWang · 2020-08-12T19:06:15Z

PR description:

EventPlane/ReactionPlane code update for the MiniAOD format. Also clean up the histograms in DQM.

PR validation:

This PR is tested with runTheMatrix workflow 140.5611 along with the new GT 110X_upgrade2018_realistic_HI_Candidate_2020_08_07_19_30_54.

@mandrenguyen @ssanders50

cmsbuild · 2020-08-12T19:06:41Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-12T19:14:31Z

-code-checks

ERROR: Build errors found during clang-tidy run.

gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

BetterWang · 2020-08-12T19:41:14Z

scram build code-checks works fine for me on lxplus7 CMSSW_11_2_X_2020-08-12-1100.

slava77 · 2020-08-12T20:04:44Z

code-checks

cmsbuild · 2020-08-12T20:05:06Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-12T20:12:20Z

-code-checks

ERROR: Build errors found during clang-tidy run.

gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

slava77 · 2020-08-12T20:15:15Z

the code-checks issue looks repeatable

the error comes after scram build -k -j 16 code-checks

scram build code-checks works fine for me on lxplus7 CMSSW_11_2_X_2020-08-12-1100.

did you run this test from scratch or just from your development area?
I can guess some incomplete commit

slava77 · 2020-08-12T20:20:01Z

the error comes after scram build -k -j 16 code-checks

here is a link to the console output in jenkins for the last code-checks
https://cmssdt.cern.ch/jenkins/job/run-pr-code-checks/17728/console

BetterWang · 2020-08-12T20:49:13Z

the error comes after scram build -k -j 16 code-checks

here is a link to the console output in jenkins for the last code-checks
https://cmssdt.cern.ch/jenkins/job/run-pr-code-checks/17728/console

I have clean working directory.
Do you seen anything wrong with this output?

[qwang@lxplus7118 src]$ scram build -k -j 16 code-checks

Local Products Rules ..... started
Local Products Rules ..... done
gmake[1]: Entering directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
Local Products Rules ..... started
Local Products Rules ..... done
gmake[2]: Entering directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
Local Products Rules ..... started
Local Products Rules ..... done
------- copying files from src/PhysicsTools/PatAlgos/scripts -------
copied patReplaceFast.pl
copied patReplaceGenerator.sh
Entering Package DQM/Physics
copied patReplaceParser.pl
Entering Package PhysicsTools/PatAlgos
Entering Package RecoHI/HiEvtPlaneAlgos
Compile sequence completed for CMSSW CMSSW_11_2_X_2020-08-12-1100
gmake[2]: Leaving directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
gmake[2]: Entering directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
Local Products Rules ..... started
Local Products Rules ..... done
clang-tidy -export-fixes tmp/slc7_amd64_gcc820/code-checks/DQM/Physics/src/CentralityDQM.cc.yaml -header-filter '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100/src/.' src/DQM/Physics/src/CentralityDQM.cc
clang-tidy -export-fixes tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/src/EvtPlaneProducer.cc.yaml -header-filter '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100/src/.' src/RecoHI/HiEvtPlaneAlgos/src/EvtPlaneProducer.cc
clang-tidy -export-fixes tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/src/HiEvtPlaneFlatProducer.cc.yaml -header-filter '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100/src/.' src/RecoHI/HiEvtPlaneAlgos/src/HiEvtPlaneFlatProducer.cc
1916 warnings generated.
Suppressed 1916 warnings (1915 in non-user code, 1 with check filters).
Use -header-filter=. to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
1954 warnings generated.
Suppressed 1954 warnings (1953 in non-user code, 1 with check filters).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
2703 warnings generated.
Suppressed 2703 warnings (2702 in non-user code, 1 with check filters).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
gmake[2]: Leaving directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
gmake[2]: Entering directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
Local Products Rules ..... started
Local Products Rules ..... done
Changed files: /src/DQM/Physics/src/CentralityDQM.h
/src/DQM/Physics/src/CentralitypADQM.h
/src/DQM/Physics/src/CentralityDQM.cc
/src/PhysicsTools/PatAlgos/python/slimming/slimming_cff.py
/src/PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py
/src/RecoHI/HiEvtPlaneAlgos/interface/LoadEPDB.h
/src/RecoHI/HiEvtPlaneAlgos/interface/HiEvtPlaneFlatten.h
/src/RecoHI/HiEvtPlaneAlgos/interface/HiEvtPlaneList.h
/src/RecoHI/HiEvtPlaneAlgos/interface/EPCuts.h
/src/RecoHI/HiEvtPlaneAlgos/python/HiEvtPlane_cfi.py
/src/RecoHI/HiEvtPlaneAlgos/python/hiEvtPlaneFlat_cfi.py
/src/RecoHI/HiEvtPlaneAlgos/python/RecoHiEvtPlane_EventContent_cff.py
/src/RecoHI/HiEvtPlaneAlgos/BuildFile.xml
/src/RecoHI/HiEvtPlaneAlgos/src/EvtPlaneProducer.cc
/src/RecoHI/HiEvtPlaneAlgos/src/HiEvtPlaneFlatProducer.cc
/src/CondCore/ESSources/plugins/CondDBESSource.h
/src/CondTools/DQM/python/DQMXMLFileEventSetupAnalyzer_cfi.py
/src/CondTools/DQM/src/DQMXMLFileSourceHandler.cc
/src/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py
/src/Configuration/PyReleaseValidation/python/MatrixInjector.py
/src/L1Trigger/L1TTrackMatch/plugins/L1TkMuonProducer.cc
Working on tmp/slc7_amd64_gcc820/code-checks/DQM/Physics/src/CentralityDQM.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/DQM/Physics/src/CentralitypADQM.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/DQM/Physics/src/CentralityDQM.cc.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/PhysicsTools/PatAlgos/python/slimming/slimming_cff.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/interface/LoadEPDB.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/interface/HiEvtPlaneFlatten.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/interface/HiEvtPlaneList.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/interface/EPCuts.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/python/HiEvtPlane_cfi.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/python/hiEvtPlaneFlat_cfi.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/python/RecoHiEvtPlane_EventContent_cff.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/BuildFile.xml.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/src/EvtPlaneProducer.cc.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/RecoHI/HiEvtPlaneAlgos/src/HiEvtPlaneFlatProducer.cc.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/CondCore/ESSources/plugins/CondDBESSource.h.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/CondTools/DQM/python/DQMXMLFileEventSetupAnalyzer_cfi.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/CondTools/DQM/src/DQMXMLFileSourceHandler.cc.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/Configuration/PyReleaseValidation/python/MatrixInjector.py.yaml
Working on tmp/slc7_amd64_gcc820/code-checks/L1Trigger/L1TTrackMatch/plugins/L1TkMuonProducer.cc.yaml
clang-apply-replacements tmp/slc7_amd64_gcc820/code-checks
gmake[2]: Leaving directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'
gmake[1]: Leaving directory '/afs/cern.ch/work/q/qwang/cleanroomRun7/CMSSW_11_2_X_2020-08-12-1100'

slava77 · 2020-08-12T22:28:24Z

I see that the PR is made against CMSSW_11_2_X_2020-08-12-2300 (base 72bdd0a), it may be easier to wait until a new IB shows up.

In the meantime, I suggest to cleanup the topic branch history so that (as much as possible) only the final files in this PR actually show up.
Specifically, to avoid things like 0ec60c9 with many files (that were added first) be removed.

BetterWang · 2020-08-13T16:50:40Z

Hi @slava77 , should we try again with code-checks, or should I start a clean branch to make another PR?

slava77 · 2020-08-13T16:51:13Z

code-checks

cmsbuild · 2020-08-13T16:51:34Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-22T00:14:45Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-51f178/10189/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2544092
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2544057
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -57.111 KiB( 34 files compared)
DQMHistoSizes: changed ( 140.53,... ): -19.037 KiB Physics/Centrality
Checked 149 log files, 22 edm output root files, 35 DQM output files

slava77 · 2020-10-22T17:43:22Z

+1

for #31129 cce5752

code changes are in line with the PR description and the follow up review: EvtPlaneProducer is adapted to read miniAOD and pp-like inputs
jenkins tests pass and comparisons with the baseline show expected changes
- hiEvtPlane{,Flat} collections show up in the miniAOD workflow
- small changes are present in the heavyIons scenario workflow 140.53 for 2011 data (which is apparently a result of updates to the track selections)
- DQM plots for workflow 158.0 (which includes PAT and by virtue of it hiEvtPlane) are now filled for Centrality/EventPlane monitoring, after the producer was added to the miniAOD/PAT step
- manual check of wf 158.01 shows sensible (non-default) values in all event plane variables saved, as expected

santocch · 2020-10-23T09:03:08Z

+1

slava77 · 2020-10-26T14:20:41Z

@cms-sw/dqm-l2
a DQM signature is needed for this PR.
Please check and comment or sign.
Thank you.

jfernan2 · 2020-10-26T14:34:01Z

+1

cmsbuild · 2020-10-26T14:34:26Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2020-10-26T14:45:05Z

+1

Martin-Grunewald · 2020-10-27T13:44:44Z

This seems to crate a problem in a HIon workflow doing --step=RAW2DIGI,L1Reco,RECO,EI,PAT,VALIDATION,DQM
(TSG validation tests):

----- Begin Fatal Exception 27-Oct-2020 13:54:11 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 12 event: 3310 stream: 3
   [1] Running path 'MINIAODSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'MINIAODSIMoutput'
   [3] Calling method for module HiEvtPlaneFlatProducer/'hiEvtPlaneFlat'
Exception Message:
No "HeavyIonRPRcd" record found in the EventSetup.n
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

More info:
GT used: phase1_2021_realistic_hi
Era used: Run3_pp_on_PbPb

Also affecting relvals 140.5611 159.0 310.0

mandrenguyen · 2020-10-27T13:54:37Z

@christopheralanwest Regarding the issue spotted by @Martin-Grunewald above, would you please update the GT corresponding to phase1_2021_realistic_hi in the same way as done for phase1_2018_realistic_hi?
That change was introduced here:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/112X_upgrade2018_realistic_HI_v3/112X_upgrade2018_realistic_HI_v4
Thanks!

christopheralanwest · 2020-10-27T13:59:42Z

Should the 2021 scenario use the same tag as that used in auto:phase1_2018_realistic_hi or is another tag more appropriate? I'm asking because a different tag HeavyIonRPRcd_EvtPlane_MC_2018_v01_offline is queued in 112X_upgrade2018_realistic_HI_Queue.

mandrenguyen · 2020-10-27T14:01:39Z

Ah, @BetterWang will have to answer that.

BetterWang · 2020-10-27T14:40:43Z

Hi, I believe we should use the auto tag. HeavyIonRPRcd_EvtPlane_MC_2018_v01_offline was replace because it is a multi-IOV tag.

cmsbuild added this to the CMSSW_11_2_X milestone Aug 12, 2020

cmsbuild added analysis-pending code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Aug 12, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Aug 12, 2020

cmsbuild added code-checks-pending and removed code-checks-rejected labels Aug 12, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Aug 12, 2020

cmsbuild added code-checks-pending and removed code-checks-rejected labels Aug 13, 2020

cmsbuild added tests-approved and removed tests-started labels Oct 21, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 22, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 22, 2020

cmsbuild added analysis-approved and removed analysis-pending labels Oct 23, 2020

cmsbuild mentioned this pull request Oct 25, 2020

Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

Merged

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Oct 26, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 26, 2020

cmsbuild merged commit 2f1749f into cms-sw:master Oct 26, 2020

christopheralanwest mentioned this pull request Oct 27, 2020

Add record type HeavyIonRPRcd to 2021 HI global tag #31957

Merged

cmsbuild mentioned this pull request Oct 27, 2020

[Do not merge] Just to test cms-bot #31945

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HeavyIon Reaction Plane MiniAOD #31129

HeavyIon Reaction Plane MiniAOD #31129

BetterWang commented Aug 12, 2020 •

edited

Loading

cmsbuild commented Aug 12, 2020

cmsbuild commented Aug 12, 2020

BetterWang commented Aug 12, 2020

slava77 commented Aug 12, 2020

cmsbuild commented Aug 12, 2020

cmsbuild commented Aug 12, 2020

slava77 commented Aug 12, 2020

slava77 commented Aug 12, 2020

BetterWang commented Aug 12, 2020

slava77 commented Aug 12, 2020

BetterWang commented Aug 13, 2020

slava77 commented Aug 13, 2020

cmsbuild commented Aug 13, 2020

cmsbuild commented Oct 22, 2020

slava77 commented Oct 22, 2020

santocch commented Oct 23, 2020

slava77 commented Oct 26, 2020

jfernan2 commented Oct 26, 2020

cmsbuild commented Oct 26, 2020

qliphy commented Oct 26, 2020

Martin-Grunewald commented Oct 27, 2020 •

edited

Loading

mandrenguyen commented Oct 27, 2020

christopheralanwest commented Oct 27, 2020

mandrenguyen commented Oct 27, 2020

BetterWang commented Oct 27, 2020

HeavyIon Reaction Plane MiniAOD #31129

HeavyIon Reaction Plane MiniAOD #31129

Conversation

BetterWang commented Aug 12, 2020 • edited Loading

PR description:

PR validation:

cmsbuild commented Aug 12, 2020

cmsbuild commented Aug 12, 2020

BetterWang commented Aug 12, 2020

slava77 commented Aug 12, 2020

cmsbuild commented Aug 12, 2020

cmsbuild commented Aug 12, 2020

slava77 commented Aug 12, 2020

slava77 commented Aug 12, 2020

BetterWang commented Aug 12, 2020

slava77 commented Aug 12, 2020

BetterWang commented Aug 13, 2020

slava77 commented Aug 13, 2020

cmsbuild commented Aug 13, 2020

cmsbuild commented Oct 22, 2020

slava77 commented Oct 22, 2020

santocch commented Oct 23, 2020

slava77 commented Oct 26, 2020

jfernan2 commented Oct 26, 2020

cmsbuild commented Oct 26, 2020

qliphy commented Oct 26, 2020

Martin-Grunewald commented Oct 27, 2020 • edited Loading

mandrenguyen commented Oct 27, 2020

christopheralanwest commented Oct 27, 2020

mandrenguyen commented Oct 27, 2020

BetterWang commented Oct 27, 2020

BetterWang commented Aug 12, 2020 •

edited

Loading

Martin-Grunewald commented Oct 27, 2020 •

edited

Loading