Bad PF candidate filter for 2018 PbPb data #31668

mandrenguyen · 2020-10-05T14:15:54Z

PR description:

Removes badly reconstructed PF charged hadrons and muons from list of PF candidates stored in the 2018 PbPb AOD. The filtered list is fed into the jet reconstruction (akCs4PFJets), as well as the packed PF candidate collection. The filtered out particles are stored in the following collection, which is stored in miniAOD
patPackedCandidates_packedPFCandidatesCleaned__PAT
The slimmedMuons use both the kept and discarded packed candidates, as input.
Discarded charged hadrons will end up in both packedPFCandidatesCleaned and lostTracks, but the cost of this double storage is small, as bad particles are rare.

Details of the studies used to tune the selection cuts for bad particles may be found here:
https://twiki.cern.ch/twiki/pub/CMS/HiHighPt2019/JetReco_MAN_291710.pdf
https://twiki.cern.ch/twiki/pub/CMS/HiJetReco2019/JetReco_MAN_191010.pdf

PR validation:

Tested with 140.5611

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2020-10-05T14:16:14Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-05T14:21:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31668/18790

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py modified in PR(s): HeavyIon Reaction Plane MiniAOD #31129, Include the HF filters used in heavy ions using struct in the MiniAOD content #31602
- File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): HeavyIon Reaction Plane MiniAOD #31129, Include the HF filters used in heavy ions using struct in the MiniAOD content #31602

cmsbuild · 2020-10-05T14:22:15Z

A new Pull Request was created by @mandrenguyen (Matthew Nguyen) for master.

It involves the following packages:

PhysicsTools/PatAlgos
RecoHI/HiJetAlgos

@perrotta, @jpata, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @yslai, @schoef, @emilbols, @peruzzim, @seemasharmafnal, @mmarionncern, @ahinzmann, @smoortga, @dgulhan, @jdolen, @ferencek, @yetkinyilmaz, @jdamgov, @nhanvtran, @gkasieczka, @hatakeyamak, @mariadalfonso, @clelange, @riga, @jazzitup, @JyothsnaKomaragiri, @yenjie, @kurtejung, @gpetruc, @andrzejnovak this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

perrotta · 2020-10-05T14:42:16Z

please test

cmsbuild · 2020-10-05T14:42:39Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-10-05-1200/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9732/console Started: 2020/10/05 16:43

RecoHI/HiJetAlgos/plugins/HiBadParticleFilter.cc

cmsbuild · 2020-10-05T14:49:47Z

-1

Tested at: 03d87db

CMSSW: CMSSW_11_2_X_2020-10-05-1200
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e7d33/9732/summary.html

I found follow errors while testing this PR

Failed tests: ClangBuild

Clang:

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

cmsbuild · 2020-10-05T14:49:48Z

Comparison not run due to Build errors/Fireworks only changes/No short matrix requested (RelVals and Igprof tests were also skipped)

RecoHI/HiJetAlgos/plugins/HiBadParticleFilter.cc

cmsbuild · 2020-10-20T20:12:56Z

+1
Tested at: 307021a
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e7d33/10142/summary.html
CMSSW: CMSSW_11_2_X_2020-10-19-2300
SCRAM_ARCH: slc7_amd64_gcc820

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

Removed unused fwkJobReports from MessageLogger in Geometry Subsystem #31858 @Dr15Jones: Removed unused fwkJobReports from MessageLogger in Geometry Subsystem
Added LogFwk* logging levels to be used by the framework #31815 @Dr15Jones: Added LogFwk* logging levels to be used by the framework
Sub-event aware genJet and parton matching for HI miniAOD #31698 @mandrenguyen: Sub-event aware genJet and parton matching for HI miniAOD
Add anti-electron tauID for phase2 #31470 @cms-tau-pog: Add anti-electron tauID for phase2
Add maximum of pileup pT hats in JME custom NanoAOD #31831 @nurfikri89: Add maximum of pileup pT hats in JME custom NanoAOD
Removed unused fwkJobReports from MessageLogger in SLHCUpgradeSimulations Subsystem #31861 @Dr15Jones: Removed unused fwkJobReports from MessageLogger in SLHCUpgradeSimulations Subsystem
Removed unused fwkJobReports from MessageLogger in IOMC Subsystem #31860 @Dr15Jones: Removed unused fwkJobReports from MessageLogger in IOMC Subsystem

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e7d33/10142/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e7d33/10142/git-merge-result

cmsbuild · 2020-10-20T20:12:59Z

Comparison job queued.

cmsbuild · 2020-10-20T21:27:01Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e7d33/10142/summary.html

The workflows 158.01 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 27 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2415615
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2415592
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 142 log files, 21 edm output root files, 35 DQM output files

perrotta · 2020-10-21T09:18:23Z

Configuration/PyReleaseValidation/python/relval_steps.py

                                   },hiDefaults2018_ppReco,step3Up2015Defaults])
 steps['REMINIAODHI2018PPRECOMB']=merge([{'-s':'PAT,VALIDATION:@miniAODValidation,DQM:@miniAODDQM',
                                         '--datatier':'MINIAODSIM,DQMIO',
                                         '--eventcontent':'MINIAODSIM,DQM',
                                         '-n':100,
                                         '--era':'Run2_2018_pp_on_AA',
-                                         '--procModifiers':'genJetSubEvent',
+                                         '--procModifiers':'genJetSubEvent,run2_miniAOD_pp_on_AA_103X',


@mandrenguyen , please check: is this additional procModifier ("run2_miniAOD_pp_on_AA_103X") really intended for REMINIAODHI2018PPRECOMB?
(I noticed it because, if I am not wrong, this showed up after the rebase, but it was not there before)

Yes, this procModifier is the one that enables the cleaning of bad PF candidates (the point of this PR). It should be enabled for the (re)miniAOD workflows, but not for the overall pp_on_AA eras. The "MB" version of the reMiniAOD relval workflow was introduced in #31698, the PR that conflicted with this one, and which caused a rebase to be needed. I'm not 100% sure of the sequence of events that led to this being introduced in the rebase, but I confirm that it is the intended behavior.

perrotta · 2020-10-21T09:38:06Z

+1

Rebased coherently with the original intentions of this PR.
Since no relevant changes appeared since then for reco, I repeat here what written in the first reco signature:
- The filter on badly reconstructed PF charged hadrons acts on re-miniaod of Run2, not on other eras, where the selection is expected to happen at the source
- Jenkins tests pass and show no differences for "regular" workflows (PS: differences that popped up here for the Phase2 workflows are originated by the additional PRs tested together with this one by bot)
- Large stat comparisons show events with filtered out hadrons, and how it propagates on the reconstructed jets and so
- Event size slightly reduced overall: packedPFCandidates shrink of the same size as the now saved separately packedPFCandidatesRemoved; on the other hand, given the smaller number of Candidates available for PFJets, the size of slimmedJets also tinily gets reduced

kpedro88 · 2020-10-21T14:45:48Z

+upgrade

silviodonato · 2020-10-21T18:53:00Z

merge

silviodonato · 2020-10-22T06:45:30Z

@mandrenguyen this PR seems to generate some problems. See #31894

santocch · 2020-10-28T13:49:11Z

+1

cmsbuild added this to the CMSSW_11_2_X milestone Oct 5, 2020

cmsbuild added analysis-pending code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 5, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 5, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 5, 2020

slava77 reviewed Oct 5, 2020

View reviewed changes

RecoHI/HiJetAlgos/plugins/HiBadParticleFilter.cc Outdated Show resolved Hide resolved

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Oct 5, 2020

perrotta reviewed Oct 5, 2020

View reviewed changes

RecoHI/HiJetAlgos/plugins/HiBadParticleFilter.cc Outdated Show resolved Hide resolved

cmsbuild added code-checks-pending comparison-pending and removed code-checks-approved comparison-notrun tests-rejected labels Oct 5, 2020

cmsbuild added tests-approved and removed tests-started labels Oct 20, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 20, 2020

perrotta reviewed Oct 21, 2020

View reviewed changes

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 21, 2020

cmsbuild added upgrade-approved and removed upgrade-pending labels Oct 21, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 21, 2020

cmsbuild merged commit 11c67e5 into cms-sw:master Oct 21, 2020

silviodonato mentioned this pull request Oct 22, 2020

Exception in egmPhotonPUPPIIsolation in several HI workflows #31894

Closed

silviodonato mentioned this pull request Oct 22, 2020

Segmentation violation in workflows 33034.0 and 33434.0 (geometry D72) #31895

Closed

This was referenced Oct 22, 2020

Add PbPb modifications to egamma sequences to Run 3 era #31897

Merged

Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

Merged

cmsbuild added analysis-approved and removed analysis-pending labels Oct 28, 2020

This was referenced Nov 29, 2020

Use cleaned PF candidates in UE subtraction in HI miniAOD #32325

Merged

Use cleaned PF candidates in UE subtraction in HI miniAOD (112X) #32330

Merged

This was referenced Mar 17, 2022

Fix Egamma PF linking in 2018 PbPb mAOD #37261

Closed

Associated PF for Egamma in 2018 PbPb miniAOD #37262

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bad PF candidate filter for 2018 PbPb data #31668

Bad PF candidate filter for 2018 PbPb data #31668

mandrenguyen commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

perrotta commented Oct 5, 2020

cmsbuild commented Oct 5, 2020 •

edited

Loading

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

perrotta Oct 21, 2020 •

edited

Loading

mandrenguyen Oct 21, 2020 •

edited

Loading

perrotta commented Oct 21, 2020

kpedro88 commented Oct 21, 2020

silviodonato commented Oct 21, 2020

silviodonato commented Oct 22, 2020

santocch commented Oct 28, 2020

Bad PF candidate filter for 2018 PbPb data #31668

Bad PF candidate filter for 2018 PbPb data #31668

Conversation

mandrenguyen commented Oct 5, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

perrotta commented Oct 5, 2020

cmsbuild commented Oct 5, 2020 • edited Loading

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 5, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

perrotta Oct 21, 2020 • edited Loading

Choose a reason for hiding this comment

mandrenguyen Oct 21, 2020 • edited Loading

Choose a reason for hiding this comment

perrotta commented Oct 21, 2020

kpedro88 commented Oct 21, 2020

silviodonato commented Oct 21, 2020

silviodonato commented Oct 22, 2020

santocch commented Oct 28, 2020

cmsbuild commented Oct 5, 2020 •

edited

Loading

perrotta Oct 21, 2020 •

edited

Loading

mandrenguyen Oct 21, 2020 •

edited

Loading