Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[14_0_X] Update to relval_gpu.pyfor Alpaka and Pixel Triplets Alpaka Workflows #45959

Open
wants to merge 2 commits into
base: CMSSW_14_0_X
Choose a base branch
from
+160 −67
Open

[14_0_X] Update to relval_gpu.pyfor Alpaka and Pixel Triplets Alpaka Workflows #45959

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b485a98
Move relval_gpu to 2024 Alpaka + add D110 wfs (140X)
Aug 13, 2024
ac5b488
Use D98 wfs + Fix Alpaka Harvesting for Phase2
AdrianoDee Sep 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Configuration/PyReleaseValidation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,15 @@ The offsets currently in use are:
* 0.402: Alpaka, pixel only quadruplets, portable
* 0.403: Alpaka, pixel only quadruplets, portable vs. CPU validation
* 0.404: Alpaka, pixel only quadruplets, portable profiling
* 0.406: Alpaka, pixel only triplets, portable
* 0.407: Alpaka, pixel only triplets, portable vs. CPU validation
* 0.407: Alpaka, pixel only triplets, portable profiling
* 0.412: Alpaka, ECAL only, portable
* 0.422: Alpaka, HCAL only, portable
* 0.423: Alpaka, HCAL only, portable vs CPU validation
* 0.424: Alpaka, HCAL only, portable profiling
* 0.492: Alpaka, full reco with pixel quadruplets
* 0.496: Alpaka, full reco with pixel triplets
* 0.5: Pixel tracking only + 0.1
* 0.501: Patatrack, pixel only quadruplets, on CPU
* 0.502: Patatrack, pixel only quadruplets, with automatic offload to GPU if available
Expand Down
106 changes: 52 additions & 54 deletions Configuration/PyReleaseValidation/python/relval_gpu.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,70 +13,68 @@

# mc WFs to run in IB:

# mc 2023 Alpaka pixel-only quadruplets: TTbar: any backend, any backend vs cpu validation, profiling
# mc 2023 #FIXME to be removed as soon as cms-bot is updated
# no PU Alpaka pixel-only TTbar: quadruplets any backend and profiling; ECAL-only any backend; HCAL-only any backend and profiling
# mc 2024
# no PU
# Alpaka pixel-only quadruplets: TTbar: any backend, any backend vs cpu validation, profiling
# Alpaka pixel-only triplets: TTbar: any backend, any backend vs cpu validation, profiling
# Alpaka ECAL-only: TTbar: any backend
# Alpaka HCAL-only: TTbar: any backend, any backend vs cpu validation, profiling
# Alpaka with full reco and pixel-only quadruplets: TTbar: any backend
# Alpaka with full reco and pixel-only: TTbar: any backend quadruplets, any backend triplets
# Alpaka pixel-only quadruplets: ZMM: any backend, any backend vs cpu validation, profiling
# Alpaka pixel-only triplets: ZMM: any backend, any backend vs cpu validation, profiling
# Alpaka pixel-only quadruplets: Single Nu E10: any backend
# Patatrack pixel-only quadruplets: ZMM - on GPU (optional), GPU-vs-CPU validation, profiling
# Patatrack pixel-only triplets: ZMM - on GPU (optional), GPU-vs-CPU validation, profiling
# Patatrack pixel-only quadruplets: TTbar - on GPU (optional), GPU-vs-CPU validation, profiling
# Patatrack pixel-only triplets: TTbar - on GPU (optional), GPU-vs-CPU validation, profiling
# Patatrack ECAL-only: TTbar - on GPU (optional), GPU-vs-CPU validation, profiling
# Patatrack HCAL-only: TTbar - on GPU (optional), GPU-vs-CPU validation, profiling
# Patatrack pixel-only quadruplets, ECAL, HCAL: TTbar - on GPU (optional), GPU-vs-CPU validation
# Patatrack pixel-only triplets, ECAL, HCAL: TTbar - on GPU (optional), GPU-vs-CPU validation
# full reco with Patatrack pixel-only quadruplets: TTbar - on GPU (optional), GPU-vs-CPU validation
# full reco with Patatrack pixel-only triplets: TTbar - on GPU (optional), GPU-vs-CPU validation
# Patatrack pixel-only quadruplets: Single Nu E10 on GPU (optional)
# with PU
# Alpaka pixel-only quadruplets: TTbar with PU: any backend, any backend vs cpu validation, profiling
# Alpaka pixel-only triplets: TTbar with PU: any backend, any backend vs cpu validation, profiling
# Alpaka ECAL-only: TTbar with PU: any backend
# Alpaka HCAL-only: TTbar with PU: any backend, any backend vs cpu validation, profiling
# Alpaka with full reco and pixel-only quadruplets: TTbar with PU: any backend
# Alpaka with full reco and pixel-only: TTbar with PU: any backend quadruplets, any backend triplets
# Alpaka pixel-only quadruplets: ZMM with PU: any backend, any backend vs cpu validation, profiling
# Alpaka pixel-only triplets: ZMM with PU: any backend, any backend vs cpu validation, profiling
# Alpaka pixel-only quadruplets: Single Nu E10 with PU: any backend
# mc 2026 Patatrack pixel-only quadruplets: Single Nu E10: on GPU (optional)
# mc 2026
# no PU
# Alpaka pixel-only: TTbar: quadruplets any backend, any backend vs cpu validation, profiling, triplets
# Alpaka pixel-only: Single Nu E10: any backend
# with PU
# Alpaka pixel-only: TTbar with PU: quadruplets any backend, any backend vs cpu validation, profiling

numWFIB = [
# 2023, Alpaka-based
12434.402, 12434.403, 12434.404,
#12434.406, 12434.407, 12434.408,
12434.412,#12434.413, 12434.414,
12434.422, 12434.423, 12434.424,
#12434.482, 12434.483, 12434.484
#12434.486, 12434.487, 12434.488
12434.492,#12434.493
12450.402, 12450.403, 12450.404,
#12450.406, 12450.407, 12450.408,
12461.402,

# 2023, CUDA-based
12450.502, 12450.503, 12450.504,
12450.506, 12450.507, 12450.508,
12434.502, 12434.503, 12434.504,
12434.506, 12434.507, 12434.508,
12434.512, 12434.513, 12434.514,
12434.522, 12434.523, 12434.524,
12434.582, 12434.583,#12434.584,
12434.586, 12434.587,#12434.588,
12434.592, 12434.593,
12434.596, 12434.597,
12461.502,

# 2023 with PU, Alpaka-based
12634.402, 12634.403, 12634.404,
#12634.406, 12634.407, 12634.408
12634.412,#12634.413, 12634.414
12634.422, 12634.423, 12634.424,
#12634.482, 12634.483, 12634.484
#12634.486, 12634.487, 12634.488
12634.492,#12634.493
12650.402, 12650.403, 12650.404,
#12650.406, 12650.407, 12650.408
12661.402,

# 2026, CUDA-based
24861.502
# 2023, Alpaka-based noPU
12434.402,12434.403,12434.412,12434.422,12434.423,

# 2024, Alpaka-based noPU
12834.402, 12834.403, 12834.404,
12834.406, 12834.407, 12834.408,
12834.412,#12834.413, 12834.414,
12834.422, 12834.423, 12834.424,
#12834.482, 12834.483, 12834.484
#12834.486, 12834.487, 12834.488
12834.492, 12834.493,
12850.402, 12850.403, 12850.404,
12450.406, 12450.407, 12450.408,
12861.402,

# 2024 with PU, Alpaka-based
13034.402, 13034.403, 13034.404,
13034.406, 13034.407, 13034.408,
13034.412, #13034.413, 13034.414
13034.422, 13034.423, 13034.424,
#13034.482, 13034.483, 13034.484
#13034.486, 13034.487, 13034.488
13034.492, 13034.493,
13050.402, 13050.403, 13050.404,
13050.406, 13050.407, 13050.408,
13061.402,

# 2026, Alpaka-based noPU
24834.402, 24834.403, 24834.404, 24834.406,
24861.402,

# 2026, Alpaka-based PU
25034.402, 25034.403, 25034.404
]

for numWF in numWFIB:
Expand Down
90 changes: 80 additions & 10 deletions Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1047,7 +1047,7 @@ def setup_(self, step, stepName, stepDict, k, properties):
'--customise': 'RecoTracker/Configuration/customizePixelTracksForTriplets.customizePixelTracksForTriplets,RecoTracker/Configuration/customizePixelOnlyForProfiling.customizePixelOnlyForProfilingGPUOnly'
},
harvest = None,
suffix = 'Patatrack_PixelOnlyTripletsGPU_Profiling',
suffix = 'Patatrack_PixelOnlyTripletsGPU_Profiling',
offset = 0.508,
)

Expand All @@ -1058,7 +1058,7 @@ def setup_(self, step, stepName, stepDict, k, properties):
upgradeWFs['PatatrackECALOnlyAlpaka'] = PatatrackWorkflow(
digi = {
# customize the ECAL Local Reco part of the HLT menu for Alpaka
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
Expand Down Expand Up @@ -1234,7 +1234,7 @@ def setup_(self, step, stepName, stepDict, k, properties):
# - HCAL-only reconstruction using Alpaka with DQM and Validation
upgradeWFs['PatatrackHCALOnlyAlpakaValidation'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
Expand All @@ -1254,7 +1254,7 @@ def setup_(self, step, stepName, stepDict, k, properties):
# - HCAL-only reconstruction using GPU and Alpaka with DQM and Validation for PF Alpaka vs CPU comparisons
upgradeWFs['PatatrackHCALOnlyGPUandAlpakaValidation'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
Expand All @@ -1274,7 +1274,7 @@ def setup_(self, step, stepName, stepDict, k, properties):
# - HCAL-only reconstruction using Alpaka
upgradeWFs['PatatrackHCALOnlyAlpakaProfiling'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
},
reco = {
'-s': 'RAW2DIGI:RawToDigi_hcalOnly,RECO:reconstruction_hcalOnly',
Expand All @@ -1291,13 +1291,13 @@ def setup_(self, step, stepName, stepDict, k, properties):
# - harvesting
upgradeWFs['PatatrackFullRecoAlpaka'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
# skip the @pixelTrackingOnlyValidation which cannot run together with the full reconstruction
'-s': 'RAW2DIGI:RawToDigi+RawToDigi_pixelOnly,L1Reco,RECO:reconstruction+reconstruction_pixelTrackingOnly,RECOSIM,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM+@pixelTrackingOnlyDQM',
'--procModifiers': 'alpaka,pixelNtupletFit',
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
harvest = {
Expand All @@ -1307,6 +1307,28 @@ def setup_(self, step, stepName, stepDict, k, properties):
offset = 0.492,
)

# Workflow running the Pixel triplets, ECAL and HCAL reconstruction on GPU (optional), PF using Alpaka, together with the full offline reconstruction on CPU
# - HLT on GPU (optional)
# - reconstruction on Alpaka, with DQM and validation
# - harvesting
upgradeWFs['PatatrackFullRecoAlpaka'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
# skip the @pixelTrackingOnlyValidation which cannot run together with the full reconstruction
'-s': 'RAW2DIGI:RawToDigi+RawToDigi_pixelOnly,L1Reco,RECO:reconstruction+reconstruction_pixelTrackingOnly,RECOSIM,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM+@pixelTrackingOnlyDQM',
'--procModifiers': 'alpaka',
'--customise' : 'RecoTracker/Configuration/customizePixelTracksForTriplets.customizePixelTracksForTriplets,HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
harvest = {
# skip the @pixelTrackingOnlyDQM harvesting
},
suffix = 'Patatrack_FullRecoAlpakaTriplets',
offset = 0.496,
)

# Workflow running the Pixel quadruplets, ECAL and HCAL reconstruction on CPU
# - HLT on CPU
# - reconstruction on CPU, with DQM and validation
Expand Down Expand Up @@ -1613,7 +1635,7 @@ def setup_(self, step, stepName, stepDict, k, properties):

upgradeWFs['PatatrackPixelOnlyAlpaka'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
Expand All @@ -1630,7 +1652,7 @@ def setup_(self, step, stepName, stepDict, k, properties):

upgradeWFs['PatatrackPixelOnlyAlpakaValidation'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
Expand All @@ -1648,7 +1670,7 @@ def setup_(self, step, stepName, stepDict, k, properties):

upgradeWFs['PatatrackPixelOnlyAlpakaProfiling'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka', # alpaka modifier activates customiseHLTForAlpaka
'--procModifiers': 'alpaka',
},
reco = {
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly',
Expand All @@ -1660,6 +1682,54 @@ def setup_(self, step, stepName, stepDict, k, properties):
offset = 0.404,
)

upgradeWFs['PatatrackPixelOnlyTripletsAlpaka'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
'--procModifiers': 'alpaka',
'--customise' : 'RecoTracker/Configuration/customizePixelTracksForTriplets.customizePixelTracksForTriplets,HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling'
},
harvest = {
'-s': 'HARVESTING:@trackingOnlyValidation+@pixelTrackingOnlyDQM'
},
suffix = 'Patatrack_PixelOnlyTripletsAlpaka',
offset = 0.406,
)

upgradeWFs['PatatrackPixelOnlyTripletsAlpakaValidation'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka',
'--customise' : 'HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling',
},
reco = {
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly,VALIDATION:@pixelTrackingOnlyValidation,DQM:@pixelTrackingOnlyDQM',
'--procModifiers': 'alpakaValidation',
'--customise' : 'RecoTracker/Configuration/customizePixelTracksForTriplets.customizePixelTracksForTriplets,HeterogeneousCore/AlpakaServices/customiseAlpakaServiceMemoryFilling.customiseAlpakaServiceMemoryFilling'
},
harvest = {
'-s': 'HARVESTING:@trackingOnlyValidation+@pixelTrackingOnlyDQM'
},
suffix = 'Patatrack_PixelOnlyTripletsAlpaka_Validation',
offset = 0.407,
)

upgradeWFs['PatatrackPixelOnlyTripletsAlpakaProfiling'] = PatatrackWorkflow(
digi = {
'--procModifiers': 'alpaka',
},
reco = {
'-s': 'RAW2DIGI:RawToDigi_pixelOnly,RECO:reconstruction_pixelTrackingOnly',
'--procModifiers': 'alpaka',
'--customise' : 'RecoTracker/Configuration/customizePixelTracksForTriplets.customizePixelTracksForTriplets,RecoTracker/Configuration/customizePixelOnlyForProfiling.customizePixelOnlyForProfilingGPUOnly'
},
harvest = None,
suffix = 'Patatrack_PixelOnlyTripletsAlpaka_Profiling',
offset = 0.408,
)

# end of Patatrack workflows

class UpgradeWorkflow_ProdLike(UpgradeWorkflow):
Expand Down
6 changes: 6 additions & 0 deletions DQM/SiPixelHeterogeneous/python/SiPixelHeterogenousDQMHarvesting_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@
siPixelPhase1RawDataHarvesterDevice *
siPixelTrackComparisonHarvesterAlpaka )

from Configuration.Eras.Modifier_phase2_tracker_cff import phase2_tracker
_siPixelHeterogeneousDQMComparisonHarvestingAlpakaPhase2 = cms.Sequence(siPixelTrackComparisonHarvesterAlpaka )

phase2_tracker.toReplaceWith(siPixelHeterogeneousDQMComparisonHarvestingAlpaka,_siPixelHeterogeneousDQMComparisonHarvestingAlpakaPhase2)


# add the harvester in case of the validation modifier is active
from Configuration.ProcessModifiers.gpuValidationPixel_cff import gpuValidationPixel
gpuValidationPixel.toReplaceWith(siPixelHeterogeneousDQMHarvesting,siPixelHeterogeneousDQMComparisonHarvesting)
Expand Down
19 changes: 17 additions & 2 deletions DQM/SiPixelHeterogeneous/python/SiPixelHeterogenousDQM_FirstStep_cff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@
siPixelVertexSoAMonitorDevice *
siPixelCompareVertices )

# Phase-2 sequence
# Phase-2 sequence ...
_monitorpixelSoACompareSource = cms.Sequence(siPixelPhase2MonitorRecHitsSoACPU *
siPixelPhase2MonitorRecHitsSoAGPU *
siPixelPhase2CompareRecHitsSoA *
Expand All @@ -250,6 +250,18 @@
siPixelMonitorVertexSoAGPU *
siPixelCompareVertexSoA)

# ...and the Alpaka version
_monitorpixelSoACompareSourceAlpakaPhase2 = cms.Sequence(
siPixelRecHitsSoAMonitorSerial *
siPixelRecHitsSoAMonitorDevice *
siPixelPhase1CompareRecHits *
siPixelTrackSoAMonitorSerial *
siPixelTrackSoAMonitorDevice *
siPixelPhase1CompareTracks *
siPixelVertexSoAMonitorSerial *
siPixelVertexSoAMonitorDevice *
siPixelCompareVertices )

# HIon sequence
_monitorpixelSoACompareSourceHIonPhase1 = cms.Sequence(siPixelHIonPhase1MonitorRecHitsSoACPU *
siPixelHIonPhase1MonitorRecHitsSoAGPU *
Expand All @@ -262,9 +274,12 @@
siPixelCompareVertexSoA)

phase2_tracker.toReplaceWith(monitorpixelSoACompareSource,_monitorpixelSoACompareSource)
phase2_tracker.toReplaceWith(monitorpixelSoACompareSourceAlpaka,_monitorpixelSoACompareSourceAlpakaPhase2)

from Configuration.ProcessModifiers.gpuValidationPixel_cff import gpuValidationPixel
gpuValidationPixel.toReplaceWith(monitorpixelSoASource, monitorpixelSoACompareSource)

from Configuration.ProcessModifiers.alpakaValidationPixel_cff import alpakaValidationPixel
(alpakaValidationPixel & ~gpuValidationPixel).toReplaceWith(monitorpixelSoASource, monitorpixelSoACompareSourceAlpaka)
(alpakaValidationPixel & ~gpuValidationPixel ).toReplaceWith(monitorpixelSoASource, monitorpixelSoACompareSourceAlpaka)


Loading