Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

makortel · 2023-05-05T20:52:43Z

PR description:

This PR makes every Alpaka EDProducer to store an unsigned short event product to denote the backend of the producer. This is intended as a temporary measure until we get a more general machinery (#30044).

Resolves cms-sw/framework-team#512

PR validation:

Touched unit tests succeed.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

To be backported to 13_0_X and 13_1_X.

cmsbuild · 2023-05-05T21:02:43Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41564/35437

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/AlpakaInterface/test/BuildFile.xml modified in PR(s): Pixel Alpaka Migration: AlpakaInterface Updates [II] #41284
- File HeterogeneousCore/AlpakaTest/plugins/TestAlpakaAnalyzer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Fixes for newer Eigen: Use Eigen::placeholders::all #40645

cmsbuild · 2023-05-05T21:03:08Z

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

HeterogeneousCore/AlpakaCore (heterogeneous)
HeterogeneousCore/AlpakaInterface (heterogeneous)
HeterogeneousCore/AlpakaTest (heterogeneous)

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@missirol, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

makortel · 2023-05-05T21:08:24Z

enable gpu

makortel · 2023-05-05T21:08:31Z

@cmsbuild, please test

makortel · 2023-05-05T21:09:43Z

@fwyzard This PR is based on top of the branch of #41019. Is this still the most recent one or do you have something newer?

cmsbuild · 2023-05-05T23:58:57Z

-1

Failed Tests: HeaderConsistency
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7e988f/32428/summary.html
COMMIT: 1c79672
CMSSW: CMSSW_13_2_X_2023-05-05-1100/el8_amd64_gcc11
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41564/32428/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 7 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3460836
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3460802
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

There are some workflows for which there are errors in the baseline:
11634.506 step 2
11634.512 step 2
11634.522 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 40073
DQMHistoTests: Total failures: 38
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 40035
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 6 edm output root files, 3 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-05-08T14:27:48Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41564/35469

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/AlpakaInterface/test/BuildFile.xml modified in PR(s): Pixel Alpaka Migration: AlpakaInterface Updates [II] #41284
- File HeterogeneousCore/AlpakaTest/plugins/TestAlpakaAnalyzer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Fixes for newer Eigen: Use Eigen::placeholders::all #40645

makortel · 2023-05-08T14:27:58Z

Just to note the header consistency check complained about headers not touched in this PR

makortel · 2023-05-08T14:28:06Z

@cmsbuild, please test

makortel · 2023-10-18T07:51:45Z

@cmsbuild please test

cmsbuild · 2023-10-18T13:52:16Z

-1

Failed Tests: RelVals-GPU
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7e988f/35247/summary.html
COMMIT: a8b3c73
CMSSW: CMSSW_13_3_X_2023-10-17-2300/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41564/35247/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-GPU

12434.58712434.587_TTbar_14TeV+2023_Patatrack_AllTripletsGPU_Validation/step2_TTbar_14TeV+2023_Patatrack_AllTripletsGPU_Validation.log

Comparison Summary

Summary:

You potentially added 4 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3357400
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3357369
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

makortel · 2023-10-19T05:28:26Z

The RelVal failure seems to be related to #42997

makortel · 2023-10-19T05:30:33Z

+heterogeneous

cmsbuild · 2023-10-19T05:31:02Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

fwyzard · 2023-10-22T08:41:32Z

please test

cmsbuild · 2023-10-22T11:09:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7e988f/35335/summary.html
COMMIT: a8b3c73
CMSSW: CMSSW_13_3_X_2023-10-22-0000/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41564/35335/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3357400
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3357372
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 21
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 39719
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

rappoccio · 2023-10-23T14:04:29Z

+1

cmsbuild added this to the CMSSW_13_2_X milestone May 5, 2023

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending labels May 5, 2023

makortel mentioned this pull request May 5, 2023

Store Alpaka EDProducer backend in an event data product cms-sw/framework-team#512

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels May 5, 2023

cmsbuild added tests-started and removed tests-pending labels May 5, 2023

cmsbuild added tests-rejected tests-pending code-checks-pending and removed tests-started tests-rejected code-checks-approved labels May 5, 2023

cmsbuild added tests-started code-checks-approved and removed tests-pending code-checks-pending labels May 8, 2023

cmsbuild added tests-started and removed tests-pending labels Oct 18, 2023

cmsbuild added tests-rejected and removed tests-started labels Oct 18, 2023

makortel changed the title ~~Make every Alpaka EDProducer to store a short int for the backend~~ Make every Alpaka EDProducer to store an unsigned short int for the backend Oct 19, 2023

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Oct 19, 2023

This was referenced Oct 19, 2023

Ports prefixScan, OneToManyAssoc and HistoContainer from CUDAUtilities. #43064

Merged

Porting Pixel Tracks to Alpaka [Not to Merge] #41117

Closed

Implement std::array-like wrapper #43076

Merged

cmsbuild added tests-started and removed tests-rejected labels Oct 22, 2023

cmsbuild added tests-approved and removed tests-started labels Oct 22, 2023

cmsbuild added orp-approved and removed orp-pending labels Oct 23, 2023

cmsbuild merged commit bca672b into cms-sw:master Oct 23, 2023
14 checks passed

makortel deleted the alpakaBackendProduct branch October 23, 2023 14:27

This was referenced Oct 23, 2023

Updated root to tip of branch master cms-sw/cmsdist#8777

Merged

Cleanup CICADA/CaloSummary emulation and make CICADA versions available as templated plugins #43091

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

makortel commented May 5, 2023 •

edited

Loading

cmsbuild commented May 5, 2023

cmsbuild commented May 5, 2023

makortel commented May 5, 2023

makortel commented May 5, 2023

makortel commented May 5, 2023

cmsbuild commented May 5, 2023

cmsbuild commented May 8, 2023

makortel commented May 8, 2023

makortel commented May 8, 2023

makortel commented Oct 18, 2023

cmsbuild commented Oct 18, 2023

makortel commented Oct 19, 2023

makortel commented Oct 19, 2023

cmsbuild commented Oct 19, 2023

fwyzard commented Oct 22, 2023

cmsbuild commented Oct 22, 2023

rappoccio commented Oct 23, 2023

Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

Conversation

makortel commented May 5, 2023 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented May 5, 2023

cmsbuild commented May 5, 2023

makortel commented May 5, 2023

makortel commented May 5, 2023

makortel commented May 5, 2023

cmsbuild commented May 5, 2023

Comparison Summary

GPU Comparison Summary

cmsbuild commented May 8, 2023

makortel commented May 8, 2023

makortel commented May 8, 2023

makortel commented Oct 18, 2023

cmsbuild commented Oct 18, 2023

RelVals-GPU

Comparison Summary

makortel commented Oct 19, 2023

makortel commented Oct 19, 2023

cmsbuild commented Oct 19, 2023

fwyzard commented Oct 22, 2023

cmsbuild commented Oct 22, 2023

Comparison Summary

GPU Comparison Summary

rappoccio commented Oct 23, 2023

makortel commented May 5, 2023 •

edited

Loading