Implement std::array-like wrapper

[fwyzard]

PR description:

Due to a ROOT limitation an std::array cannot be serialised to a ROOT file.
See root-project/root#12007 for a discussion on the issue.

This class reimplements the full std::array<T,N> interface, using a regular Reflex dictionary for the ROOT serialisation.
To be more GPU-friendly, all methods are constexpr, and out-of-bound data access aborts instead of throwing an exception.

Note: dictionaries for edm::StdArray<T,N> where T is a standard C/C++ type should be declared in DataFormats/Common/src/classed_def.xml.

PR validation:

The new unit tests pass.

The updated tests using an StdArray in a SoA collection work on CPU and GPU, and the collection can be serialised to a ROOT filed and read back.

[fwyzard]

enable gpu

[fwyzard]

please test

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43076/37303

• This PR adds an extra 32KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DataFormats/Common/test/BuildFile.xml modified in PR(s): [RFC] Replace RangeMap used in Muon hit storage #42917
  • File DataFormats/PortableTestObjects/interface/TestSoA.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
  • File HeterogeneousCore/AlpakaTest/plugins/TestAlpakaAnalyzer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Make every Alpaka EDProducer to store an unsigned short int for the backend #41564
  • File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlgo.dev.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Porting Pixel Tracks to Alpaka [Not to Merge]  #41117

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43076/37303/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43076/37303/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[fwyzard]

enable gpu

[fwyzard]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43076/37304

• This PR adds an extra 32KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DataFormats/Common/test/BuildFile.xml modified in PR(s): [RFC] Replace RangeMap used in Muon hit storage #42917
  • File DataFormats/PortableTestObjects/interface/TestSoA.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
  • File HeterogeneousCore/AlpakaTest/plugins/TestAlpakaAnalyzer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Make every Alpaka EDProducer to store an unsigned short int for the backend #41564
  • File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlgo.dev.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Porting Pixel Tracks to Alpaka [Not to Merge]  #41117

[cmsbuild]

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

• DataFormats/Common (core)
• DataFormats/PortableTestObjects (heterogeneous)
• HeterogeneousCore/AlpakaTest (heterogeneous)

@smuzaffar, @Dr15Jones, @fwyzard, @makortel can you please review it and eventually sign? Thanks.
@rovere, @wddgit, @makortel, @missirol this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Failed Tests: Build HeaderConsistency
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f6a1a4/35320/summary.html
COMMIT: e01d6f0
CMSSW: CMSSW_13_3_X_2023-10-20-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43076/35320/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/HeterogeneousCore/CUDATest/plugins/TestCUDAAnalyzerGPU.cc
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/HeterogeneousCore/CUDATest/plugins/TestCUDAAnalyzerGPUKernel.cu
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/HeterogeneousCore/CUDATest/plugins/TestCUDAProducerCPU.cc
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/HeterogeneousCore/CUDATest/plugins/TestCUDAProducerGPU.cc
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/HeterogeneousCore/CUDATest/plugins/TestAlgo.cc: In function 'void cudatest::testAlgoKernel(cms::cuda::PortableHostCollection >::View, int32_t)':
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/HeterogeneousCore/CUDATest/plugins/TestAlgo.cc:13:43: error: no match for 'operator=' (operand types are 'portabletest::TestSoALayout<>::ViewTemplateFreeParams<128, false, true, false>::element' and '')
   13 |       view[i] = {0., 0., 0., i, matrix * i};
      |                                           ^
In file included from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02807/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-10-20-1100/src/DataFormats/SoATemplate/interface/SoALayout.h:14,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/DataFormats/PortableTestObjects/interface/TestSoA.h:11,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_3_X_2023-10-20-1100/src/CUDADataFormats/PortableTestObjects/interface/TestHostCollection.h:5,


[makortel]

Other related CMSSW issues

• SoA layout with std::array type column crashes PoolOutputModule #40982
• Support for std::array event products #30493

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43076/37347

• This PR adds an extra 28KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DataFormats/Common/test/BuildFile.xml modified in PR(s): [RFC] Replace RangeMap used in Muon hit storage #42917
  • File DataFormats/PortableTestObjects/interface/TestSoA.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
  • File HeterogeneousCore/AlpakaTest/plugins/TestAlpakaAnalyzer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285
  • File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlgo.dev.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285, Porting Pixel Tracks to Alpaka [Not to Merge]  #41117

[cmsbuild]

Pull request #43076 was updated. @Dr15Jones, @makortel, @fwyzard, @smuzaffar can you please check and sign again.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f6a1a4/35396/summary.html
COMMIT: a5be164
CMSSW: CMSSW_13_3_X_2023-10-24-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43076/35396/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 10 lines to the logs
• Reco comparison results: 11 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3357400
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3357372
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 214 log files, 167 edm output root files, 50 DQM output files
• TriggerResults: no differences found

GPU Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 3
• DQMHistoTests: Total histograms compared: 39740
• DQMHistoTests: Total failures: 20
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 39720
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
• Checked 8 log files, 10 edm output root files, 3 DQM output files
• TriggerResults: no differences found

[fwyzard]

+heterogeneous

[makortel]

+core

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

[antoniovilela]

+1