Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] MueLu: Refactor CoalesceDropFactory_kokkos #12861

Draft
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

cgcgcg
Copy link
Contributor

@cgcgcg cgcgcg commented Mar 25, 2024

@trilinos/muelu

Motivation

We eventually want to merge CoalesceDropFactory and CoalesceDropFactory_kokkos. Some of the dropping schemes (e.g. cut-drop) in CoalesceDropFactory need access to the entire row. Moreover, the two factories are so messy because we want to apply a whole variety of different criteria.

I refactored the code to enable more composability than before and break up the algorithms into smaller, more readable units. Criteria take the form of functors that are applied to each row. This means that we need at most 3 kernel launches, instead of launching kernels for different criteria one-by-one.
One large change is that I allocate a view for the dropping decisions. On the other hand, we no longer allocate and then shrink a copy of the original matrix.

Questions/todos:

  • Are there conceptual issues with this?
  • Currently, this requires to build with Kokkos_ENABLE_CUDA_CONSTEXPR=ON to allow the use of std::get. Is there a way around that?
  • I have not yet performance tested it.
  • The tests pass on my workstation.

@cgcgcg cgcgcg added pkg: MueLu MueLuReview To request that the MueLu Team nitpick this PR labels Mar 25, 2024
@cgcgcg cgcgcg self-assigned this Mar 25, 2024
@cgcgcg cgcgcg changed the title MueLu: Refactor CoalesceDropFactory_kokkos [DO NOT MERGE] MueLu: Refactor CoalesceDropFactory_kokkos Mar 25, 2024
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3862
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any&&!sandybridge
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2361
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2351
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2350
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3517
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3353
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1992
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_cuda-11.4.20-uvm

  • Build Num: 97
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Using Repos:

Repo: TRILINOS (cgcgcg/Trilinos)
  • Branch: coalesceDrop
  • SHA: 5c1d34a
  • Mode: TEST_REPO

Pull Request Author: cgcgcg

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3862
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any&&!sandybridge
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2361
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2351
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2350
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3517
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3353
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1992
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a

Build Information

Test Name: Trilinos_PR_cuda-11.4.20-uvm

  • Build Num: 97
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: MueLu;MueLuReview
PULLREQUESTNUM 12861
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA 5c1d34a
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 216c43a


CDash Test Results for PR# 12861.


Wiki: How to Reproduce PR Testing Builds and Errors.

@cgcgcg cgcgcg added the AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) label Mar 26, 2024
@cgcgcg cgcgcg force-pushed the coalesceDrop branch 2 times, most recently from bc183ae to 5f90623 Compare March 26, 2024 16:52
auto aiiajj = ATS::magnitude(diag(rlid)) * ATS::magnitude(diag(clid)); // |a_ii|*|a_jj|
auto aij2 = ATS::magnitude(val) * ATS::magnitude(val); // |a_ij|^2

results(offset + k) = (aij2 <= eps * eps * aiiajj) ? DROP : KEEP;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be useful to note somewhere this is classical (SA) dropping, not classical (RS) dropping

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand. What's the difference?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GrahamBenHarper means that classical could refer to "Ruge Stueben" or to the standard smoothed aggregation dropping criterion.

@cgcgcg cgcgcg force-pushed the coalesceDrop branch 3 times, most recently from 9a28985 to 1362ac3 Compare July 24, 2024 23:36
Signed-off-by: Christian Glusa <caglusa@sandia.gov>
Signed-off-by: Christian Glusa <caglusa@sandia.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) MueLuReview To request that the MueLu Team nitpick this PR pkg: MueLu
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants