Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IndexMap imbalance "diagnostic" function #2653

Merged
merged 10 commits into from
Oct 3, 2023
Merged

Add IndexMap imbalance "diagnostic" function #2653

merged 10 commits into from
Oct 3, 2023

Conversation

IgorBaratta
Copy link
Member

This PR adds a new imbalance() function to the IndexMap class.
The imbalance is defined as the maximum number of indices on any process divided by the average number of indices per process, and is computed separately for owned and ghost indices.

Usage example:

import ufl
import dolfinx
from dolfinx import fem
import numpy as np
from dolfinx.io import XDMFFile
from mpi4py import MPI

comm = MPI.COMM_WORLD

# Create a mesh and function space.
mesh = dolfinx.mesh.create_unit_cube(comm, 10, 10, 10,
                                     cell_type=dolfinx.mesh.CellType.tetrahedron,
                                     ghost_mode=dolfinx.mesh.GhostMode.none)
V = fem.FunctionSpace(mesh, ("Lagrange", 5))
index_map = V.dofmap.index_map
tdim = mesh.topology.dim

dof_imbalance = index_map.imbalance
cell_imbalance = mesh.topology.index_map(tdim).imbalance

if (comm.rank == 0):
    print("dof imbalance", dof_imbalance)
    print("cell imbalance", cell_imbalance)
dof imbalance [1.0140896035461473, 1.3187465232709068]
cell imbalance [1.006, -1.0]

@chrisrichardson
Copy link
Contributor

So, perfect balance = 1? And it can never be less than 1. I am not sure about the -1, maybe this should also just be 1 - because zero everywhere is also perfectly balanced?

And then, why not just subtract the 1? In the example above, we are mostly interested that one process has 31.8% more ghost dofs than average... 0.318 would be the "imbalance", 0.0 perfect balance...

/// separately for owned indices and ghost indices and returns them as a
/// std::array<double, 2>. If the total number of owned or ghost indices is
/// zero, the respective entry in the array is set to -1.
///
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be useful to say that "1.0" represents a "perfect" balance.

@jhale
Copy link
Member

jhale commented May 12, 2023

This seems useful, and is non-trivial to implement. Thanks!

@garth-wells garth-wells added the enhancement New feature or request label Jul 17, 2023
@jhale jhale added this pull request to the merge queue Oct 3, 2023
Merged via the queue into main with commit a88a52e Oct 3, 2023
20 checks passed
@jhale jhale deleted the igor/imbalance branch October 3, 2023 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants