Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors and add mypy to CI #2190

Merged
merged 68 commits into from
May 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
9fbe4ed
Remove adjoint, unused and untested.
jhale May 20, 2022
17f5d63
Remove this, cannot work with missing RTLD_NOW in modern ctypes.
jhale May 20, 2022
f7f9d33
Work in progress for mypy pass (MetaClass stuff is tricky).
jhale May 20, 2022
0a849cd
Add mypy to ubuntu image
jhale May 20, 2022
bd8a508
Again
jhale May 20, 2022
d695e3e
Merge branch 'main' into jhale/mypy-fixes-and-ci
jhale May 20, 2022
c417f26
Move later
jhale May 20, 2022
2aa08c1
Merge branch 'jhale/mypy-fixes-and-ci' of github.com:FEniCS/dolfinx i…
jhale May 20, 2022
3494584
Bit more typing, remove adjoint import
jhale May 20, 2022
068c3e8
Fix but more typing errors back
jhale May 20, 2022
764faae
Adding if makes type check pass
jhale May 20, 2022
5d2c439
35 errors left in python/dolfinx
jhale May 20, 2022
129fda7
Remaining issues are due to use of single dispatch
jhale May 20, 2022
edff60b
Add py.typed files so that users actually see type annotations
jhale May 20, 2022
a3b5a86
Generalised arguments types for mesh constructors
jhale May 20, 2022
1d50a2e
Fix singledispatch definition to raise NotImplementedError if no meth…
jhale May 21, 2022
31d55be
Fixes all typing issues in assemble.
jhale May 21, 2022
6a20000
9 left in demos, 22 in main library (mainly in petsc.py)
jhale May 21, 2022
5a02e4b
flake8 fixes
jhale May 21, 2022
cdebd1a
Merge branch 'main' into jhale/mypy-fixes-and-ci
jhale May 21, 2022
2b1e356
Demos pass, petsc.py and two strange ones to go
jhale May 22, 2022
ff21edf
Liberal mypy passes, all tests pass
jhale May 22, 2022
5f49d1a
Fix fallback vector
jhale May 22, 2022
04dd71e
Fixes
jhale May 22, 2022
1d9c836
mypy settings in cfg file
jhale May 22, 2022
ea5165a
Small tweak
jhale May 22, 2022
23d7f69
Bound numpy while 1.22 breaks numba
jhale May 22, 2022
7876b16
pep8
jhale May 22, 2022
a4ccbbf
Remove numpy version
jhale May 23, 2022
89e89cc
Move back to old use
jhale May 23, 2022
af65b6b
fix
jhale May 23, 2022
e3c049d
flake8
jhale May 23, 2022
07b8f05
Test fixes
jhale May 23, 2022
98e2140
move mypy up
jhale May 23, 2022
2edac28
fix mypy install
jhale May 23, 2022
c70f63b
Fixes
jhale May 23, 2022
e6e5fb2
isort
jhale May 23, 2022
d7a7e1f
Free numpy version
jhale May 23, 2022
aa04fd6
Merge branch 'main' into jhale/mypy-fixes-and-ci
jhale May 23, 2022
675e85e
Might need to re-run tests once Docker images update
jhale May 23, 2022
1179dc7
Add mypy to image
jhale May 23, 2022
5b62896
Type pybind11 .
jhale May 24, 2022
8bcf985
Remove mypy install from CI
jhale May 24, 2022
917e3cf
Also wrap cell_perm_gmsh
jhale May 24, 2022
239d93d
Make dim optional, grab tdim?
jhale May 24, 2022
d078159
Reallow value to be passed
jhale May 24, 2022
99bc314
Allow lists to eval
jhale May 24, 2022
73f4d96
Remove comment
jhale May 24, 2022
ade54ba
Better API?
jhale May 24, 2022
7962032
Prefer this design.
jhale May 24, 2022
89c891b
Better singledispatch la
jhale May 24, 2022
16ec231
Bug fixes
jhale May 24, 2022
e0e30ec
Fix
jhale May 25, 2022
458bf41
Merge branch 'main' into jhale/mypy-fixes-and-ci
jhale May 25, 2022
947051b
Ignore return error
jhale May 25, 2022
9296b15
flake8
jhale May 25, 2022
d4244c6
Remove explicit casts
jhale May 25, 2022
b352006
Remove cryptic typings, ignore errors
jhale May 25, 2022
83cb9eb
Revert some files to main
jhale May 25, 2022
5477334
Remove explicit dim
jhale May 25, 2022
4dadcba
Small tweaks.
jhale May 25, 2022
e431eb0
Revert
jhale May 25, 2022
6bd4d06
flake8
jhale May 25, 2022
cca3b9f
Merge branch 'main' of github.com:FEniCS/dolfinx into jhale/mypy-fixe…
jhale May 25, 2022
8ebad9d
Merge branch 'main' into jhale/mypy-fixes-and-ci
jhale May 25, 2022
ecc87b3
Fix, should discuss this again
jhale May 25, 2022
a2d49c5
Merge remote-tracking branch 'origin/main' into jhale/mypy-fixes-and-ci
jhale May 25, 2022
2163afb
Merge branch 'jhale/mypy-fixes-and-ci' of github.com:FEniCS/dolfinx i…
jhale May 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ccpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,13 @@ jobs:
python3 -m isort --check dolfinx
python3 -m isort --check demo
python3 -m isort --check test
- name: mypy checks (non-blocking)
continue-on-error: true
run: |
cd python/
mypy dolfinx
mypy demo
mypy test
- name: clang-format C++ checks (non-blocking)
continue-on-error: true
run: |
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ RUN if [ "$MPI" = "mpich" ]; then \
# - Second set of packages are recommended and/or required to build
# documentation or run tests.
RUN pip3 install --no-binary="numpy" --no-cache-dir cffi mpi4py numba numpy==${NUMPY_VERSION} scipy && \
pip3 install --no-cache-dir cppimport flake8 isort jupytext matplotlib myst-parser pybind11==${PYBIND11_VERSION} pytest pytest-xdist sphinx sphinx_rtd_theme
pip3 install --no-cache-dir cppimport flake8 isort jupytext matplotlib mypy myst-parser pybind11==${PYBIND11_VERSION} pytest pytest-xdist sphinx sphinx_rtd_theme

# Install xtl, xtensor
RUN git clone -b ${XTL_VERSION} --single-branch --depth 1 https://github.com/xtensor-stack/xtl.git && \
Expand Down
7 changes: 4 additions & 3 deletions python/demo/demo_gmsh.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
from dolfinx.mesh import CellType, create_mesh, meshtags_from_entities

from mpi4py import MPI

# -

# Generate a mesh on each rank with the gmsh API, and create a DOLFINx
Expand Down Expand Up @@ -119,7 +120,7 @@
entities, values = distribute_entity_data(msh, 2, marked_facets, facet_values)

msh.topology.create_connectivity(2, 0)
mt = meshtags_from_entities(msh, 2, create_adjacencylist(entities), np.int32(values))
mt = meshtags_from_entities(msh, 2, create_adjacencylist(entities), values)
mt.name = "ball_d1_surface"

with XDMFFile(MPI.COMM_WORLD, "out_gmsh/mesh.xdmf", "w") as file:
Expand Down Expand Up @@ -179,7 +180,7 @@

entities, values = distribute_entity_data(msh, 2, marked_facets, facet_values)
msh.topology.create_connectivity(2, 0)
mt = meshtags_from_entities(msh, 2, create_adjacencylist(entities), np.int32(values))
mt = meshtags_from_entities(msh, 2, create_adjacencylist(entities), values)
mt.name = "ball_d2_surface"
with XDMFFile(MPI.COMM_WORLD, "out_gmsh/mesh.xdmf", "a") as file:
file.write_mesh(msh)
Expand Down Expand Up @@ -256,7 +257,7 @@

entities, values = distribute_entity_data(msh, 2, marked_facets, facet_values)
msh.topology.create_connectivity(2, 0)
mt = meshtags_from_entities(msh, 2, create_adjacencylist(entities), np.int32(values))
mt = meshtags_from_entities(msh, 2, create_adjacencylist(entities), values)
mt.name = "hex_d2_surface"

with XDMFFile(MPI.COMM_WORLD, "out_gmsh/mesh.xdmf", "a") as file:
Expand Down
12 changes: 5 additions & 7 deletions python/demo/demo_lagrange_variants.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@
# We begin this demo by importing everything we require.

# +
import matplotlib.pylab as plt
import numpy as np

import basix
import basix.ufl_wrapper
import ufl
from dolfinx import fem, mesh
from ufl import ds, dx, grad, inner
Expand All @@ -31,16 +34,12 @@
print("Demo should only be executed with DOLFINx real mode")
exit(0)

import matplotlib.pylab as plt

# -

# In addition to the imports seen in other demos, we also import Basix
# and its UFL wrapper directly. Basix is the element definition and
# tabulation library that is used by FEniCSx.

import basix
import basix.ufl_wrapper

# ## Equispaced points vs GLL points
# The basis function of Lagrange elements are defined by placing points
Expand Down Expand Up @@ -184,10 +183,9 @@ def saw_tooth(x):
for i in range(51):
pts.append([cell / 10 + i / 50 / 10, 0, 0])
cells.append(cell)
pts = np.array(pts)
values = uh.eval(pts, cells)
plt.plot(pts[:, 0], [saw_tooth(i[0]) for i in pts], "k--")
plt.plot(pts[:, 0], values, "r-")
plt.plot(pts, [saw_tooth(i[0]) for i in pts], "k--")
plt.plot(pts, values, "r-")

plt.legend(["function", "approximation"])
plt.ylim([-0.1, 0.4])
Expand Down
8 changes: 4 additions & 4 deletions python/demo/demo_pyvista.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ def in_circle(x):
midpoints = compute_midpoints(msh, msh.topology.dim, list(np.arange(num_cells, dtype=np.int32)))
cell_tags = meshtags(msh, msh.topology.dim, np.arange(num_cells), in_circle(midpoints))

cells, types, x = plot.create_vtk_mesh(msh, msh.topology.dim)
cells, types, x = plot.create_vtk_mesh(msh)
grid = pyvista.UnstructuredGrid(cells, types, x)

# As the mesh tags contain a value for every cell in the
Expand All @@ -129,7 +129,7 @@ def in_circle(x):
# only consisting of those entities that has value one in the
# mesh tags
cells, types, x = plot.create_vtk_mesh(
msh, msh.topology.dim, cell_tags.indices[cell_tags.values == 1])
msh, entities=cell_tags.indices[cell_tags.values == 1])

# We add this grid to the second plotter
sub_grid = pyvista.UnstructuredGrid(cells, types, x)
Expand Down Expand Up @@ -191,7 +191,7 @@ def in_circle(x):
# that as we have done previously
num_cells = msh.topology.index_map(msh.topology.dim).size_local
cell_entities = np.arange(num_cells, dtype=np.int32)
cells, types, x = plot.create_vtk_mesh(msh, msh.topology.dim, cell_entities)
cells, types, x = plot.create_vtk_mesh(msh, entities=cell_entities)
org_grid = pyvista.UnstructuredGrid(cells, types, x)

# We visualize the data
Expand Down Expand Up @@ -225,7 +225,7 @@ def plot_nedelec():
position="upper_edge", font_size=14, color="black")

# Next, we create a pyvista.UnstructuredGrid based on the mesh
pyvista_cells, cell_types, x = plot.create_vtk_mesh(msh, msh.topology.dim)
pyvista_cells, cell_types, x = plot.create_vtk_mesh(msh)
grid = pyvista.UnstructuredGrid(pyvista_cells, cell_types, x)

# Add this grid (as a wireframe) to the plotter
Expand Down
5 changes: 3 additions & 2 deletions python/demo/demo_tnt-elements.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
grad, inner, sin)

from mpi4py import MPI

# -

# ## Defining a degree 1 TNT element
Expand Down Expand Up @@ -68,8 +69,8 @@
# +
geometry = basix.geometry(basix.CellType.quadrilateral)
topology = basix.topology(basix.CellType.quadrilateral)
x = [[], [], [], []]
M = [[], [], [], []]
x = [[], [], [], []] # type: ignore [var-annotated]
M = [[], [], [], []] # type: ignore [var-annotated]

for v in topology[0]:
x[0].append(np.array(geometry[v]))
Expand Down
12 changes: 4 additions & 8 deletions python/dolfinx/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,11 @@

stored_dlopen_flags = sys.getdlopenflags()

# Developer note: below is related to OpenMPI
# Fix dlopen flags (may need reorganising)
# Developer note: below is related to OpenMPI
# Fix dlopen flags
if "linux" in sys.platform:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This need to be checked. My recollection os that the removed code was required when MKL is used.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it stands it won't work because ctypes.RTLD_NOW no longer exists, at least in any of my installs. There is ctypes.RTLD_GLOBAL and ctypes.RTLD_LOCAL.

The static checker picked this up as an error which was quite nice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrisrichardson do you know the purpose/necessity of this code path?

# FIXME: What with other platforms?
try:
from ctypes import RTLD_GLOBAL, RTLD_NOW
except ImportError:
RTLD_NOW = 2
RTLD_GLOBAL = 256
RTLD_NOW = 2
RTLD_GLOBAL = 256
sys.setdlopenflags(RTLD_NOW | RTLD_GLOBAL)
del sys

Expand Down
1 change: 0 additions & 1 deletion python/dolfinx/fem/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
from dolfinx.fem.bcs import (DirichletBCMetaClass, bcs_by_block, dirichletbc,
locate_dofs_geometrical, locate_dofs_topological)
from dolfinx.fem.dofmap import DofMap
from dolfinx.fem.formmanipulations import adjoint
from dolfinx.fem.forms import FormMetaClass, extract_function_spaces, form
from dolfinx.fem.function import (Constant, Expression, Function,
FunctionSpace, TensorFunctionSpace,
Expand Down
84 changes: 47 additions & 37 deletions python/dolfinx/fem/assemble.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Garth N. Wells
# Copyright (C) 2018-2022 Garth N. Wells, Jack S. Hale
#
# This file is part of DOLFINx (https://www.fenicsproject.org)
#
Expand All @@ -8,13 +8,8 @@
from __future__ import annotations

import collections
import typing

if typing.TYPE_CHECKING:
from dolfinx.fem.forms import FormMetaClass
from dolfinx.fem.bcs import DirichletBCMetaClass

import functools
import typing

import numpy as np

Expand All @@ -23,6 +18,8 @@
from dolfinx import la
from dolfinx.cpp.fem import pack_coefficients as _pack_coefficients
from dolfinx.cpp.fem import pack_constants as _pack_constants
from dolfinx.fem.bcs import DirichletBCMetaClass
from dolfinx.fem.forms import FormMetaClass, form_types


def pack_constants(form: typing.Union[FormMetaClass,
Expand Down Expand Up @@ -133,11 +130,17 @@ def assemble_scalar(M: FormMetaClass, constants=None, coeffs=None):
# -- Vector assembly ---------------------------------------------------------

@functools.singledispatch
def assemble_vector(L: FormMetaClass, constants=None, coeffs=None) -> la.VectorMetaClass:
def assemble_vector(L: typing.Any,
constants=None, coeffs=None):
return _assemble_vector_form(L, constants, coeffs)


@assemble_vector.register(FormMetaClass)
def _assemble_vector_form(L: form_types, constants=None, coeffs=None) -> la.VectorMetaClass:
"""Assemble linear form into a new Vector.

Args:
L: The linear form assemble.
L: The linear form to assemble.
constants: Constants that appear in the form. If not provided,
any required constants will be computed.
coeffs: Coefficients that appear in the form. If not provided,
Expand All @@ -162,12 +165,12 @@ def assemble_vector(L: FormMetaClass, constants=None, coeffs=None) -> la.VectorM
b.array[:] = 0
constants = constants or _pack_constants(L)
coeffs = coeffs or _pack_coefficients(L)
_cpp.fem.assemble_vector(b.array, L, constants, coeffs)
_assemble_vector_array(b.array, L, constants, coeffs)
return b


@assemble_vector.register(np.ndarray)
def _(b: np.ndarray, L: FormMetaClass, constants=None, coeffs=None):
def _assemble_vector_array(b: np.ndarray, L: FormMetaClass, constants=None, coeffs=None):
"""Assemble linear form into a new Vector.

Args:
Expand Down Expand Up @@ -201,67 +204,74 @@ def _(b: np.ndarray, L: FormMetaClass, constants=None, coeffs=None):


@functools.singledispatch
def assemble_matrix(a: FormMetaClass, bcs: typing.List[DirichletBCMetaClass] = None,
diagonal: float = 1.0,
constants=None, coeffs=None) -> la.MatrixCSRMetaClass:
def assemble_matrix(a: typing.Any,
bcs: typing.List[DirichletBCMetaClass] = None,
diagonal: float = 1.0, constants=None, coeffs=None):
return _assemble_matrix_form(a, bcs, diagonal, constants, coeffs)


@assemble_matrix.register
def _assemble_matrix_csr(A: la.MatrixCSRMetaClass, a: form_types,
bcs: typing.List[DirichletBCMetaClass] = None,
diagonal: float = 1.0, constants=None, coeffs=None) -> la.MatrixCSRMetaClass:
"""Assemble bilinear form into a matrix.

Args:
Args:
a: The bilinear form assemble.
bcs: Boundary conditions that affect the assembled matrix.
Degrees-of-freedom constrained by a boundary condition will
have their rows/columns zeroed and the value ``diagonal``
set on on the matrix diagonal.
have their rows/columns zeroed and the value ``diagonal`` set
on on the matrix diagonal.
constants: Constants that appear in the form. If not provided,
any required constants will be computed.
coeffs: Coefficients that appear in the form. If not provided,
any required coefficients will be computed.

Returns:
Matrix representation of the bilinear form ``a``.

Note:
The returned matrix is not finalised, i.e. ghost values are not
accumulated.

"""
bcs = [] if bcs is None else bcs
A = create_matrix(a)
assemble_matrix(A, a, bcs, diagonal, constants, coeffs)
constants = _pack_constants(a) if constants is None else constants
coeffs = _pack_coefficients(a) if coeffs is None else coeffs
_cpp.fem.assemble_matrix(A, a, constants, coeffs, bcs)

# If matrix is a 'diagonal'block, set diagonal entry for constrained
# dofs
if a.function_spaces[0] is a.function_spaces[1]:
_cpp.fem.insert_diagonal(A, a.function_spaces[0], bcs, diagonal)
return A


@assemble_matrix.register(la.MatrixCSRMetaClass)
def _(A: la.MatrixCSRMetaClass, a: FormMetaClass,
bcs: typing.List[DirichletBCMetaClass] = None,
diagonal: float = 1.0, constants=None, coeffs=None) -> la.MatrixCSRMetaClass:
@assemble_matrix.register(FormMetaClass)
def _assemble_matrix_form(a: form_types, bcs: typing.List[DirichletBCMetaClass] = None,
diagonal: float = 1.0,
constants=None, coeffs=None) -> la.MatrixCSRMetaClass:
"""Assemble bilinear form into a matrix.

Args:
Args:
a: The bilinear form assemble.
bcs: Boundary conditions that affect the assembled matrix.
Degrees-of-freedom constrained by a boundary condition will
have their rows/columns zeroed and the value ``diagonal`` set
on on the matrix diagonal.
have their rows/columns zeroed and the value ``diagonal``
set on on the matrix diagonal.
constants: Constants that appear in the form. If not provided,
any required constants will be computed.
coeffs: Coefficients that appear in the form. If not provided,
any required coefficients will be computed.

Returns:
Matrix representation of the bilinear form ``a``.

Note:
The returned matrix is not finalised, i.e. ghost values are not
accumulated.

"""
bcs = [] if bcs is None else bcs
constants = _pack_constants(a) if constants is None else constants
coeffs = _pack_coefficients(a) if coeffs is None else coeffs
_cpp.fem.assemble_matrix(A, a, constants, coeffs, bcs)

# If matrix is a 'diagonal'block, set diagonal entry for constrained
# dofs
if a.function_spaces[0] is a.function_spaces[1]:
_cpp.fem.insert_diagonal(A, a.function_spaces[0], bcs, diagonal)
A: la.MatrixCSRMetaClass = create_matrix(a)
_assemble_matrix_csr(A, a, bcs, diagonal, constants, coeffs)
return A


Expand Down
Loading