Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate VariableArithmetic to NamedArrayArithmetic #8244

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

andersy005
Copy link
Member

  • towards NamedArray tracking issue #8238
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

@Illviljan
Copy link
Contributor

Could you do dtypes.py in separate PR @andersy005 ? That one can be merged quite easily I think.

@dcherian
Copy link
Contributor

@andersy005 what are your thoughts on migrating tests for this.

In theory it seems like we'd want to test both Variable and NamedArray, but I also don't like the idea of them potentially diverging once we switch to a new repo. OTOH perhaps there won't be much updating of such core functionality.

cc @pydata/xarray for opinions

@andersy005
Copy link
Member Author

Could you do dtypes.py in separate PR @andersy005 ? That one can be merged quite easily I think.

@TomNicholas
Copy link
Contributor

In theory it seems like we'd want to test both Variable and NamedArray, but I also don't like the idea of them potentially diverging once we switch to a new repo. OTOH perhaps there won't be much updating of such core functionality.

Maybe we could avoid divergence by importing from the NamedArray test suite and using the same arithmetic tests but on the Variable class?

@andersy005 andersy005 added the topic-NamedArray Lightweight version of Variable label Sep 29, 2023
@andersy005
Copy link
Member Author

In theory it seems like we'd want to test both Variable and NamedArray, but I also don't like the idea of them potentially diverging once we switch to a new repo. OTOH perhaps there won't be much updating of such core functionality.

Maybe we could avoid divergence by importing from the NamedArray test suite and using the same arithmetic tests but on the Variable class?

i'm of the same opinion as @TomNicholas that we can package the shared tests in namedarray, and have the tests for Variable inherit those tests while also including their own specific tests.

@dcherian dcherian mentioned this pull request Oct 5, 2023
20 tasks
@maxrjones
Copy link
Contributor

In theory it seems like we'd want to test both Variable and NamedArray, but I also don't like the idea of them potentially diverging once we switch to a new repo. OTOH perhaps there won't be much updating of such core functionality.

Maybe we could avoid divergence by importing from the NamedArray test suite and using the same arithmetic tests but on the Variable class?

i'm of the same opinion as @TomNicholas that we can package the shared tests in namedarray, and have the tests for Variable inherit those tests while also including their own specific tests.

It should work to use pytest_collection_modifyitems to collect tests from namedarray that should be also tested in xarray without modification. It's slightly more complicated to add external tests that need to override fixtures, but pytest-dev/pytest#421 (comment) lays out a nice solution (see example pseudo-code below for NamedArray/Variable example). I recognize that this type of property testing would be well suited for Hypothesis, but also expect that would slow down progress too much to be an immediate solution.

@andersy005 when do you think would be the best time to test out this method for migrating tests? I could try it out anytime and don't see many downsides to testing while namedarray is still coupled with xarray.

# namedarray/testing.py

import numpy as np

class NamedArrayTestSuite:
    @pytest.fixture
    def target(self):
        """Fixture that needs to be re-declared"""
        assert 0
    
    @pytest.fixture
    def data(self):
        return 0.5 * np.arange(10).reshape(2, 5)

    def test_properties(self, target, data):
        assert target.dims == ("x", "y")
        assert np.array_equal(target.data, data)
        assert target.attrs == {"key": "value"}
        assert target.ndim == 2
        assert target.sizes == {"x": 2, "y": 5}
        assert target.size == 10
        assert target.nbytes == 80
        assert len(target) == 2
# namedarray/tests/test_namedarray.py

from namedarray.testing import NamedArrayTestSuite
from namedarray import NamedArray

class TestNamedArray(NamedArrayTestSuite):

    @pytest.fixture
    def target(self, data):
        return NamedArray(["x", "y"], data, {"key": "value"})
# xarray/tests/test_variable.py

from namedarray.testing import NamedArrayTestSuite
from xarray import Variable

class TestVariable(NamedArrayTestSuite):
    
    @pytest.fixture
    def target(self, data):
        data = 0.5 * np.arange(10).reshape(2, 5)
        return Variable(["x", "y"], data, {"key": "value"})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
io topic-backends topic-NamedArray Lightweight version of Variable
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants