Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Add the "Doctests" workflow to run doctests weekly #2456

Merged
merged 15 commits into from
Apr 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .github/workflows/ci_doctests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# This workflow installs PyGMT and runs all doctests

name: Doctests

on:
# push:
# branches: [ main ]
# pull_request:
# Schedule weekly tests on Sunday
schedule:
- cron: '0 0 * * 0'

jobs:
test:
name: ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macOS-latest, windows-latest]
timeout-minutes: 30
defaults:
run:
shell: bash -l {0}

steps:
# Cancel previous runs that are not completed
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.11.0
with:
access_token: ${{ github.token }}

# Checkout current git repository
- name: Checkout
uses: actions/checkout@v3.4.0
with:
# fetch all history so that setuptools-scm works
fetch-depth: 0

# Install Mambaforge with conda-forge dependencies
- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2.2.0
with:
activate-environment: pygmt
python-version: '3.11'
channels: conda-forge,nodefaults
channel-priority: strict
miniforge-version: latest
miniforge-variant: Mambaforge
mamba-version: "*"
run-post: false
use-mamba: true

# Install GMT and other required dependencies from conda-forge
- name: Install dependencies
run: |
mamba install gmt=6.4.0 numpy \
pandas xarray netCDF4 packaging \
build make 'pytest>=6.0' \
pytest-cov pytest-doctestplus pytest-mpl sphinx-gallery \
contextily geopandas ipython rioxarray

# Show installed pkg information for postmortem diagnostic
- name: List installed packages
run: mamba list

# Download cached remote files (artifacts) from GitHub
- name: Download remote data from GitHub
uses: dawidd6/action-download-artifact@v2.26.0
with:
workflow: cache_data.yaml
workflow_conclusion: success
name: gmt-cache
path: .gmt

# Move downloaded files to ~/.gmt directory and list them
- name: Move and list downloaded remote files
run: |
mkdir -p ~/.gmt
mv .gmt/* ~/.gmt
# Change modification times of the two files, so GMT won't refresh it
touch ~/.gmt/server/gmt_data_server.txt ~/.gmt/server/gmt_hash_server.txt
ls -lhR ~/.gmt

# Install the package that we want to test
- name: Install the package
run: make install

# Run the doctests
- name: Run doctests
run: make doctest PYTEST_EXTRA="-r P"
6 changes: 0 additions & 6 deletions .github/workflows/ci_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -130,14 +130,8 @@ jobs:

# Run the regular tests
- name: Run tests
if: ${{ !endsWith(github.event.schedule, '3') }}
run: make test PYTEST_EXTRA="-r P"

# Run full tests including doctests on Wednesday
- name: Run full tests
if: github.event_name == 'schedule' && endsWith(github.event.schedule, '3')
run: make fulltest PYTEST_EXTRA="-r P"

# Upload diff images on test failure
- name: Upload diff images if any test fails
uses: actions/upload-artifact@v3
Expand Down
28 changes: 16 additions & 12 deletions doc/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,7 @@ There are 11 configuration files located in `.github/workflows`:
2. `ci_tests.yaml` (Tests on Linux/macOS/Windows)
Copy link
Member Author

@seisman seisman Mar 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Off-topic question: Currently, we're documenting what a workflow does in the maintainers' guide, but it's very likely that we may forget to update the guides when we make changes to a workflow.

Perhaps we should only keep a short, one-sentence description for each workflow in the maintainers' guide and put a much longer description in the workflow file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @GenericMappingTools/pygmt-maintainers for comments on this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe have a README.md under https://github.com/GenericMappingTools/pygmt/tree/v0.9.0/.github/workflows to document, so that people looking at the folder can see what the *.yml files are for? Also fine with including it in the YAML file since that will be more visible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fine with including it in the YAML file since that will be more visible.

I'm inclined to this option because (1) we won't forget to update the workflow descriptions; (2) the workflows are more self-explanatory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fine with including it in the YAML file since that will be more visible.

I'm inclined to this option because (1) we won't forget to update the workflow descriptions; (2) the workflows are more self-explanatory.

Ok, so just document things in the YAML files and remove the descriptions from the doc/maintenance.md file. Can do this in another PR once this one is merged.


This is run on every commit to the *main* and pull request branches.
It is also scheduled to run regular tests daily and run full tests
(including doctests) on Wednesday on the *main* branch.
It is also scheduled to run regular tests daily on the *main* branch.
In draft pull requests, only two jobs on Linux are triggered to save on
Continuous Integration resources:

Expand All @@ -93,7 +92,12 @@ There are 11 configuration files located in `.github/workflows`:
via the [Codecov GitHub Action](https://github.com/codecov/codecov-action).
More codecov related configurations are stored in `.github/codecov.yml`.

3. `ci_docs.yml` (Build documentation on Linux/macOS/Windows)
3. `ci_doctests.yaml` (Doctests on Linux/macOS/Windows)

This workflow is scheduled to run all doctests every Sunday on the *main*
branch.

4. `ci_docs.yml` (Build documentation on Linux/macOS/Windows)

This is run on every commit to the *main* and pull request branches.
In draft pull requests, only the job on Linux is triggered to save on
Expand All @@ -106,47 +110,47 @@ There are 11 configuration files located in `.github/workflows`:
from the *main* branch onto the `dev` folder of the *gh-pages* branch.
* Updating the `latest` documentation link to the new release.

4. `ci_tests_dev.yaml` (GMT Dev Tests on Linux/macOS/Windows).
5. `ci_tests_dev.yaml` (GMT Dev Tests on Linux/macOS/Windows).

This is triggered when a PR is marked as "ready for review", or using the
slash command `/test-gmt-dev`. It is also scheduled to run on Monday,
Wednesday and Friday on the *main* branch.

5. `cache_data.yaml` (Caches GMT remote data files needed for GitHub Actions CI)
6. `cache_data.yaml` (Caches GMT remote data files needed for GitHub Actions CI)

This is scheduled to run every Sunday at 12:00 (UTC).
If new remote files are needed urgently, maintainers can manually uncomment
the 'pull_request:' line in that `cache_data.yaml` file to refresh the cache.

6. `publish-to-pypi.yml` (Publish wheels to TestPyPI and PyPI)
7. `publish-to-pypi.yml` (Publish wheels to TestPyPI and PyPI)

This workflow is ran to publish wheels to TestPyPI (for testing only) and
PyPI. Archives will be pushed to TestPyPI on every commit to the *main*
branch and tagged releases, and to PyPI for tagged releases only. Note that
authentication to TestPyPI/PyPI is done via OpenID Connect, see also
https://github.com/pypa/gh-action-pypi-publish/tree/release/v1#publishing-with-openid-connect

7. `release-drafter.yml` (Drafts the next release notes)
8. `release-drafter.yml` (Drafts the next release notes)

This workflow is run to update the next releases notes as pull requests are
merged into the main branch.
merged into the *main* branch.

8. `check-links.yml` (Check links in the repository and website)
9. `check-links.yml` (Check links in the repository and website)

This workflow is run weekly to check all external links in plaintext and
HTML files. It will create an issue if broken links are found.

9. `format-command.yml` (Format the codes using slash command)
10. `format-command.yml` (Format the codes using slash command)

This workflow is triggered in a PR if the slash command `/format` is used.

10. `dvc-diff.yml` (Report changes to test images on dvc remote)
11. `dvc-diff.yml` (Report changes to test images on dvc remote)

This workflow is triggered in a PR when any *.png.dvc files have been added,
modified, or deleted. A GitHub comment will be published that contains a summary
table of the images that have changed along with a visual report.

11. `release-baseline-images.yml` (Upload the ZIP archive of baseline images as a release asset)
12. `release-baseline-images.yml` (Upload the ZIP archive of baseline images as a release asset)

This workflow is run to upload the ZIP archive of baseline images as a release
asset when a release is published.
Expand Down