Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linting #55

Merged
merged 13 commits into from
Mar 11, 2023
153 changes: 90 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,43 @@
# LinkML Project Cookiecutter

A [Cookiecutter](https://cookiecutter.readthedocs.io/en/stable/) template for projects using [`Linkml`](https://github.com/linkml/linkml).

A [Cookiecutter](https://cookiecutter.readthedocs.io/en/stable/) template for
projects using [LinkML](https://github.com/linkml/linkml).

## Standard Protocol

### Step 1: Create a virtual environment

Create a Python virtual environment. You can read [this
guide](https://realpython.com/python-virtual-environments-a-primer/) to learn more about them and how to create one.
We suggest using poetry, but you can use any tool you like. Please note, most LinkML tools work best in Python 3.8 or higher.
Create a Python virtual environment.
You can read [this guide](https://realpython.com/python-virtual-environments-a-primer/)
to learn more about them and how to create one. We suggest using poetry, but
you can use any tool you like. Please note, most LinkML tools work best in
Python 3.8 or higher.

An example using poetry:

```bash
curl -sSL https://install.python-poetry.org | python3 -
```

```bash
mkdir linkml-projects
cd linkml-projects
poetry init # creates a new poetry project with pyproject.toml. Note this is not a new linkml project, it is just a virtual environment to install cruft.
poetry init # creates a new poetry project with pyproject.toml
# Note this is not a new linkml project,
# it is just a virtual environment to install cruft.
poetry add click==8.1.3 # this creates your virtual environment.
```

Add `poetry-dynamic-versioning` as a plugin
```

```bash
poetry self add "poetry-dynamic-versioning[plugin]"
```

### Step 2: Install the cruft tool in your virtual environment
This tool will help you keep your project up to date with the latest LinkML tools and best practices.

This tool will help you keep your project up to date with the latest LinkML
tools and best practices.

In your poetry virtual environment:

Expand All @@ -38,61 +46,73 @@ poetry shell
poetry add cruft
```

### Step 3: Use cruft to create your brand new LinkML project:
### Step 3: Use cruft to create your brand new LinkML project

In your poetry virtual environment:

```bash
cruft create https://github.com/linkml/linkml-project-cookiecutter
```

You will be prompted for a few values. The defaults are fine for most projects, but do name your project something
that makes sense to you! The interactive session will guide you through the process:
You will be prompted for a few values. The defaults are fine for most
projects, but do name your project something that makes sense to you!
The interactive session will guide you through the process:

- `project_name`: Name of the project, use kebab-case with no spaces. Suggestions:
- `patient-observation-schema`
- `sample-collection-metadata`
- `resume-standard`
- `project_name`: Name of the project, use kebab-case with no spaces.
Suggestions:
- `patient-observation-schema`
- `sample-collection-metadata`
- `resume-standard`
- `project_description`: Description of the project.
- A single brief sentence is recommended
- Can easily be modified later
- A single brief sentence is recommended
- Can easily be modified later
- `full_name`: Your name
- `email`: your email
- `main_schema_class`:
- This is used to generate an example schema which you can edit
- The value of this field is ignored if this is a schemasheets project
- You can always change this later
- This is used to generate an example schema which you can edit
- The value of this field is ignored if this is a schemasheets project
- You can always change this later
- Examples: `Person`, `Observation`, `Sample`, `Entry`, `Gene`, `Event`
- `create_python_project`
- If "Yes", set this up as a python project
- Select Yes if you want to make data classes that can be used by developers
- If "Yes", set this up as a python project
- Select Yes if you want to make data classes that can be used by developers
- `use_schemasheets`
- If "Yes", set this to obtain your schema from [schemasheets](https://linkml.io/schemasheets)
- If "Yes", set this to obtain your schema from
[schemasheets](https://linkml.io/schemasheets)
- `google_sheet_id`
- Ignore/use default value if answer to previous question was "No"
- If you are using schemasheets then enter your google doc ID here
- If you like you can leave the default value, and this will use the demo schema
- Ignore/use default value if answer to previous question was "No"
- If you are using schemasheets then enter your google doc ID here
- If you like you can leave the default value, and this will use the demo
schema
- `google_sheet_tabs`
- Ignore/use default value if not using schemasheets
- If you are using schemasheets, enter a space-separated list of tabs
- your tabs in your sheet MUST NOT have spaces in them
- Ignore/use default value if not using schemasheets
- If you are using schemasheets, enter a space-separated list of tabs
- your tabs in your sheet MUST NOT have spaces in them
- `github_token_for_pypi_deployment`:
- The github token name which aligns with your autogenerated PyPI token for making releases.
- This helps automated releases to PyPI
- This should be ignored if this is not a python project
- Even if this is a python project, you can leave blank and fill in later
- The github token name which aligns with your autogenerated PyPI token for
making releases.
- This helps automated releases to PyPI
- This should be ignored if this is not a python project
- Even if this is a python project, you can leave blank and fill in later

This will generate the project folder abiding by the template configuration specified by `linkml-project-cookiecutter` in the [`cookiecutter.json`](https://github.com/linkml/linkml-project-cookiecutter/blob/main/cookiecutter.json) file.
This will generate the project folder abiding by the template configuration
specified by `linkml-project-cookiecutter` in the
[`cookiecutter.json`](https://github.com/linkml/linkml-project-cookiecutter/blob/main/cookiecutter.json)
file.

This will generate the project folder very similar to what is mentioned in the [linkml-project-template](https://github.com/linkml/linkml-project-template) project.
This will generate the project folder very similar to what is mentioned in the
[linkml-project-template](https://github.com/linkml/linkml-project-template)
project.

See [linkml/linkml-project-cookiecutter](https://github.com/linkml/linkml-project-cookiecutter) for more docs.
For more docs, see
[linkml/linkml-project-cookiecutter](https://github.com/linkml/linkml-project-cookiecutter).

### Step 4: Setup the LinkML project

Optionally set project generation environment variables (see `gen-doc --help` and `gen-project --help`):
Optionally set project generation environment variables
(see `gen-doc --help` and `gen-project --help`):

```
```bash
export LINKML_COOKIECUTTER_GEN_DOC_ARGS=--no-mergeimports # example
export LINKML_COOKIECUTTER_GEN_PROJECT_ARGS=--no-mergeimports # example
```
Expand All @@ -106,7 +126,9 @@ make setup

### Step 5: Edit the schema

Edit the schema (the .yaml file) in the linkml-projects/my-awesome-schema/src/my_awesome_schema/schema folder
Edit the schema (the .yaml file) in the
`linkml-projects/my-awesome-schema/src/my_awesome_schema/schema` folder

```bash
nano src/my_awesome_schema/schema/my_awesome_schema.yaml
```
Expand All @@ -117,28 +139,31 @@ nano src/my_awesome_schema/schema/my_awesome_schema.yaml
make test
```

### Step 7: Auto-generate your documentation locally.
LinkML generates schema documenation automatically. Step 7 here, allows you to preview the documentation
that LinkML generates before pushing to GitHub. Note, this template comes with GitHub
Actions that autogenerate this documentation on release of your schema repository at a URL like this one:
https://my-user-or-organization.github.io/my-awesome-schema/
### Step 7: Auto-generate your documentation locally

LinkML generates schema documenation automatically. Step 7 here, allows you to
preview the documentation that LinkML generates before pushing to GitHub.
Note, this template comes with GitHub Actions that autogenerate this
documentation on release of your schema repository at a URL like this one:
[https://my-user-or-organization.github.io/my-awesome-schema/]

```bash
make serve
```

### Step 8: Create a github project

8a: Go to https://github.com/new and follow the instructions, being sure to NOT add a README or .gitignore file (this
cookiecutter template will take care of this for you)
1. Go to [https://github.com/new] and follow the instructions, being sure to
NOT add a README or .gitignore file (this cookiecutter template will take
care of this for you)

8b: Add the remote to your local git repository`
2. Add the remote to your local git repository

```bash
git remote add origin https://github.com/my-user-or-organization/my-awesome-schema.git
git branch -M main
git push -u origin main
```
```bash
git remote add origin https://github.com/my-user-or-organization/my-awesome-schema.git
git branch -M main
git push -u origin main
```

### Step 9: Deploy documentation

Expand All @@ -150,29 +175,31 @@ See [How to register a schema](../faq/contributing)

## Keeping your project up to date

In order to be up-to-date with the template, first check if there is a mismatch between the project's boilerplate
code and the template by running:
In order to be up-to-date with the template, first check if there is a mismatch
between the project's boilerplate code and the template by running:

```bash
poetry shell
cruft check
```

This indicates if there is a difference between the current project's boilerplate code and the latest version of the
project template. If the project is up-to-date with the template:
This indicates if there is a difference between the current project's
boilerplate code and the latest version of the project template. If the project
is up-to-date with the template:

```
```output
SUCCESS: Good work! Project's cruft is up to date and as clean as possible :).
```

Otherwise, it will indicate that the project's boilerplate code is not up-to-date by the following:
Otherwise, it will indicate that the project's boilerplate code is not
up-to-date by the following:

```
```output
FAILURE: Project's cruft is out of date! Run `cruft update` to clean this mess up.
```

For viewing the difference, run `cruft diff`. This shows the difference between the project's boilerplate code and the
template's latest version.

After running `cruft update`, the project's boilerplate code will be updated to the latest version of the template.
For viewing the difference, run `cruft diff`. This shows the difference between
the project's boilerplate code and the template's latest version.

After running `cruft update`, the project's boilerplate code will be updated to
the latest version of the template.
3 changes: 0 additions & 3 deletions hooks/post_gen_project.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
"""Code to run after generating the project."""

import re
import sys
import shutil
from pathlib import Path

shutil.rmtree("licenses")

Expand Down
1 change: 0 additions & 1 deletion hooks/pre_gen_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import re
import sys
from pathlib import Path

MODULE_REGEX = re.compile(r'^[_a-zA-Z][_a-zA-Z0-9\-]+$')
project_name = '{{ cookiecutter.project_name }}'
Expand Down
43 changes: 22 additions & 21 deletions {{cookiecutter.project_name}}/.github/workflows/deploy-docs.yaml
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
---
name: Auto-deployment of {{cookiecutter.__project_slug}} Documentation
on:
push:
branches: [ main ]
branches: [main]

jobs:
build-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@main
with:
fetch-depth: 0 # otherwise, you will failed to push refs to dest repo

- name: Set up Python 3.
uses: actions/setup-python@v3
with:
python-version: 3.9
- name: Checkout
uses: actions/checkout@main
with:
fetch-depth: 0 # otherwise, you will failed to push refs to dest repo

- name: Install Poetry.
uses: snok/install-poetry@v1.3
- name: Set up Python 3.
uses: actions/setup-python@v3
with:
python-version: 3.9

- name: Install dependencies.
run: poetry install -E docs

- name: Build documentation.
run: |
mkdir -p docs
touch docs/.nojekyll
poetry run gen-doc -d docs src/{{cookiecutter.__project_slug}}/schema/{{cookiecutter.__project_slug}}.yaml
poetry run mkdocs gh-deploy
- name: Install Poetry.
uses: snok/install-poetry@v1.3

- name: Install dependencies.
run: poetry install -E docs

- name: Build documentation.
run: |
mkdir -p docs
touch docs/.nojekyll
poetry run gen-doc -d docs src/{{cookiecutter.__project_slug}}/schema/{{cookiecutter.__project_slug}}.yaml
poetry run mkdocs gh-deploy
22 changes: 3 additions & 19 deletions {{cookiecutter.project_name}}/.github/workflows/main.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Built from:
# https://docs.github.com/en/actions/guides/building-and-testing-python
# https://github.com/snok/install-poetry#workflows-and-tips

---
name: Build and test {{cookiecutter.__project_slug}}

on: [pull_request]
Expand All @@ -16,9 +16,6 @@ jobs:

steps:

#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v2

Expand All @@ -27,27 +24,14 @@ jobs:
with:
python-version: ${{ "{{" }} matrix.python-version {{ "}}" }}

#----------------------------------------------
# install & configure poetry
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1.3

#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
run: poetry install --no-interaction --no-root

#----------------------------------------------
# install your root project, if required
#----------------------------------------------
- name: Install library
- name: Install project
run: poetry install --no-interaction

#----------------------------------------------
# run test suite
#----------------------------------------------
- name: Run tests
- name: Run test suite
run: make test

Loading