Skip to content

Commit

Permalink
Merge pull request #84 from linkml/issue-81
Browse files Browse the repository at this point in the history
Rewrite README instructions to use pipx
  • Loading branch information
pkalita-lbl committed Dec 1, 2023
2 parents 50134ee + 2ff1c80 commit dc8af4e
Show file tree
Hide file tree
Showing 4 changed files with 91 additions and 116 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,6 @@ dmypy.json

# Pyre type checker
.pyre/

.idea
.vscode
190 changes: 82 additions & 108 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,51 @@
# LinkML Project Cookiecutter

A [Cookiecutter](https://cookiecutter.readthedocs.io/en/stable/) template for
projects using [LinkML](https://github.com/linkml/linkml).
A [Cookiecutter](https://cookiecutter.readthedocs.io/en/stable/) template for projects using [LinkML](https://github.com/linkml/linkml).

## Standard Protocol
## Prerequisites

### Step 1: Create a virtual environment
The following are required and recommended tools for using this cookiecutter and the LinkML project that it generates. This is all one-time setup, so if you have already done it skip to the [next section]()!

Create a Python virtual environment.
You can read [this guide](https://realpython.com/python-virtual-environments-a-primer/)
to learn more about them and how to create one. We suggest using poetry, but
you can use any tool you like. Please note, most LinkML tools work best in
Python 3.8 or higher.
* **Python >= 3.8**

LinkML tools are mainly written in Python, so you will need a recent Python interpreter to run this generator and to use the generated project.

An example using poetry:

```bash
curl -sSL https://install.python-poetry.org | python3 -
```
* **pipx**
pipx is a tool for managing isolated Python-based applications. It is the recommended way to install Poetry and cruft. To install pipx follow the instructions here: https://pypa.github.io/pipx/installation/

```bash
mkdir linkml-projects
cd linkml-projects
poetry init # creates a new poetry project with pyproject.toml
# Note this is not a new linkml project,
# it is just a virtual environment to install cruft.
poetry add click==8.1.3 # this creates your virtual environment.
```

Add `poetry-dynamic-versioning` as a plugin
* **Poetry**

Poetry is a Python project management tool. You will use it in your generated project to manage dependencies and build distribution files. If you have pipx installed you can install Poetry by running:
```shell
pipx install poetry
```
For other installation methods see: https://python-poetry.org/docs/#installation


```bash
poetry self add "poetry-dynamic-versioning[plugin]"
```
* **Poetry Dynamic Versioning Plugin**:

### Step 2: Install the cruft tool in your virtual environment
This plugin automatically updates certain version strings in your generated project when you publish it. Your generated project will automatically be set up to use it. Install it by running:
```shell
poetry self add "poetry-dynamic-versioning[plugin]"
```

This tool will help you keep your project up to date with the latest LinkML
tools and best practices.

In your poetry virtual environment:
* **cruft**

```bash
poetry shell
poetry add cruft
```
cruft is a tool for generating projects based on a cookiecutter (like this one!) as well as keeping those projects updated if the original cookiecutter changes. Install it with pipx by running:
```shell
pipx install cruft
```
You may also choose to not have a persistent installation of cruft, in which case you would replace any calls to the `cruft` command below with `pipx run cruft`.

## Creating a new project

### Step 3: Use cruft to create your brand new LinkML project
### Step 1: Generate the project files

In your poetry virtual environment:
To generate a new LinkML project run the following:

```bash
cruft create https://github.com/linkml/linkml-project-cookiecutter
Expand All @@ -58,115 +55,95 @@ You will be prompted for a few values. The defaults are fine for most
projects, but do name your project something that makes sense to you!
The interactive session will guide you through the process:

- `project_name`: Name of the project, use kebab-case with no spaces.
Suggestions:
- `patient-observation-schema`
- `sample-collection-metadata`
- `resume-standard`
- `project_description`: Description of the project.
- A single brief sentence is recommended
- Can easily be modified later
- `full_name`: Your name
- `email`: your email
- `main_schema_class`:
- This is used to generate an example schema which you can edit
- The value of this field is ignored if this is a schemasheets project
- You can always change this later
1. `project_name`: Name of the project, use kebab-case with no spaces.
Suggestions:
- `patient-observation-schema`
- `sample-collection-metadata`
- `resume-standard`
2. `github_org`: Your github username or organization name. This is used to construct links to documentation pages.
3. `project_description`: Description of the project.
- A single brief sentence is recommended
- Can easily be modified later
4. `full_name`: Your name
5. `email`: Your email
6. `license`: Choose a license for the project. If your desired license is not listed you can update or remove the `LICNSE` file in the generated project.
7. `main_schema_class`:
- This is used to generate an example schema which you can edit
- The value of this field is ignored if this is a schemasheets project
- You can always change this later
- Examples: `Person`, `Observation`, `Sample`, `Entry`, `Gene`, `Event`
- `create_python_project`
- If "Yes", set this up as a python project
- Select Yes if you want to make data classes that can be used by developers
- `use_schemasheets`
- If "Yes", set this to obtain your schema from
8. `create_python_project`
- If "Yes", set this up as a python project
- Select Yes if you want to make data classes that can be used by developers
9. `use_schemasheets`
- If "Yes", set this to obtain your schema from
[schemasheets](https://linkml.io/schemasheets)
- `google_sheet_id`
- Ignore/use default value if answer to previous question was "No"
- If you are using schemasheets then enter your google doc ID here
- If you like you can leave the default value, and this will use the demo
schema
- `google_sheet_tabs`
- Ignore/use default value if not using schemasheets
- If you are using schemasheets, enter a space-separated list of tabs
- your tabs in your sheet MUST NOT have spaces in them
- `github_token_for_pypi_deployment`:
- The github token name which aligns with your autogenerated PyPI token for
making releases.
- This helps automated releases to PyPI
- This should be ignored if this is not a python project
- Even if this is a python project, you can leave blank and fill in later

This will generate the project folder abiding by the template configuration
specified by `linkml-project-cookiecutter` in the
[`cookiecutter.json`](https://github.com/linkml/linkml-project-cookiecutter/blob/main/cookiecutter.json)
file.

This will generate the project folder very similar to what is mentioned in the
[linkml-project-template](https://github.com/linkml/linkml-project-template)
project.

For more docs, see
[linkml/linkml-project-cookiecutter](https://github.com/linkml/linkml-project-cookiecutter).

### Step 4: Setup the LinkML project
10. `google_sheet_id`
- Ignore/use default value if answer to previous question was "No"
- If you are using schemasheets then enter your google doc ID here
- If you like you can leave the default value, and this will use the demo schema
11. `google_sheet_tabs`
- Ignore/use default value if not using schemasheets
- If you are using schemasheets, enter a space-separated list of tabs
- your tabs in your sheet MUST NOT have spaces in them
12. `github_token_for_pypi_deployment`:
- The github token name which aligns with your autogenerated PyPI token for making releases.
- This helps automated releases to PyPI
- This should be ignored if this is not a python project
- Even if this is a python project, you can leave blank and fill in later

### Step 2: Set up the LinkML project

Optionally, pass custom configuration to linkml generators by tuning the global configuration file 'config.yaml' with preferred options. An example file is supplied by the project to illustrate interface and defaults.

Additionally, pass command-line arguments to linkml generators inside the Makefile via environment variables in 'config.env' file. An example file is supplied by the project, passing '--config-file config.yaml' to gen-project.


Change to the folder your generated project is in

```bash
cd my-awesome-schema # using the folder example above
make setup
```

### Step 5: Edit the schema
### Step 3: Edit the schema

Edit the schema (the .yaml file) in the
`linkml-projects/my-awesome-schema/src/my_awesome_schema/schema` folder
`src/my_awesome_schema/schema` folder

```bash
nano src/my_awesome_schema/schema/my_awesome_schema.yaml
```

### Step 6: Validate the schema
### Step 4: Validate the schema

```bash
make test
```

### Step 7: Auto-generate your documentation locally
### Step 5: Generate documentation locally

LinkML generates schema documentation automatically. Step 7 here, allows you to
preview the documentation that LinkML generates before pushing to GitHub.
Note, this template comes with GitHub Actions that autogenerate this
documentation on release of your schema repository at a URL like this one:
[https://my-user-or-organization.github.io/my-awesome-schema/]
LinkML generates schema documentation automatically. The template comes with GitHub Actions that generate and publish the documentation when you push schema changes to GitHub. The published documentation can be found at a URL like this one:
`https://{github-user-or-organization}.github.io/{project-name}/`

You can also preview the documentation locally before pushing to GitHub by running:

```bash
make serve
```

### Step 8: Create a github project
### Step 6: Create a GitHub project

1. Go to [https://github.com/new] and follow the instructions, being sure to
NOT add a README or .gitignore file (this cookiecutter template will take
care of this for you)
1. Go to https://github.com/new and follow the instructions, being sure to NOT add a `README` or `.gitignore` file (this cookiecutter template will take care of those files for you)

2. Add the remote to your local git repository

```bash
git remote add origin https://github.com/my-user-or-organization/my-awesome-schema.git
git remote add origin https://github.com/{github-user-or-organization}/{project-name}.git
git branch -M main
git push -u origin main
```

### Step 9: Deploy documentation

`make deploy`

### Step 10: Register the schema
### Step 7: Register the schema

See [How to register a schema](https://linkml.io/linkml/faq/contributing.html#how-do-i-register-my-schema)

Expand All @@ -176,7 +153,6 @@ In order to be up-to-date with the template, first check if there is a mismatch
between the project's boilerplate code and the template by running:
```bash
poetry shell
cruft check
```
Expand All @@ -195,8 +171,6 @@ up-to-date by the following:
FAILURE: Project's cruft is out of date! Run `cruft update` to clean this mess up.
```
For viewing the difference, run `cruft diff`. This shows the difference between
the project's boilerplate code and the template's latest version.
For viewing the difference, run `cruft diff`. This shows the difference between the project's boilerplate code and the template's latest version.
After running `cruft update`, the project's boilerplate code will be updated to
the latest version of the template.
After running `cruft update`, the project's boilerplate code will be updated to the latest version of the template.
8 changes: 3 additions & 5 deletions {{cookiecutter.project_name}}/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ endif


# basename of a YAML file in model/
.PHONY: all clean
.PHONY: all clean setup gen-project gen-examples gendoc git-init-add git-init git-add git-commit git-status

# note: "help" MUST be the first target in the file,
# when the user types "make" they should get help info
Expand All @@ -58,11 +58,10 @@ status: check-config
@echo "Source: $(SOURCE_SCHEMA_PATH)"

# generate products and add everything to github
setup: install gen-project gen-examples gendoc git-init-add
setup: git-init install gen-project gen-examples gendoc git-add git-commit

# install any dependencies required for building
install:
git init
poetry install
.PHONY: install

Expand Down Expand Up @@ -172,8 +171,7 @@ git-init-add: git-init git-add git-commit git-status
git-init:
git init
git-add: .cruft.json
git add .gitignore .github .cruft.json Makefile LICENSE *.md examples utils about.yaml mkdocs.yml poetry.lock project.Makefile pyproject.toml src/{{cookiecutter.__project_slug}}/schema/*yaml src/*/datamodel/*py src/data src/docs tests src/*/_version.py
git add $(patsubst %, project/%, $(PROJECT_FOLDERS))
git add .
git-commit:
git commit -m 'chore: initial commit' -a
git-status:
Expand Down
6 changes: 3 additions & 3 deletions {{cookiecutter.project_name}}/tests/test_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import unittest

from linkml_runtime.loaders import yaml_loader
from {{cookiecutter.__project_slug}}.datamodel.{{cookiecutter.__project_slug}} import {{cookiecutter.main_schema_class}}
from {{cookiecutter.__project_slug}}.datamodel.{{cookiecutter.__project_slug}} import {{cookiecutter.main_schema_class}}Collection

ROOT = os.path.join(os.path.dirname(__file__), '..')
DATA_DIR = os.path.join(ROOT, "src", "data", "examples")
Expand All @@ -16,7 +16,7 @@ class TestData(unittest.TestCase):
"""Test data and datamodel."""

def test_data(self):
"""Date test."""
"""Data test."""
for path in EXAMPLE_FILES:
obj = yaml_loader.load(path, target_class={{cookiecutter.main_schema_class}})
obj = yaml_loader.load(path, target_class={{cookiecutter.main_schema_class}}Collection)
assert obj

0 comments on commit dc8af4e

Please sign in to comment.