-
Notifications
You must be signed in to change notification settings - Fork 383
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create Github Action for committing and pushing changes to submodules #256
Open
glatosinski
wants to merge
20
commits into
google:main
Choose a base branch
from
antmicro:cell_doc_tree_commit
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
0ca0b29
Proto of cell README generator
wgryncewicz 9fcefb0
cell_list generator and lib README patcher
wgryncewicz c74b4d8
Cell_list now appends links to cell readmes
wgryncewicz 5eb305b
Docs: generate cells documentation in CI
kgugala 9db2820
netlistsvg-generate removed - belongs to another branch
wgryncewicz b366982
Cell readme generator updated, includes GDS2 layouts
wgryncewicz 10c5e8f
Cell list generator includes cell links in table
wgryncewicz ab19ce3
Merge branch 'cell_doc_tree' of github.com:antmicro/skywater-pdk into…
wgryncewicz 1f3fcbb
Removed redundant cell_list Sphinx extension script
wgryncewicz ed83490
Rename cell readme generator script
wgryncewicz 3f498d9
Rename cell list generator script
wgryncewicz 98da26c
Added committing changes made to submodules
glatosinski 550cab4
github-actions: cleaned up generate-rst, added using conda
glatosinski 058f1d0
github-actions: added docutils to requirements.txt
glatosinski 3ecbc35
github-actions: added activating conda environment
glatosinski 43ea361
github-actions: fixed name for cell_readme_generate
glatosinski c594c49
github-actions: added ignoring failing commits in submodules
glatosinski 6a50169
github-actions: updated script names in generate-rst
glatosinski 4dbb833
github-actions: cleaned up generate-rst
glatosinski cc86239
github-actions: silenced git commands
glatosinski File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
name: Generate cells docs | ||
|
||
on: | ||
push: | ||
pull_request: | ||
|
||
jobs: | ||
|
||
generate-cells: | ||
runs-on: ubuntu-18.04 | ||
steps: | ||
|
||
- uses: actions/checkout@v2 | ||
with: | ||
fetch-depth: 0 | ||
|
||
- name: Generate cells READMEs | ||
run: | | ||
SUBMODULE_VERSION=latest make submodules -j3 || make submodules -j1 | ||
|
||
git submodule foreach 'git checkout master -q' | ||
|
||
make env | ||
source env/conda/bin/activate skywater-pdk-scripts | ||
which python | ||
python scripts/python-skywater-pdk/skywater_pdk/cells/list.py libraries/*/latest/ | ||
python scripts/python-skywater-pdk/skywater_pdk/cells/generate/readme.py libraries/*/latest/cells/* | ||
|
||
git submodule foreach 'git add .' | ||
export DAT=`date +"%Y-%m-%d %k:%M"` | ||
|
||
# TODO replace below with appropriate credentials | ||
git config --global user.email "action@github.com" | ||
git config --global user.name "Github Action" | ||
git submodule foreach 'git commit -q -m "$DAT" || true' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
193 changes: 193 additions & 0 deletions
193
scripts/python-skywater-pdk/skywater_pdk/cells/generate/readme.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,193 @@ | ||
#!/usr/bin/env python3 | ||
# -*- coding: utf-8 -*- | ||
# | ||
# Copyright 2020 The SkyWater PDK Authors. | ||
# | ||
# Use of this source code is governed by the Apache 2.0 | ||
# license that can be found in the LICENSE file or at | ||
# https://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
''' This is a prototype of cell documentation generation script. | ||
''' | ||
|
||
import csv | ||
import json | ||
import os | ||
import sys | ||
import argparse | ||
import pathlib | ||
import glob | ||
import subprocess | ||
import textwrap | ||
|
||
|
||
readme_template ="""\ | ||
{header} | ||
{headerUL} | ||
|
||
**{description}** | ||
|
||
*This is a stub of cell description file* | ||
|
||
- **Cell name**: {name} | ||
- **Type**: {deftype} | ||
- **Verilog name**: {verilog_name} | ||
- **Library**: {library} | ||
- **Inputs**: {inputs} | ||
- **Outputs**: {outputs} | ||
|
||
Symbols | ||
------- | ||
|
||
.. list-table:: | ||
|
||
* - .. figure:: {symbol1} | ||
- | ||
- .. figure:: {symbol2} | ||
|
||
Schematic | ||
--------- | ||
|
||
.. figure:: {schematic} | ||
:align: center | ||
|
||
GDSII Layouts | ||
------------- | ||
|
||
""" | ||
|
||
figure_template =""" | ||
|
||
.. figure:: {fig} | ||
:align: center | ||
:width: 50% | ||
|
||
{name} | ||
""" | ||
|
||
def write_readme(cellpath, define_data): | ||
''' Generates README for a given cell. | ||
|
||
Args: | ||
cellpath - path to a cell [str of pathlib.Path] | ||
define_data - cell data from json [dic] | ||
|
||
''' | ||
netlist_json = os.path.join(cellpath, define_data['file_prefix']+'.json') | ||
assert os.path.exists(netlist_json), netlist_json | ||
outpath = os.path.join(cellpath, 'README.rst') | ||
|
||
prefix = define_data['file_prefix'] | ||
header = prefix | ||
symbol1 = prefix + '.symbol.svg' | ||
symbol2 = prefix + '.pp.symbol.svg' | ||
schematic = prefix + '.schematic.svg' | ||
inputs = [] | ||
outputs = [] | ||
for p in define_data['ports']: | ||
try: | ||
if p[0]=='signal' and p[2]=='input': | ||
inputs.append(p[1]) | ||
if p[0]=='signal' and p[2]=='output': | ||
outputs.append(p[1]) | ||
except: | ||
pass | ||
gdssvg = [] | ||
svglist = list(pathlib.Path(cellpath).glob('*.svg')) | ||
for s in svglist: | ||
gdsfile = pathlib.Path(os.path.join(cellpath, s.stem +'.gds')) | ||
if gdsfile.is_file(): | ||
gdssvg.append(s) | ||
|
||
with open(outpath, 'w') as f: | ||
f.write (readme_template.format ( | ||
header = header, | ||
headerUL = '=' * len(header), | ||
description = define_data['description'].rstrip('.'), | ||
name = ':cell:`' + prefix +'`', | ||
deftype = define_data['type'], | ||
verilog_name = define_data['verilog_name'], | ||
library = define_data['library'], | ||
inputs = f'{len(inputs)} (' + ', '.join(inputs) + ')', | ||
outputs = f'{len(outputs)} (' + ', '.join(outputs) + ')', | ||
symbol1 = symbol1, | ||
symbol2 = symbol2, | ||
schematic = schematic, | ||
)) | ||
for gs in sorted(gdssvg): | ||
f.write (figure_template.format ( | ||
fig = gs.name, | ||
name = gs.stem | ||
)) | ||
|
||
def process(cellpath): | ||
''' Processes cell indicated by path. | ||
Opens cell definiton and calls further processing | ||
|
||
Args: | ||
cellpath - path to a cell [str of pathlib.Path] | ||
''' | ||
|
||
print() | ||
print(cellpath) | ||
define_json = os.path.join(cellpath, 'definition.json') | ||
if not os.path.exists(define_json): | ||
print("No definition.json in", cellpath) | ||
assert os.path.exists(define_json), define_json | ||
define_data = json.load(open(define_json)) | ||
|
||
if define_data['type'] == 'cell': | ||
write_readme(cellpath, define_data) | ||
|
||
return | ||
|
||
|
||
def main(): | ||
''' Generates README.rst for cell.''' | ||
|
||
prereq_txt = '' | ||
output_txt = 'output:\n generates README.rst' | ||
allcellpath = '../../../libraries/*/latest/cells/*' | ||
|
||
parser = argparse.ArgumentParser( | ||
description = main.__doc__, | ||
epilog = prereq_txt +'\n\n'+ output_txt, | ||
formatter_class=argparse.RawDescriptionHelpFormatter) | ||
parser.add_argument( | ||
"--all_libs", | ||
help="process all cells in "+allcellpath, | ||
action="store_true") | ||
parser.add_argument( | ||
"cell_dir", | ||
help="path to the cell directory", | ||
type=pathlib.Path, | ||
nargs="*") | ||
|
||
args = parser.parse_args() | ||
|
||
if args.all_libs: | ||
path = pathlib.Path(allcellpath).expanduser() | ||
parts = path.parts[1:] if path.is_absolute() else path.parts | ||
paths = pathlib.Path(path.root).glob(str(pathlib.Path("").joinpath(*parts))) | ||
args.cell_dir = list(paths) | ||
|
||
cell_dirs = [d.resolve() for d in args.cell_dir if d.is_dir()] | ||
|
||
errors = 0 | ||
for d in cell_dirs: | ||
try: | ||
process(d) | ||
except KeyboardInterrupt: | ||
sys.exit(1) | ||
except (AssertionError, FileNotFoundError, ChildProcessError) as ex: | ||
print (f'Error: {type(ex).__name__}') | ||
print (f'{ex.args}') | ||
errors +=1 | ||
print (f'\n{len(cell_dirs)} files processed, {errors} errors.') | ||
return 0 if errors else 1 | ||
|
||
if __name__ == "__main__": | ||
sys.exit(main()) | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
git submodule update --init --recursive
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
git submodule update --init --recursive
will fetch all the submodules and we need only thelatest
ones