Update to version 1.1.0 #229

mumichae · 2021-07-15T16:17:10Z

Foolproof snakemake (Foolproof snakemake #216)
Speedup CI (Speedup CI #218)
Html table cutoff (Html table cutoff #217)
Separate modules (Separate modules #209)
Update docs (Update docs #228)

* remove unnecessary DNA_ID for export counts * updated bumversion config * Bump version: 1.0.3 → 1.0.4

* outrider saving results simplified * fraser saving results simplified * fraser saving results simplified * paths in header Co-authored-by: Vicente <yepez@in.tum.de> Co-authored-by: Michaela Mueller <51025211+mumichae@users.noreply.github.com>

Better error handling if the GATK ASECounter fails to count the reads.

Fix dependency graph fails by ignoring print stateme

Foolproof snakemake

* using micromamba Github action * renamed micromamba step * micromamba run working locally * add missing changes to github actions * include genome in test installation

* hard cutoff of 1000 entries for results viewing * change formatting for MAE results * change MAE format * simplified html truncation syntax * Update Summary.R * Update Summary.R * Update Results.R * Update Summary.R * update caption message * typo fixed Co-authored-by: Michaela Mueller <mumichae@in.tum.de> Co-authored-by: Michaela Mueller <51025211+mumichae@users.noreply.github.com>

* Turn off analysis for Analysis scripts html based on config * Renamed sample annotation variable and reformatted files * pass working directory to DropConfig and Submodule objects * Revert "Merge branch 'dep_graph_fix' into separate_modules" This reverts commit 8a3c1aa, reversing changes made to 0c8ae7f. * add run param to AS and MAE overviews * update library calls for Overview.R * create dummy files via sampleParams * AberrantSplicing Overview.R is defined by function * AberrantSplicing should be based on function. but not being recognized * fuctions not working * fixed html output indent * fixed typo and order of snakemake includes * remove duplicated (copy paste error) line * fix MAE typos * move preprocessing used in AE and AS to Pipeline and save to processedDataDir/preprocess * change Scripts/module names based on cfg.run. Also update cli to remove hidden files. * update conftest * update error message and tests * update cli to better handle hidden module paths * re-copy Scripts path * fix dummy files to be module specific, update tests * remove self from pipeline_test * remove test typo * changes based on code review * renamed getSnakefile * hard cutoff of 1000 entries for results viewing * Cleaned up HTML scripts with generic functions * change formatting for MAE results * update test to include working path * cleaned up Monoallelic HTML script * update some docs, add warning message Co-authored-by: Michaela Mueller <mumichae@in.tum.de> Co-authored-by: Smith Nicholas <smith@in.tum.de>

* update docs * table syntax * Update prepare.rst * update docs * Update pipeline.rst

* skip checks if run:false * add comments

nickhsmith

Seems good to me. Should be a useful bump

drop/template/Scripts/html_functions.R

c-mertes · 2021-07-21T06:35:47Z

drop/modules/mae-pipeline/Snakefile

-    index_name=MAE_index_name
-)
+MAE_WORKDIR = cfg.MAE.renameLocalDir()
+MAE_index_input,MAE_graph_file,MAE_index_output = cfg.MAE.getModuleIndexFiles("mae-pipeline",MAE_WORKDIR)

 rule mae:


I think this could be moved up to the all rule by having there if-else statements of what to include. This could ease a lot of code.

tests/pipeline/test_AE.py

tests/pipeline/test_AS.py

tests/pipeline/test_MAE.py

tests/pipeline/test_pipeline.py

c-mertes · 2021-07-21T06:57:24Z

tests/pipeline/test_pipeline.py

+    # change all "run: true" config values to "run: false" and save result into new config file
+    # runthe pipeline with each module turned off
+    run("sed 's/run: true/run: false/g' config.yaml > config_norun.yaml  ",demo_dir)
+    pipeline_run = run(["snakemake",  f"-j{CORES}", "--configfile", "config_norun.yaml"], demo_dir)


Does this test make sense here? Everything should be done anyway. So we should rather turn it around and test first if I can switch on a module and then run it.

Same goes for the other parts.

* Update to version 1.1.0 (#229) Major changes included in this update is: * adapt to new snakemake version (#208) * Speedup CI (use micromamba, #218) * Add HTML cutoff for results (#217) * Separate modules (#209) * Update documentation (#228) On top of this many small bug-fixes are included: * #175 #192 #198 #231 * Simplify result reporting Co-authored-by: Vicente Yepez <30469316+vyepez88@users.noreply.github.com> Co-authored-by: Vicente <yepez@in.tum.de> Co-authored-by: Michaela Mueller <51025211+mumichae@users.noreply.github.com> Co-authored-by: Michaela Mueller <mumichae@in.tum.de> Co-authored-by: Smith Nicholas <smith@in.tum.de> Co-authored-by: nickhsmith <smithnickh@gmail.com> Co-authored-by: Christian Mertes <mertes@in.tum.de> * add splitting to MAE filterSNVs.sh * fix broken link and maxVarFreqCohort * fix broken link and maxVarFreqCohort * Update test_MAE.py * code review fixes Co-authored-by: Michaela Mueller <51025211+mumichae@users.noreply.github.com> Co-authored-by: Vicente Yepez <30469316+vyepez88@users.noreply.github.com> Co-authored-by: Vicente <yepez@in.tum.de> Co-authored-by: Michaela Mueller <mumichae@in.tum.de> Co-authored-by: Smith Nicholas <smith@in.tum.de> Co-authored-by: Christian Mertes <mertes@in.tum.de>

mumichae and others added 21 commits May 12, 2021 05:18

Small fixes (#192)

091396e

* remove unnecessary DNA_ID for export counts * updated bumversion config * Bump version: 1.0.3 → 1.0.4

Merge branch 'master' of https://github.com/gagneurlab/drop into dev

2a00c3e

Fix out res (#198)

790f73c

* outrider saving results simplified * fraser saving results simplified * fraser saving results simplified * paths in header Co-authored-by: Vicente <yepez@in.tum.de> Co-authored-by: Michaela Mueller <51025211+mumichae@users.noreply.github.com>

grep any output before graph output

719c51d

Add error messages to MAE shell scripts (#175)

b0e5c79

Better error handling if the GATK ASECounter fails to count the reads.

Merge branch 'dev' into dep_graph_fix

5fd5ff8

match end of graph output as well, using escaped curly bracket

d8842c6

Merge pull request #200 from gagneurlab/dep_graph_fix

3b18830

Fix dependency graph fails by ignoring print stateme

Bump version: 1.0.4 → 1.0.5

2744288

Merge branch 'master' of https://github.com/gagneurlab/drop into dev

88e9ab6

updated tmp dir for download

c05d146

specified cores for every snakemake call with --cores

bc01cff

fixed subprocess call

f9c94ce

changed yield_fixture to fixture

ec16bc3

Merge pull request #216 from gagneurlab/foolproof_snakemake

2bf56c3

Foolproof snakemake

Speedup CI (#218)

03e2c66

* using micromamba Github action * renamed micromamba step * micromamba run working locally * add missing changes to github actions * include genome in test installation

Update docs (#228)

5d46131

* update docs * table syntax * Update prepare.rst * update docs * Update pipeline.rst

Bump version: 1.0.5 → 1.1.0

3781db6

skip checks if run:false (#231)

ad6f744

* skip checks if run:false * add comments

mumichae requested review from vyepez88 and nickhsmith July 15, 2021 18:45

nickhsmith previously approved these changes Jul 16, 2021

View reviewed changes

vyepez88 reviewed Jul 16, 2021

View reviewed changes

drop/template/Scripts/html_functions.R Outdated Show resolved Hide resolved

updated html script description

58bf07a

mumichae dismissed nickhsmith’s stale review via 58bf07a July 19, 2021 12:52

mumichae requested review from nickhsmith and vyepez88 July 19, 2021 14:57

c-mertes requested changes Jul 21, 2021

View reviewed changes

c-mertes added 4 commits July 21, 2021 08:59

Update tests/pipeline/test_AE.py

5288de4

Update tests/pipeline/test_AS.py

3e85ca6

Update tests/pipeline/test_pipeline.py

cd98bcd

Update tests/pipeline/test_MAE.py

572beb4

c-mertes previously approved these changes Jul 21, 2021

View reviewed changes

Update tests/pipeline/test_MAE.py

2b4bf5d

c-mertes dismissed their stale review via 2b4bf5d July 21, 2021 07:06

c-mertes requested review from c-mertes and removed request for nickhsmith and vyepez88 July 21, 2021 07:08

c-mertes approved these changes Jul 21, 2021

View reviewed changes

c-mertes merged commit bf2eb91 into master Jul 21, 2021

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to version 1.1.0 #229

Update to version 1.1.0 #229

mumichae commented Jul 15, 2021

nickhsmith left a comment

c-mertes Jul 21, 2021

c-mertes Jul 21, 2021

c-mertes Jul 21, 2021

Update to version 1.1.0 #229

Update to version 1.1.0 #229

Conversation

mumichae commented Jul 15, 2021

nickhsmith left a comment

Choose a reason for hiding this comment

c-mertes Jul 21, 2021

Choose a reason for hiding this comment

c-mertes Jul 21, 2021

Choose a reason for hiding this comment

c-mertes Jul 21, 2021

Choose a reason for hiding this comment