The pipeline randomly crashes with java.nio.file.NoSuchFileException: XXX_empty.pdf

[mfoll]

The issue seems to be related to an NFS copy error on our IARC cluster. Sometimes an empty pdf file is not correctly copied at the end of R_regression process, but as nextflow is tracking it, it crashes later on when it tries to delete it.

It's not a bug in our pipeline neither in nextflow.

Nextflow is not checking at the end of a process if files are correctly copied. The last line of the R_regression file .command.run.1 is:

cp -fR *.pdf /path/to/my/folder/BAM/VCF || true

I do see the XXX_empty.pdf in the corresponding nextflow workfolder but it has not been copied into the /path/to/my/folder/BAM/VCF folder, even though the process ended with no error.

[mfoll]

A work around is to replace:

PDF.flatten().filter { it.size() == 0 }.subscribe { it.delete() }

with this bit in the collect_vcf_resultprocess:

empty_files = file(params.out_folder+'/VCF/*_empty.pdf')
    for( def file : empty_files ) {
            file.delete()
    }

Doing so we just delete the files effectively copied.