This release contains some template changes and bug fixes.
Highlights
We refactored the pipelines template a bit:
- The
lib
directory is removed: 👋 Groovy code - Instead we now use nf-core subworkflows for pipeline initialisation
- Some nf-core pipelines adopted this already: nf-core/fetchngs and nf-core/rnaseq
- The
nf-validation
plugin is now used to create an input channel from a sample sheet file.
If you haven’t started merging in the template updates yet, it may almost be easier to ignore the template updates for now and try and remove the lib/
directory on the pipeline dev
branch by:
- Individually installing the
utils_*
subworkflows with the following commands:
- Creating a local
utils_*
subworkflow for the pipeline. You can copy the one in rnaseq (dev
) or the pipeline template and customise this to your requirements. Make sure you move any custom functions inlib/
directory to this file. - Include the
utils_*
subworkflows in the main.nf as done in rnaseq (dev
). - Including the
utils_*
subworkflows in the workflow main.nf as done in rnaseq (dev
). - Delete the
lib/
directory after you have confirmed everything has been ported across. - Once you have merged this to
dev
the template sync PR will be updated and tell you whether you have missed anything. - The
nf-core lint
command might complain about having to recompute checksum of subworkflow(s).- Be sure to check in the
modules.json
file that non of the previously installed subworkflows have dissapeared from it. - A new
subworkflows
section with the newutils_*
subworkflows might have been added during the merged. Reinstalling the subworkflows usingnf-core subworkflows install
should fix this, otherwise one could manually edit the modules.json file (only recommended for advanced users).
- Be sure to check in the
💡 It helped to disable running the main workflow whilst wiring all of this in to speed up development.
🎥 Last but not least, we have a short bytesize talk showing the necessary steps:
Additional bug fixes and improvements
-
Have you seen a
no space left on device
error or your CI tests lately? This is now fixed by adding a clean up step in the github actions. -
New api-docs on the nf-core website to get clarifications on linting errors etc.
-
Updating a module won’t remove custom nextflow.config files anymore.
-
The commands
nf-core modules test
andnf-core subworkflows test
now have a new parameter--profile
to specify the container to run nf-test with.
Thanks to all the contributors who made this release possible, especially to Harshil Patel and his team for kicking off the lib
restructuring and the utils_*
subworkflows.
You can find the whole changelog on GitHub.