iPOP-UP Configuration
All nf-core pipelines have been successfully configured for use on the iPOP-UP cluster.
To use, run the pipeline with -profile ipop_up. This will download and launch the ipop_up.config which has been pre-configured with a setup suitable for the iPOP-UP cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
Request an account
You will need an account to use iPOP-UP HPC cluster in order to run the pipeline, please refer to https://parisepigenetics.github.io/bibs/cluster/ipopup/.
Running the workflow on the iPOP-UP cluster
Guidelines to start nf-core workflows are provided at https://parisepigenetics.github.io/bibs/edctools/workflows/nf-cores.
In brief, Nextflow is installed on the iPOP-UP cluster and you need to activate it like this:
module load nextflowNextflow manages each process as a separate job that is submitted to the cluster by using the sbatch command. Nextflow shouldn’t run directly on the submission node but on a compute node. We recommand using a sbatch script:
nfcore-atac.sh
#!/bin/bash
#SBATCH --partition=ipop-up
#SBATCH --mem=4G
module purge
export JAVA_LD_LIBRARY_PATH=/shared/software/conda/envs/nextflow-21.04.0/lib/server
export JAVA_HOME=/shared/software/conda/envs/nextflow-21.04.0
module load nextflow/21.04.0
nextflow run nf-core/atacseq -profile ipop_up -params-file nf-params.jsonLaunch on the cluster with sbatch:
sbatch nfcore-atac.shTest dataset
nf-core provides some test for each workflow:
#!/bin/bash
#SBATCH --partition=ipop-up
#SBATCH --mem=4G
module purge
export JAVA_LD_LIBRARY_PATH=/shared/software/conda/envs/nextflow-21.04.0/lib/server
export JAVA_HOME=/shared/software/conda/envs/nextflow-21.04.0
module load nextflow/21.04.0
nextflow run nf-core/atacseq -profile ipop_up,testDatabanks
A local copy of several genomes are available in /shared/banks/ directory.
Config file
//Profile config names for nf-core/configs
params {
config_profile_description = 'iPOP-UP cluster profile, Université paris Cité'
config_profile_contact = 'Magali Hennion, bibs@parisepigenetics.com'
config_profile_url = 'https://discourse.rpbs.univ-paris-diderot.fr/c/ipop-up'
}
singularity {
// need one image per execution
enabled = true
runOptions = '-B /shared'
}
process {
resourceLimits = [
memory: 100.GB,
cpus: 28,
time: 96.h
]
executor = 'slurm'
queue = 'ipop-up'
}
params {
igenomes_ignore = true
// Max resources requested by a normal node on iPOP-UP cluster.
max_memory = 100.GB
max_cpus = 28
max_time = 96.h
}
executor
contact
