Skip to content

Selecting profiles

What are profiles?

Note that some form of configuration will be needed so that Nextflow knows how to fetch the required software. This is usually done in the form of a config profile (YOURPROFILE in the example command above). You can chain multiple config profiles in a comma-separated string. They are a set of "sensible defaults" for the resource requirements of each of the steps in the workflow, that can be enabled with the command line flag -profile. You can learn more about nextflow profiles at:

Available profiles

The pipeline have "standard profiles" set to run the workflows with either docker or singularity using the local executor, which is nextflow's default executor and basically runs the pipeline processes in the computer where Nextflow is launched.

If you need to run the pipeline using another executor such as sge, lsf, slurm, etc. you can take a look at nextflow's manual page to proper configure one in a new custom profile and take advantage that nextflow allows multiple profiles to be used at once, e.g. -profile docker,slurm.

Note

If -profile is not specified, the pipeline will run locally and expect all software to be installed and available on the PATH. This is not recommended and will most likely fail.

Note on Singularity

If you are using singularity and are persistently observing issues downloading Singularity images directly due to timeout or network issues, try downloading it first. Setting the NXF_SINGULARITY_CACHEDIR or singularity.cacheDir Nextflow options enables you to store and re-use the images from a central location for future pipeline runs.

This is exemplified in the installation page

# run
nextflow run deng-lab/viroprofiler -profile singularity [OPTIONS]

Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Shifter, Charliecloud, Conda) - see below. When using Biocontainers, most of these software packaging methods pull Docker containers from quay.io e.g FastQC except for Singularity which directly downloads Singularity images via https hosted by the Galaxy project and Conda which downloads and installs software locally from Bioconda.

A container engine is required. There is a conda profile, but it is not a working alternative: the pipeline's own processes carry no conda directive, so a Conda run would find most of its tools missing. Use Docker, Singularity/Apptainer, Podman, Shifter or Charliecloud.

nf-core's institutional configs are not loaded: the includeConfig that would fetch them from nf-core/configs is commented out in nextflow.config. Point at your own site config with -c instead — and note that a -c file setting params.max_cpus and friends arrives after the resource ceiling has been fixed, so set process.resourceLimits there rather than params.max_*.

Note that multiple profiles can be loaded, for example: -profile test,docker - the order of arguments is important! They are loaded in sequence, so later profiles can overwrite earlier profiles.

If -profile is not specified, the pipeline will run locally and expect all software to be installed and available on the PATH. This is not recommended.

  • docker
  • A generic configuration profile to be used with Docker
  • singularity
  • A generic configuration profile to be used with Singularity
  • podman
  • A generic configuration profile to be used with Podman
  • shifter
  • A generic configuration profile to be used with Shifter
  • charliecloud
  • A generic configuration profile to be used with Charliecloud
  • conda
  • A generic configuration profile to be used with Conda. Please only use Conda as a last resort i.e. when it's not possible to run the pipeline with Docker, Singularity, Podman, Shifter or Charliecloud.
  • test
  • A profile with a complete configuration for automated testing
  • Includes links to test data so needs no other parameters
  • test_stub
  • A lightweight profile for pipeline topology validation using Nextflow's -stub mode
  • Runs all processes with stub blocks (no real computation or databases required)
  • Used for CI testing on GitHub Actions