Skip to main content

Containerized Installation with Docker

Docker is the recommended runtime for workstations and servers. The published image contains bff-tools, Python, Java, bcftools, SnpEff, and SnpSift. Large reference and annotation databases remain outside the image and are mounted at runtime.

The image does not install MongoDB, mongosh, or MongoDB Database Tools. bff-tools produces BFF files; database loading is an optional downstream step.

Requirements​

  • Linux on amd64 or arm64;
  • a working Docker Engine with permission to run containers;
  • at least 4 GB RAM for validation and more for Java annotation;
  • at least 200 GB for the annotation bundle, plus working space for intermediate VCFs and BFF output.

1. Pull the Published Image​

Use a numbered tag for reproducible work. latest follows the newest published release:

docker pull manuelrueda/beacon2-cbi-tools:latest
docker run --rm manuelrueda/beacon2-cbi-tools:latest --version

2. Validate Metadata​

Mount the working directory at /work. Supplying the host user and group avoids root-owned output files:

docker run --rm \
  --user "$(id -u):$(id -g)" \
  -v "$PWD:/work" \
  manuelrueda/beacon2-cbi-tools:latest \
  validate -i /work/metadata.xlsx -o /work/bff

Export a fresh workbook template in the same way:

docker run --rm \
  --user "$(id -u):$(id -g)" \
  -v "$PWD:/work" \
  manuelrueda/beacon2-cbi-tools:latest \
  validate --template-out /work/metadata.xlsx

3. Prepare Annotation Data​

Raw VCF and SNP-array input requires the external annotation bundle. Create a persistent host directory and run the installer from the image:

export BFF_TOOLS_DATA=/absolute/path/to/beacon2-cbi-tools-data
mkdir -p "$BFF_TOOLS_DATA"
docker run --rm \
  --user "$(id -u):$(id -g)" \
  -v "$BFF_TOOLS_DATA:/bundle" \
  manuelrueda/beacon2-cbi-tools:latest \
  install-resources --data-dir /bundle

Use install-resources --print-links when Google Drive requires manual download. Full storage and recovery guidance is in Annotation Data.

The image defaults BFF_TOOLS_DATA to /beacon2-cbi-tools-data. The host directory and container path do not have to match; the environment value must name the path visible inside the container.

4. Annotate and Convert a Raw VCF​

Place cohort.vcf.gz in the working directory:

docker run --rm \
  --user "$(id -u):$(id -g)" \
  -v "$PWD:/work" \
  -v "$BFF_TOOLS_DATA:/beacon2-cbi-tools-data" \
  -e BFF_TOOLS_DATA=/beacon2-cbi-tools-data \
  manuelrueda/beacon2-cbi-tools:latest \
  vcf -i /work/cohort.vcf.gz \
  --genome hg38 \
  --dataset-id cohort-1 \
  -o /work/cohort-bff

Annotation is enabled by default. The command normalizes the VCF, applies SnpEff, dbNSFP, ClinVar, and COSMIC, then writes BFF genomic variations.

If tmpdir is inside the annotation-data mount, that mount must be writable. A stricter deployment can mount reference directories read-only and bind a separate writable temporary directory.

5. Convert an Already Annotated VCF​

Use --no-annotate only when the input has a compatible SnpEff ANN header and record annotations:

docker run --rm \
  --user "$(id -u):$(id -g)" \
  -v "$PWD:/work" \
  manuelrueda/beacon2-cbi-tools:latest \
  vcf -i /work/cohort.annotated.vcf.gz \
  --genome hg38 \
  --dataset-id cohort-1 \
  --no-annotate \
  -o /work/cohort-bff

dbNSFP and ClinVar fields are not mandatory for parsing, but they are strongly recommended for complete BFF identifiers, frequencies, predictions, and clinical interpretations.

6. Open an Interactive Shell​

The normal image entry point is bff-tools. Override it only when a shell is useful for inspecting mounts or logs:

docker run --rm -it \
  --user "$(id -u):$(id -g)" \
  --entrypoint bash \
  -v "$PWD:/work" \
  -v "$BFF_TOOLS_DATA:/beacon2-cbi-tools-data" \
  -e BFF_TOOLS_DATA=/beacon2-cbi-tools-data \
  manuelrueda/beacon2-cbi-tools:latest

One-shot docker run --rm commands are preferred for production because inputs, mounts, and image tags remain visible in job provenance.

7. Build Locally​

The Dockerfile builds the source in the current repository checkout. For a release build, check out the release tag and confirm that the worktree is clean before building:

git checkout v2.0.13
git status --short

The status command should produce no output.

The Dockerfile provides two targets. Pass the version and revision as image metadata; these arguments identify the source but do not select or download it:

version=$(python3 -c 'import runpy; print(runpy.run_path("src/bff_tools/version.py")["VERSION"])')

docker build --target core \
  --build-arg BFF_TOOLS_VERSION="$version" \
  --build-arg VCS_REF="$(git rev-parse HEAD)" \
  -t beacon2-cbi-tools:core -f docker/Dockerfile .

docker build --target runtime \
  --build-arg BFF_TOOLS_VERSION="$version" \
  --build-arg VCS_REF="$(git rev-parse HEAD)" \
  -t beacon2-cbi-tools:annotation -f docker/Dockerfile .

The core target supports metadata validation and --no-annotate VCF conversion. The default runtime target adds the annotation executables but still requires the external databases.

Inspect the recorded source revision with:

docker inspect beacon2-cbi-tools:annotation \
  --format '{{ index .Config.Labels "org.opencontainers.image.revision" }}'

The manual Docker GitHub Action follows the same source model: it checks out the release_tag entered when launching the workflow, verifies that it matches the Python package version, records that exact commit, and publishes both manuelrueda/beacon2-cbi-tools:<VERSION> and :latest. For a release, create and push the release tag first, then enter that tag when launching the action. Pull an immutable image digest when byte-for-byte identity with a published image is required.

Verification​

Smoke-test the installed command:

docker run --rm manuelrueda/beacon2-cbi-tools:latest validate --help
docker run --rm manuelrueda/beacon2-cbi-tools:latest vcf --help

After mounting the complete bundle, process a small representative VCF with the production configuration before starting a cohort-scale run. Image and bundle maintainers can additionally run the packaged compact integration test.

Troubleshooting​

A configured file does not exist​

The path is checked inside the container. Confirm the host bind source exists, BFF_TOOLS_DATA names the bind destination, and architecture-specific paths resolve to x86_64 or arm64.

SnpEff tries to use the network​

Verify that $BFF_TOOLS_DATA/databases/snpeff/v5.0 is present inside the container. bff-tools passes this path to SnpEff with -dataDir and disables network downloads; no edit to snpEff.config is required.

The output directory already exists​

Runs do not overwrite project directories. Select a new -o path or archive and move the previous result.

Docker cannot resolve package or image hosts​

This is a Docker daemon or host-networking problem rather than a bff-tools error. Correct proxy, DNS, certificate, or registry access and retry.

MongoDB​

There is no Compose stack in this repository. For an optional MongoDB deployment, install the clients separately and follow the MongoDB import guide.