Quick Start

This path builds and validates BFF metadata and shows the normal raw-VCF workflow, the explicit shortcut for compatible pre-annotated VCFs, and SNP-array conversion.

Which Command Do I Need?

Starting data	Command	Result
XLSX metadata workbook	bff-tools validate	Validated BFF JSON collections
Existing BFF JSON	bff-tools validate	Validation report only
Raw or annotated VCF	bff-tools vcf	BFF genomicVariations
Supported SNP-array TSV/TXT	bff-tools tsv	BFF genomicVariations
External annotation bundle	bff-tools install-resources	Verified local annotation resources
Packaged annotated example	bff-tools demo	Validated BFF and standalone browser

Raw VCF and TSV input is annotated by default and requires the external annotation data. Only an already annotated VCF can use --no-annotate.

Configuration inputs

The optional parameter file (-p) stores run choices such as genome, datasetid, and bff2html. The installed package already contains the standard annotation-resource layout; BFF_TOOLS_DATA selects its external root. Use -c only for a different layout or site-specific executable paths.

Research use

The toolkit prepares research data and annotations. It is not a medical device and its output must not be used by itself for clinical or medical decisions. See the full disclaimer.

1. Check the Command

bff-tools --version
bff-tools --help
bff-tools doctor

The primary user commands are validate, vcf, tsv, and install-resources. The doctor command checks what the current installation can run without executing a pipeline. Before external data is configured, its expected final status is CORE READY; the demo command is a resource-free first run, while test is a developer integration check.

2. Run the Packaged Demo

Confirm the installed converter, validator, and browser without downloading the annotation bundle:

bff-tools demo

This converts a packaged, fully annotated GRCh37 VCF fixture and writes bff-tools-demo/vcf/genomicVariationsVcf.json.gz, a standalone browser, and a short README.txt. It does not run SnpEff, dbNSFP, ClinVar, or COSMIC annotation; raw VCFs still require the external resources. Pass --output-dir DIR to choose another new directory or --no-browser to skip HTML generation.

3. Create and Validate Metadata JSON

Export the packaged Beacon workbook template:

bff-tools validate --template-out metadata.xlsx

After filling the workbook, convert each populated worksheet into a BFF JSON collection and validate its records:

bff-tools validate -i metadata.xlsx -o bff

The output directory contains collections such as individuals.json, biosamples.json, analyses.json, and datasets.json. Each collection is written only when its rows pass the corresponding Beacon v2 schema, unless --ignore-validation is explicitly used.

Existing JSON follows a validation-only path and is not rewritten:

bff-tools validate -i bff/individuals.json bff/biosamples.json

4. Convert and Annotate Variants

For most raw VCFs, first prepare the annotation data, then run:

export BFF_TOOLS_DATA=/absolute/path/to/beacon2-cbi-tools-data
bff-tools install-resources
bff-tools doctor --genome hg38
bff-tools vcf \
  -i cohort.vcf.gz \
  --genome hg38 \
  --dataset-id cohort-1 \
  --annotate \
  -o cohort-bff

For a repeatable run, put the same choices in an optional parameter file:

cohort.yaml

genome: hg38
datasetid: cohort-1
projectdir: cohort-bff
annotate: true
bff2html: true

Then keep the command shorter:

bff-tools vcf -i cohort.vcf.gz -p cohort.yaml

CLI options override values from cohort.yaml. See Configuration for all accepted keys and defaults.

If the VCF already contains a compatible SnpEff ANN header and annotations, disable re-annotation explicitly. dbNSFP and ClinVar fields are still strongly recommended for complete output:

bff-tools vcf -i cohort.annotated.vcf.gz \
  --genome hg38 --dataset-id cohort-1 --no-annotate -o cohort-bff

The primary output is:

cohort-bff/vcf/genomicVariationsVcf.json.gz

Add --browser to generate a standalone HTML report alongside the BFF output:

bff-tools vcf -i cohort.vcf.gz \
  --genome hg38 --dataset-id cohort-1 \
  --browser -o cohort-bff-browser

5. Convert SNP-Array Data

TSV/TXT conversion needs a sample identifier, the matching reference assembly, and the annotation bundle selected above:

bff-tools tsv \
  -i genotypes.txt.gz \
  --sample-id sample-1 \
  --genome hg19 \
  --dataset-id cohort-1 \
  -o sample-1-bff

The command creates a VCF intermediate, annotates it, and converts it through the same production VCF-to-BFF path. --no-annotate is not accepted for TSV input.

6. Verify Annotation and Output

Annotation-enabled runs retain normalized and annotated VCF intermediates. Inspect representative ANN, dbNSFP, ClinVar, and COSMIC fields and record every database version with the run.

bff-tools validate -i cohort-bff/vcf/genomicVariationsVcf.json.gz --gv-vcf

See Configuration for the profile, Annotation Data for the complete setup and integration test, and Outputs for the resulting directory layout.

For a connected metadata-and-variants workflow, continue with the end-to-end tutorial. The GRCh38 worked example shows how the included 1000 Genomes subset was prepared, and the FAQ retains common errors and fixes.