--- title: "Fast DP: Fast Data Processing with XDS" url: "https://help.dectris.cloud/en/article/fast-dp-fast-data-processing-with-xds" language: "en" last_updated: "2026-07-16T01:09:29.593Z" --- # Fast DP: Fast Data Processing with XDS ## **Introduction** The [**Fast DP**](https://app.dectris.cloud/sections/library/environments/public?id=019ad734-5587-72b3-bd0a-8e012abb1416) environment is a lightweight, command-line–focused workspace for macromolecular crystallography (MX) data processing and rapid downstream analysis. It is designed for users who want to run `**fast_dp**` and related tooling in a reproducible container setup, while still having access to commonly used MX utilities for inspection, plotting, and scripting. The environment is built on **Ubuntu 24.04** and combines: - **Fast DP** (via conda) for fast automated processing and summary output, - **CCTBX** components (via `cctbx-base`) for crystallographic functionality used by fast\_dp and related tools, - **XDS** tools for integration workflows and supporting utilities, - **CCP4** command-line tools (GUI components removed) for standard MX utilities, - **PyMOL** for structure visualization, - common Python libraries for scripting and plotting (numpy, matplotlib, pandas, h5py, hdf5plugin). This environment is primarily intended for **scripted and automated workflows**, in particular **DECTRIS D.LAB job templates**. - ![](https://cdn-assets.productfruits.com/8cc6d31e-0e66-495a-9ef9-77fc06bb5f3e) ## **Environment configuration** ### **Installation directories** Key software components are installed in fixed locations inside the container: - **Conda (Miniconda)**: `/opt/conda/` - **fast\_dp conda environment**: `/opt/conda/envs/fastdp/` This conda environment contains: - `fast_dp` - **CCTBX components** (via `cctbx-base`) - Python (3.11) - Scientific Python stack (numpy, matplotlib, pandas, h5py, hdf5plugin) - **CCTBX (via conda)**: Installed **inside the** `**fastdp**` **conda environment** at: `/opt/conda/envs/fastdp/` CCTBX libraries, binaries, and Python modules are available automatically when the `fastdp` environment is on `PATH`. No separate CCTBX installation directory is used. - **XDS**: `/opt/xds/` Includes: - `xds`, `xds_par` - helper scripts such as `generate_XDS.INP` - HDF5 plugins: - `/opt/xds/dectris-neggia.so` - `/opt/xds/durin-plugin.so` - **CCP4 (command-line)**: `/opt/ccp4/` CCP4 is installed without GUI components and is intended for command-line use and scripting. ### **PATH and CCP4 setup** Interactive shells are configured to automatically: - prepend the following directories to `PATH`: - `/opt/conda/envs/fastdp/bin` - `/opt/conda/bin` - `/opt/xds` - source the CCP4 environment: - `. /opt/ccp4/bin/ccp4.setup-sh` - This configuration is applied via: - `/etc/profile.d/fastdp-path.sh` - `/etc/bash.bashrc` As a result, tools such as `fast_dp`, CCTBX utilities, XDS programs, and CCP4 binaries are available immediately in interactive terminals. ### **Usage Examples** #### **Running fast\_dp command** Due to the configuration of the PATH variable, the `fast_dp` command can be immediately run with a path to a dataset inside a terminal: ```plaintext fast_dp /path/to/data ``` ![](https://cdn-assets.productfruits.com/3e101010-6e2e-4825-9271-676d6a906a89) For HDF5 / NeXus input (for example EIGER detector data), one can specify a path to the DECTRIS HDF5 reader plugin: ```plaintext fast_dp -l /opt/xds/dectris-neggia.so /path/to/data.h5 ``` ![](https://cdn-assets.productfruits.com/5d999ed6-e73e-4f40-98d1-8fa28ad037dc) [Additional fast\_dp options](https://pypi.org/project/fast-dp/) may also be specified explicitly, for example to constrain space group or resolution: ```plaintext fast_dp \ -s P212121 \ -r 2.0 \ -R 50.0 \ /path/to/data ``` ![](https://cdn-assets.productfruits.com/2aa4044b-f01f-4b70-a059-2f872d314535) #### **Using CCTBX for inspection and scripting** You can access CCTBX functionality from Python, for example to inspect reflection data in an MTZ file produced by `fast_dp`: ```plaintext python3 from iotbx import reflection_file_reader mtz = reflection_file_reader.any_reflection_file("fast_dp.mtz") arrays = mtz.as_miller_arrays() for arr in arrays: print(arr.info().label_string()) ``` ![](https://cdn-assets.productfruits.com/e055758b-6255-4a8a-9079-e45ac3273b50) In the example above, the column labels contained in the MTZ file are printed. This pattern can be extended in Python scripts or job templates to compute statistics, inspect resolution limits, or prepare input files for downstream processing. ### **Configuring job templates** Job templates run in non-interactive shells and should therefore configure their environment explicitly. At the start of your script: ```plaintext # Load CCP4 command-line environment source /opt/ccp4/bin/ccp4.setup-sh # Make fast_dp, CCTBX, and XDS tools available export PATH="/opt/conda/envs/fastdp/bin:/opt/conda/bin:/opt/xds:$PATH" ``` This ensures that: - `fast_dp` and the associated CCTBX Python modules are available, - XDS command-line tools (e.g. `xds`, `xds_par`) are on `PATH`, - CCP4 programs (e.g. `ctruncate`, `pointless`) can be invoked directly. ## **Public Job Templates** ## **Version History** ### **Version 1** - Changes compared to Version 0: - Added `libquadmath0` to the runtime image to ensure compatibility with XDS-based workflows. - No changes were made to the installed software versions. ### **Version 0** - Initial release of the Fast DP command-line environment, including: - **XDS**: VERSION Jan 19, 2025 (BUILT=20250714) - **CCP4**: software suite version **9.0.011** (patch level 9.0.011) - **CCTBX**: version **2024.2** (via `cctbx-base` conda package) - **PyMOL**: version **3.0.0** ## **References** ### **XDS** 1. XDS program package: [**https://xds.mr.mpg.de/**](https://xds.mr.mpg.de/) 2. XDS references: [**https://xds.mr.mpg.de/html\_doc/references.html**](https://xds.mr.mpg.de/html_doc/references.html) 3. XDS wiki: [**https://wiki.uni-konstanz.de/xds/index.php/Main\_Page**](https://wiki.uni-konstanz.de/xds/index.php/Main_Page) ### **CCP4** 1. CCP4 page: [https://www.ccp4.ac.uk/](https://www.ccp4.ac.uk/) 2. CCP4 wiki: [https://wiki.uni-konstanz.de/ccp4/index.php/Main\_Page](https://wiki.uni-konstanz.de/ccp4/index.php/Main_Page) 3. General CCP4 program reference: J. Agirre et al. *Acta. Cryst.* **D79**, 449-461 (2023) “The CCP4 suite: integrative software for macromolecular crystallography” [\[doi:10.1107/S2059798323003595\]](https://doi.org/10.1107/S2059798323003595) 4. Specific CCP4 program reference info: [https://www.ccp4.ac.uk/ccp4-v7-0-program-references/](https://www.ccp4.ac.uk/ccp4-v7-0-program-references/) ### **PyMOL** 1. PyMOL page: [https://pymol.org/](https://pymol.org/) 2. PyMOL wiki: [https://pymolwiki.org/index.php/Main\_Page](https://pymolwiki.org/index.php/Main_Page) 3. PyMOL citation info (bottom of page): [https://www.pymol.org/support.html?](https://www.pymol.org/support.html?) ### **CCTBX** 1. CCTBX socumentation: [https://cci.lbl.gov/docs/cctbx/](https://cci.lbl.gov/docs/cctbx/)  2. CCTBX github: [https://github.com/cctbx/cctbx\_project](https://github.com/cctbx/cctbx_project) 3. CCTBX reference: Grosse-Kunstleve, R.W., Sauter, N.K., Moriarty, N.W. and Adams, P.D., 2002. The Computational Crystallography Toolbox: crystallographic algorithms in a reusable software framework. *Applied Crystallography*, *35*(1), pp.126-136. [**https://doi.org/10.1107/S0021889801017824**](https://doi.org/10.1107/S0021889801017824) ### **Fast DP** 1. Fast DP github**:** [**https://github.com/DiamondLightSource/fast\_dp**](https://github.com/DiamondLightSource/fast_dp)  2. Fast DP reference**:** [Winter, G. & McAuley, K. E. "Automated data collection for macromolecular crystallography." Methods 55, 81-93 (2011).](https://doi.org/10.1016/j.ymeth.2011.06.010)