This proof-of-concept middleware enables the use of Crypt4GH files as inputs for tasks that are run in TES implementations (e.g., funnel, TESK).
Currently, there are no implementations of TES that natively support the use of files encrypted with Crypt4GH. This middleware supports the use of Crypt4GH files by prepending the list of executors in a TES request with a decryption executor. This decryption executor decrypts the contents of any Crypt4GH files and places them in a volume so that subsequent executors can work on the decrypted contents.
The middleware alters the initial TES request such that a decryption executor and a new volume (/vol/crypt/) are added
to the request. Since the decryption executor places all input files in /vol/crypt/, all input paths in subsequent
executors are altered to /vol/crypt/{filename}.
The functionality of the decryption executor lies in decrypt.py. This script moves all input files to a
specified output directory (in this case, /vol/crypt/). If a Crypt4GH file is detected and the secret key used to
encrypt it is provided, the executor decrypts the contents of the Crypt4GH file and places it in /vol/crypt/.
Subsequent executors then refer to the files in /vol/crypt/, not their original locations.
You should not use this middleware in untrusted environments, as it requires transmission of secret keys and stores the decrypted contents of Crypt4GH files on disk. This middleware is meant to be used with a Trusted Execution Environment (TEE), which allows for the secure transmission and storage of data. Integration with TEEs is a work in progress.
pip install poetry
poetry installTests require a running TES instance and an S3 bucket containing the input files. Instances of Funnel
and MinIO were used in development. TES_URL in tests/tasks/test_tasks.py can be altered depending on the
TES instance being used.
Run tests using pytest.
poetry run pytest testsThis project is a community effort and lives off your contributions, be it in the form of bug reports, feature requests, discussions, ideas, fixes, or other code changes. Please read these guidelines if you want to contribute. And please mind the code of conduct for all interactions with the community.
This project is distributed under the Apache License 2.0, a
copy of which is also available in LICENSE.
The project is maintained by ELIXIR Cloud & AAI, a Driver Project of the Global Alliance for Genomics and Health (GA4GH), under the umbrella of the ELIXIR Compute Platform. For filing bug reports, feature requests or other code-related issues, please make use of the project's issue tracker.
