Bioconductor Docker Images For Multi-Node Parallel Computing On The Cloud
Nitesh Turaga, with Dana Farber Cancer Institute, gave a workshop at the BioConductor Conference 2022. Turaga’s workshop was on Bioconductor Docker Images For Multi-Node Parallel Computing On The Cloud.
From the Bioconductor Conference 2022
Author(s): Nitesh Turaga
Affiliation(s): Dana Farber Cancer Institute
Bioconductor produces docker images that are widely used because they containerize system dependencies of all Bioconductor packages along with the community version of RStudio. Using Kubernetes, a container orchestration software, it is now possible to deploy these docker images on a cluster and use them for multi-node parallel computing. In this workshop, we introduce commands to launch such a cluster on a cloud provider (Google, Azure, AWS) and use a new BiocParallel back-end called 'RedisParam' to distribute jobs from the manager to the workers. In addition, the paradigm creates a traditional parallel computing framework on the cloud using the same containerized applications available to experiment with on local machines. The advantage of such a cluster launched by Kubernetes is fault tolerance and the potential of auto-scaling. Prequiresites: Some familiarity with BiocParallel and Bioconductor docker images.
Workshop details
https://bit.ly/cloud-parallel-computi...
Source code
https://github.com/nturaga/bioc-k8s-c...
More Resources
Bioconductor Conference Site: https://bioc2022.bioconductor.org/
BioC2022 Github: https://github.com/Bioconductor/BioC2022
Main Site: https://www.r-consortium.org/
News: https://www.r-consortium.org/news
Twitter: / rconsortium
LinkedIn: / r-consortium