Best Practices: Pipeline Development, Part 02
PHA4GE Ten Best Practices for Public Health Bioinformatics Pipelines:
https://github.com/pha4ge/public-heal...
Summary
In this episode, Kevin Libuit and Andrew Page discuss the 10 best practices for public health pipeline development. They start by emphasizing the use of common file formats and the importance of avoiding reinventing the wheel. They highlight the benefits of standard file formats and the availability of parsers for different languages. They also discuss the implementation of software testing, including the use of automated testing and the integration of testing with Docker containers. They emphasize the need for accessibility to benchmark or validation data sets and the importance of reference data requirements. They also touch on the significance of hiring bioinformaticians and the documentation practices that should be followed.
Takeaways
Use common file formats to avoid reinventing the wheel and enable compatibility with other programs.
Implement software testing, including automated testing, to ensure functionality and identify bugs.
Provide benchmark or validation data sets to allow users to compare and evaluate the performance of the pipeline.
Consider the reference data requirements and ensure accessibility to curated databases.
Hire bioinformaticians with domain expertise to navigate the complexities of pipeline development.
Follow documentation practices, including communication of authorship, pipeline maintenance statements, and community guidelines for contribution and support.
Chapters
00:00 Introduction
00:26 Use of Common File Formats
01:22 Barriers to Common File Formats
02:52 Importance of Standard Libraries and Parsers
03:21 Exceptions to Using Common File Formats
04:25 Infamous Examples of Non-Standard File Formats
05:06 Implementation of Software Testing
05:57 Benefits of Automated Testing
06:10 Testing as a Best Practice
07:32 Higher Level Testing
08:23 Integration of Testing with Docker Containers
09:09 Accessibility to Benchmark or Validation Data Sets
10:19 Reference Data Requirements
11:57 Importance of Hiring Bioinformaticians
12:26 Documentation Practices
15:26 Pipeline Maintenance Statement
15:55 Community Guidelines for Contribution and Support