'Auxiliary signal-guided knowledge encoder-decoder for medical report generation' - Dr Xiaojun Chang
For computer-generated medical imaging reports to be useful, they must include written information that satisfies medical common sense and logic. Previous attempts have focussed on extracting the global image features and generating a paragraph that is similar to referenced reports; however, this approach has two limitations. Firstly, radiologists are usually focused on a small area of the global image, and the rest of the image could be irrelevant noise. Secondly, data bias occurs from similar sentences being repeatedly used in each report to describe the normal regions of the image.
Dr Xiaojun Chang will discuss a new Auxiliary Signal-Guided Knowledge Encoder-Decoder (ASGK) that improves medical report generation by mimicking radiologists’ working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning. Experiments on a COVID-19 CT report dataset indicate that this ASGK can generate a robust and accurate report, and outperforms state-of-the-art methods on both medical terminology classification and paragraph generation metrics.
Dr Xiaojun Chang is a Senior Lecturer in the Vision and Language Group, Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University. He is an ARC Discovery Early Career Researcher Award Fellow. Dr Chang has been working on developing deep learning models to automatically annotate the disease labels from multi-source patient data in Intensive Care Units. Recently, he successfully developed an automatic report generation system for critically ill COVID-19 patients using deep learning techniques.