CircuitVQA: A VQA Dataset for Electrical Circuit Images
A visual question answering (VQA) system for electrical circuit images could be useful as a quiz generator, design and verification assistant or an electrical diagnosis tool. Although there exists a vast literature on VQA, to the best of our knowledge, there is no existing work on VQA for electrical circuit images. To this end, we curate a new dataset, CIRCUITVQA, of 115K+ questions on 5725 electrical images with ~70 circuit symbols. The dataset contains schematic as well as hand-drawn images. The questions span various categories like counting, value, junction and position based questions. To be effective, models must demonstrate skills like object detection, text recognition, spatial understanding, question intent understanding and answer generation. We experiment with multiple foundational visio-linguistic models for this task and find that a finetuned BLIP model with component descriptions as additional input provides best results. We make the code and dataset publicly available.
In this video, I talk about the following: What is VQA for electrical circuit images, and why do this? Related Work. CircuitVQA Dataset Curation. Generation of Question Templates. Generation of Question Answer Pairs. CircuitVQA Dataset Analysis. Methods for CircuitVQA. Input Representations. Input Prompt Templates for Instruction-based Models. Results. Examples of Predictions from our best model. Examples of error cases from our best model.
For more details, please look at https://drive.google.com/file/d/1kaqb...
CircuitVQA: A Visual Question Answering Dataset for Electrical Circuit Images. Rahul Mehta, Bhavyajeet Singh, Vasudeva Varma, Manish Gupta. The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD). 2024.