Recent advances in machine learning have enabled impressive results in taxonomic classification using image data alone. However, DNA barcodes provide a wealth of information about species relationships that is not always visually apparent. In this talk, I’ll present our work on CLIBD and the BIOSCAN-5M dataset, which leverage multi-modal contrastive learning to improve image-based taxonomic classification by incorporating DNA barcode data during training. CLIBD extends the self-supervised learning approach of BioCLIP to align representations of images, DNA barcodes, and taxonomic labels in a shared embedding space. Using the newly released BIOSCAN-5M dataset of over 5 million insect specimens with paired images and DNA barcodes, we show that this multi-modal training approach significantly improves zero-shot image classification performance – boosting accuracy by over 11% compared to image-only methods. Crucially, while DNA data is leveraged during training, our model enables rapid taxonomic classification using images alone at inference time. This work demonstrates the potential for integrating genetic information to enhance computer vision systems for biodiversity monitoring at scale.
Bio
Graham Taylor is a Canada CIFAR AI Chair at the Vector Institute, a CIFAR Azrieli Global Scholar 2016-2018, a Canada Research Chair in Machine Learning, a professor at the school of engineering at the University of Guelph, and an academic director at NextAI. He was research director at the Vector Institute between 2021-2023.
Taylor’s research spans a number of topics in deep learning. He is interested in open problems such as how to effectively learn with less labeled data, and how to build human-centred AI systems. He is interested in methodologies such as generative modelling, graph representation learning and sequential decision making. He also pursues applied projects that leverage computer vision to mitigate biodiversity loss. He co-organizes the annual CIFAR Deep Learning + Reinforcement Learning Summer School (DLRLSS), and has trained more than 80 students and staff members on AI-related projects.