Abstract:
Modern foundation models show high performance in language-related tasks such as question answering, and they have also been applied to modalities outside of text, particularly vision. Such multimodal foundation models can unlock a variety of complex reasoning capabilities beyond text-based reasoning. Extending models to new modalities brings unique challenges, including the ability to generalize to more niche domains and the ability to generalize to variation in user input. In this talk, I will survey the landscape of modern multimodal foundation models, then discuss two recent projects meant to empower models with more robust inference performance and domain-specific reasoning.
Ian Stewart is an NLP researcher at the Pacific Northwest National Laboratory, where he develops novel methods for interpretable and robust machine learning systems. Ian holds a PhD in Human-Centered Computing from the Georgia Institute of Technology.
