Former math student, now exploring the boundaries between mathematics, artificial intelligence, and language. My work focuses on language models and their ability to reason, reflect, and sometimes hallucinate.
My research is rarely done alone – I share my home office with five cats ("the demons"), who are convinced every keyboard was made for them.
I am fascinated by the intersection of psychology and AI. In particular, I apply psychological frameworks and experimental paradigms to study, challenge, and sometimes surprise artificial models.
Publications
LLMs
Wait, That’s Not an Option: LLMs Robustness with In-correct Multiple-Choice Options
Explores LLMs' threshold between following instruction and accuracy.
arXiv · Project Website
Accepted at The 63rd Annual Meeting of the Association for Computational Linguistics
Explores LLMs' threshold between following instruction and accuracy.
arXiv · Project Website
Accepted at The 63rd Annual Meeting of the Association for Computational Linguistics
VLMs
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models
Investigates VLMs' perspective-taking abilities, benchmarking 12 models.
arXiv · Project Website
Accepted at Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models, NeurIPS 2024
Investigates VLMs' perspective-taking abilities, benchmarking 12 models.
arXiv · Project Website
Accepted at Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models, NeurIPS 2024
Beyond Recognition: Evaluating Visual Perspective Taking in Vision Language Models
Evaluates VLMs’ performance on controlled perspective-taking tasks, finding strong scene understanding but lower accuracy on spatial reasoning and VPT.
arXiv · Project Website: Coming soon
Submitted
Evaluates VLMs’ performance on controlled perspective-taking tasks, finding strong scene understanding but lower accuracy on spatial reasoning and VPT.
arXiv · Project Website: Coming soon
Submitted
Planning
What Matters in Hierarchical Search for Combinatorial Reasoning Problems?
Analyzes subgoal-planning effectiveness in combinatorial reasoning problems.
arXiv · Project Website: Coming soon
Accepted at Generative Models for Decision Making Workshop, ICLR 2024
Analyzes subgoal-planning effectiveness in combinatorial reasoning problems.
arXiv · Project Website: Coming soon
Accepted at Generative Models for Decision Making Workshop, ICLR 2024
Datasets
The Isle (I spy with my little eye)
The Isle dataset enables research on visual perspective taking, scene understanding, and spatial reasoning, with over 4.5k downloads.
The Isle dataset enables research on visual perspective taking, scene understanding, and spatial reasoning, with over 4.5k downloads.
BlenderGaze
The BlenderGaze expands the Isle dataset in scale to further investigate visual perspective taking in VLMs.
The BlenderGaze expands the Isle dataset in scale to further investigate visual perspective taking in VLMs.
Blog Posts
Building Trust With Invisible Robots
From domestic humanoids to the open-source.
From domestic humanoids to the open-source.