Research Profile

I am a Student Researcher at Google DeepMind and a PhD student in Machine Learning at TU Berlin and the Berlin Institute for the Foundations of Learning and Data (BIFOLD). I am also a guest researcher in the ViCCo Group at the Max Planck Institute for Human Cognitive and Brain Sciences. I am mainly advised by Klaus-Robert Müller and co-supervised by Martin Hebart, Simon Kornblith, and Andrew Lampinen. During my PhD, I've been part of a Research Collaboration between TU Berlin and Google Brain, where I was advised by Simon Kornblith. Previously, I was a MSc student in IT & Cognition / Computer Science of Isabelle Augenstein and Johannes Bjerva at the University of Copenhagen where I mostly worked on Question Answering and Machine Translation.

My research mainly revolves around representation learning in computer vision. In particular, I try to understand the factors that influence the degree of alignment between human mental and neural network representations and use inspiration from human cognition to improve deep learning models. My goal is to build interpretable (vision) foundation models that generalize to downstream out-of-distribution settings (similar to how the human brain does); something that we partly achieved here.

Occasionally I dabble in philosophical discussions about representational alignment and try to develop common language across research fields together with other people in the field. Recently, I've been thinking a lot about the transferability of representational similarities across datasets. Have a look at my Google Scholar for more information about my work. Feel free to reach out to me, if you believe our research intentions are aligned (pun intended) and you are keen to collaborate on a project.

Academic Background

2023-2025 | Student Researcher, Google DeepMind
2021-2025 | PhD, Machine Learning, Machine Learning Group & BIFOLD, TU Berlin, Germany
2021-2024 | Guest researcher, Computational Cognitive Neuroscience ViCCo Group, Max Planck Insitute for Human Cognitive and Brain Sciences, Germany
2020-2021 | Research Associate, Computational Cognitive Neuroscience ViCCo Group, Max Planck Insitute for Human Cognitive and Brain Sciences, Germany
2018-2020 | M.Sc. IT & Cognition Department of Computer Science, University of Copenhagen, Denmark – Thesis: Subjective Question Answering: Deciphering the inner workings of Transformers in the realm of subjectivity
2015-2018 | B.Sc. Cognitive Psychology Faculty of Psychology, University of Vienna, Austria

Service

Conference & Workshop Organization

Organizer for Re-Align: The First Workshop on Representational Alignment at ICLR 2024 in Vienna
Part of the Trainee Organizing Committee for CCN 2024 in Boston

Reviewing

Reviewer for NeurIPS, ICLR, ICML, AAAI, TMLR.

News

2024

November: We updated our Perspective on representational alignment and included feedback that we received from the community over the year! Go check it out! This version is more crisp and more comprehensive.
October: New preprint on the variables that drive the consistency of representational similarity across vision datasets out on arXiv! This was my first senior author project ever!
September: One paper on the benefits of perceptual alignment for vision representations accepted to NeurIPS 2024 in Vancouver!
September: My one-year research internship project at Google DeepMind about Aligning Machine and Human Visual Representations across Abstraction Levels is finally out as a preprint on arXiv! Massive team effort!
August: I attended CCN 2024 in Boston and presented some of our most recent work on alignment in the vision space!
August: I am back at Google DeepMind as a part-time Student Researcher!
June: New preprint on dimensions underlying the alignment of deep neural nets with humans available on arXiv!
June: I gave an invited talk on human alignment at the Donders Institute in the Netherlands!
May: I am attending ICLR in Vienna as a poster presenter and workshop organizer! Hit me up!
May: One paper accepted to CCN 2024!
March: I finished my internship at Google DeepMind!
March: I am serving as a reviewer for ICML 2024!
March: I gave an invited talk at Helmholtz AI & TUM in the Human-Centered AI Lab and the Explainable Machine Learning Lab of Eric Schulz and Zeynep Akata respectively!
January: Our paper on set learning for accurate and calibrated models was accepted to ICLR 2024! Hooray!
January: Our Call for Papers for Re-Align: The First Workshop on Representational Alignment at ICLR 2024 in Vienna is out! We are looking forward to your submissions! Submission deadline is February 3rd.

2023

December: Our workshop proposal on representational alignment was accepted to ICLR 2024! See y’all in Vienna!
October: I gave an invited talk on human alignment of neural network representations in the lab of Mariya Toneva!
October: One workshop proposal on representational alignment submitted to ICLR 2024!
October: I am serving as a reviewer for ICLR 2024!
October: I’ve started working as a Student Researcher @ Google DeepMind!
September: Our paper on improving neural network representations using human similarity judgments was accepted to NeurIPS 2023! Hooray!
August: I am presenting work on interpretable object dimensions at CCN! Hello Oxford!
August: I gave an invited talk on human alignment of neural network representations at the Max Planck Institute for Human Cognitive and Brain Sciences!
July: Our workshop proposal on representational alignment was unfortunately rejected at NeurIPS 2023 but we will not give up and re-submit a revised version to ICLR!
July: I am serving as a reviewer for NeurIPS 2023!
July: I gave a spotlight talk on human alignment of neural network representations at the annual BIFOLD retreat!
June: New preprint on set learning for accurate and calibrated models available on arXiv!
June: New preprint on improving neural network representations using human similarity judgments available on arXiv!
May: Paper on interpretable object dimensions in Deep Neural Nets accepted to CCN 2023! See y’all in Oxford!
May: One workshop proposal on representational alignment submitted to NeurIPS 2023!
May: Two papers submitted to NeurIPS 2023!
May: I am presenting our paper on human alignment of neural network representations virtually at ICLR 2023!
January: Paper on human alignment of neural network representations accepted to ICLR 2023 in Kigali, Rwanda!

2022

October: Paper accepted to the SVRHM workshop @ NeurIPS 2022!
October: I gave my first ever Machine Learning lecture at TU Berlin!
September: One paper submitted to ICLR 2023!
September: Our paper on Variational Interpretable Concept Embeddings (VICE) was accepted to NeurIPS 2022!
September: I received a Google Research Collabs grant with Simon Kornblith to work on human alignment (80.000 USD research funding + 10.000 USD Google Cloud credits)! I will be commencing the collaboration as part of the Google Research Collabs Cohort 2022/23 in November!
July: I was awarded 1,000 USD as part of the Google Cloud Platform research credits programme for working on long-tail image classification with Thomas Unterthiner!
May: One paper submitted to NeurIPS 2022!
February: Abstract on interpretable object dimensions in DNNs accepted as an oral to VSS 2022!
February: I was accepted into the Berlin Institute for the Foundations of Learning and Data (BIFOLD) to continue my PhD studies at TU Berlin with a fully-funded position!

2021

September: Paper on my Python toolbox thingsvision for extracting features from state-of-the-art computer vision models accepted to Frontiers in Neuroinformatics!
July: As of July 1st, I am a PhD student in the Machine Learning Group at TU Berlin, working on cognitively-inspired Machine Learning! Excited to continue my journey at the intersection of ML and Cognitive Science; this time with a stronger emphasis on ML than on CogSci.

2020

September: Paper on Unsupervised QA Evaluation of Transformers with my previous MSc. supervisors Isabelle Augenstein and Johannes Bjerva accepted to BlackBoxNLP @ EMNLP 2020!
August: I started as a full-time Research Assistant (RA) in the Vision and Computational Cognition Group at the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, Germany. Excited to start working at the intersection of Machine Learning and Cognitive Neuroscience!
June/July: In June I successfully defended my MSc thesis on Subjective Question Answering and in July I graduated with distinction from the University of Copenhagen (UCPH).

2019

November: Our paper on Assisted Declarative Process Creation from Natural Language Descriptions won the Best Demonstration Award at EDOC 2019!
September: Paper on Assisted Declarative Process Creation from Natural Language Descriptions accepted to the 23rd IEEE International Enterprise Distributed Object Computing Conference (EDOC 2019)!
July: I started working on NLP for law documents as a Research Assitant (RA) in the Department of Computer Science at the University of Copenhagen.
June: Our algorithm that won the PAN autorship attribution competition at CLEF 2019 was featured in the CLEF 2019 Working Notes!
May: Our team won the PAN authorship attribution competition at CLEF 2019 in Lugano, Switzerland!
February: Paper on Visual Working Memory from my time as a research intern during my BSc accepted to Frontiers in Psychology!
January: Paper on Amblyopia from my time as a research intern during my BSc accepted to the Journal of European Psychology Students (JEPS)!

2018

June: I was addmitted to the IT & Cognition MSc programme at the University of Copenhagen. Excited to start working on Natural Language Processing in Copenhagen! I will be starting my studies in September.

Lukas Muttenthaler