Loading Events

« All Events

  • This event has passed.

Spring 2022 GRASP SFI: Paloma Sodhi, Carnegie Mellon University, “Learning in factor graphs for tactile perception”

January 26, 2022 @ 3:00 pm - 4:00 pm

*This was a HYBRID Event with in-person attendance in Levine 307 and Virtual attendance…


Factor graphs offer a flexible and powerful framework for solving large-scale, nonlinear inference problems as encountered in robot perception. Typically these methods rely on handcrafted models that are efficient to optimize. However, robots often perceive the world through complex, high-dimensional sensor observations. For instance, consider a robot manipulating an object in-hand and receiving high-dimensional tactile observations from which it must infer latent object poses. Can we learn models for such observations directly from sensor data?

In this talk, I will discuss algorithms and representations for learning observation models end-to-end with optimizers in the loop. I will present a novel approach, LEO, that casts the problem of learning observation models as cost function learning that makes no assumptions on the differentiability of the underlying optimizer. I will also discuss different feature representations for extracting salient information from tactile image observations. We will evaluate these approaches on a real-world application of tactile perception for robot manipulation where we demonstrate reliable object tracking in hundreds of trials across planar pushing and in-hand manipulation tasks.


Paloma Sodhi

Paloma Sodhi

Paloma Sodhi is a Ph.D. student in Robotics at Carnegie Mellon University advised by Michael Kaess. She is currently also a Visiting Researcher at Meta AI Research. Her research lies at the intersection of machine learning and optimization for robot perception. Her thesis leverages machine learning to extract salient information from measurements and optimization to efficiently fuse such information. Her research has been recognized as best paper finalists at IROS 2017 and IROS 2021. She is a University Presidential Fellow 2018 and RSS Pioneer 2021.


January 26, 2022
3:00 pm - 4:00 pm
Event Category:


Levine Hall Conference Room 307
3330 Walnut Street
+ Google Map