Jianbo Shi
Professor, CIS
Jianbo’s group is developing vision algorithms for both human and image recognition. Their ultimate goal is to develop computation algorithms to understand human behavior and interaction with objects in video, and to do so at multiple levels of abstractions: from the basic body limb tracking, to human identification, gesture recognition, and activity inference. Jianbo and his group are working to develop a visual thinking model that allows computers not only to understand their surroundings, but also to achieve higher level cognitive abilities such as machine memory and learning.
Startups
Projects
Publications
Transfer Entropy as a Measure of Information Flow in LLMs and VLMs
Publisher NeurIPS
ID-LoRA: Intrinsic Dimension-Based Pruning with Low-Rank Adaptation for Vision-Language Models
Publisher TechRxiv
Publisher CVPR
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Publisher CVPR
Publisher Frontiers in Surgery
Perceptual Artifacts Localization for Image Synthesis Tasks
Publisher ICCV
Publisher ArXiv
Publisher Neurosurgery
Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis
Publisher ArXiv
Publisher Journal of Neurological Surgery Part B: Skull Base
Guest Editorial: Special Issue on Advances in Computer Vision and Applications (ACCV 2020)
Publisher International Journal of Computer Vision
HashEncoding: Autoencoding with Multiscale Coordinate Hashing
Publisher ArXiv
Inpainting at Modern Camera Resolution by Guided PatchMatch with Auto-curation
Publisher European Conference on Computer Vision
Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications
Publisher European Conference on Computer Vision
Perceptual Artifacts Localization for Inpainting
Publisher European Conference on Computer Vision
Publisher ArXiv
FCNet: A feature context network based on ensemble framework for image retrieval
Publisher IET Computer Vision
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Publisher CVPR
Towards Statistically Provable Geometric 3D Human Pose Recovery
Publisher SIAM Journal on Imaging Sciences
Publisher IEEE Winter Conference on Applications of Computer Vision (WACV)
Enhanced generative adversarial network for 3D brain MRI super-resolution
Publisher IEEE Winter Conference on Applications of Computer Vision (WACV)
Force from Motion: Decoding Control Force of Activity in a First-Person Video
Publisher IEEE Transactions on Pattern Analysis and Machine Intelligence
FoveaBox: Beyound Anchor-based Object Detection
Publisher IEEE Transactions on Image Processing
Publisher Imaging Informatics for Healthcare, Research, and Applications
Multimodal image outpainting with regularized normalized diversification
Publisher IEEE Winter Conference on Applications of Computer Vision (WACV)
Nested Scale-Editing for Conditional Image Synthesis
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Interpretable Identification of Interstitial Lung Disease (ILD) Associated Findings from CT
Publisher MICCAI
Learning Object Placement by Inpainting for Compositional Data Augmentation
Publisher European Conference on Computer Vision
ForkGAN: Seeing into the Rainy Night
Publisher European Conference on Computer Vision
Toward Continuous-Time Representations of Human Motion
Publisher European Conference on Computer Vision
Adversarial structure matching for structured prediction tasks
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Consistent optimization for single-shot object detection
Publisher arXiv
Neural Embedding for Physical Manipulations
Publisher arXiv
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher Angewandte Chemie
SegSort: Segmentation by discriminative sorting of segments
Publisher IEEE International Conference on Computer Vision (ICCV)
Trajectory Normalized Gradients for Distributed Optimization
Publisher arXiv
Zoom-in-to-check: Boosting video interpolation via instance-level discrimination
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Learning Temporal Pose Estimation from Sparsely-Labeled Videos
Publisher NeurIPS
Egocentric Basketball Motion Planning from a Single First-Person Image
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Learning discriminative motion features through detection
Publisher arXiv
Object detection in video with spatiotemporal sampling networks
Publisher Lecture Notes in Computer Science (LNCS)
Am i a Baller? Basketball Performance Assessment from First-Person Videos
Publisher IEEE International Conference on Computer Vision (ICCV)
Connecting the dots: Embodied visual perception from first-person cameras
Publisher IAPR International Conference on Machine Vision Applications (MVA)
Convolutional random walk networks for semantic image segmentation
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Customizing First Person Image Through Desired Actions
Publisher arXiv
First-person action-object detection with EgoNet
Publisher Robotics: Science and Systems (RSS)
Local perturb-and-MAP for structured prediction
Publisher International Conference on Artificial Intelligence and Statistics (AISTATS)
Motion-Based Temporal Alignment of Independently Moving Cameras
Publisher IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)
Predicting behaviors of basketball players from first person videos
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Synchronization of Video Sequences Through 3D Trajectory Reconstruction
Publisher Zidonghua Xuebao/Acta Automatica Sinica
Unsupervised Learning of Important Objects from First-Person Videos
Publisher IEEE International Conference on Computer Vision (ICCV)
Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention
Publisher IEEE International Conference on Computer Vision Workshops (ICCVW)
Publisher Lecture Notes in Computer Science (LNCS)
Egocentric future localization
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Force from Motion: Decoding Physical Sensation in a First Person Video
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Integrated intelligence for human-robot teams
Publisher International Symposium on Experimental Robotics (ISER)
Rich Representations with Exposed Semantics for Deep Visual Reasoning
Publisher Carnegie Melon University Pittsburgh United States
Semantic Segmentation with Boundary Neural Fields
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Social behavior prediction from first person videos
Publisher arXiv
DeepEdge: A multi-scale bifurcated deep network for top-down contour detection
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Exploiting egocentric object prior for 3d saliency detection
Publisher arXiv
Future Localization from an Egocentric Depth Image
Publisher arXiv
Publisher IEEE International Conference on Computer Vision (ICCV)
Publisher IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Pulling Things out of Perspective
Publisher CVPR
Multi-Camera System Based Driver Behavior Analysis
Publisher TRID
Machine Learning of Hierarchical Clustering to Segment 2D and 3D Images
Publisher PloS one
Publisher ICCV
Holistic Shape Recognition: Where-to-Look and How-to-Look
Publisher Shape Perception in Human and Computer Vision: An Interdisciplinary Perspective
Pose from Flow and Flow from Pose
Publisher CVPR
Two-Granularity Tracking: Mediating Trajectory and Detection Graphs for Tracking under Occlusions
Publisher European Conference on Computer Vision
Structural-Flow Trajectories for Unravelling 3D Tubular Bundles
Publisher International Conference on Medical Image Computing and Computer-Assisted Intervention
Publisher CVPR
Discriminative image warping with attribute flow
Publisher CVPR
Contour cut: Identifying salient contours in images by solving a Hermitian eigenvalue problem
Publisher CVPR
Generalized Random Walks for Fusion of Multi-Exposure Images
Publisher IEEE Transactions on Image Processing
Publisher University of Pennsylvania
Multi-hypothesis motion planning for visual object tracking
Publisher ICCV
Figure-Ground Image Segmentation Helps Weakly-Supervised Learning of Objects
Publisher European Conference on Computer Vision
Building recognition from aerial images combining segmentation and shadow
Publisher IEEE International Conference on Intelligent Computing and Intelligent Systems
Color space selection for color image segmentation by spectral clustering
Publisher IEEE International Conference on Signal and Image Processing Applications
Publisher CVPR
Shape Extraction through Region-Contour Stitching
Publisher International Symposium on Visual Computing
Saliency Based Opportunistic Search for Object Part Extraction and Labeling
Publisher European Conference on Computer Vision
Contour Context Selection for Object Detection: A Set-to-Set Contour Matching Approach
Publisher European conference on computer vision
Grouping Contours Via a Related Image
Publisher NeurIPS
Object Detection Combining Recognition and Segmentation
Publisher Asian Conference on Computer Vision
Untangling Cycles for Contour Grouping
Publisher ICCV
Bottom-Up Recognition and Parsing of the Human Body
Publisher International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition
Image Matching via Saliency Region Correspondences
Publisher CVPR
Solving Markov Random Fields with Spectral Relaxation
Publisher Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics
Comparing Ensembles of Learners: Detecting Prostate Cancer from High Resolution MRI
Publisher International Workshop on Computer Vision Approaches to Medical Image Analysis
Publisher Advances in Neural Information Processing Systems 19
Spectral segmentation with multiscale graph decomposition
Publisher CVPR
Learning spectral graph segmentation
Publisher Workshop on Artificial Intelligence and Statistics (AISTATS)
Publisher Medical Image Computing and Computer-Assisted Intervention–MICCAI 2005
Publisher Advances in Neural Information Processing Systems 17: Proceedings of the 2004 Conference
Detecting unusual activity in video
Publisher CVPR
A Learnable Spectral Memory Graph for Recognition and Segmentation
Publisher University of Pennsylvania Technical Report
Multiple frame motion inference using belief propagation
Publisher Sixth IEEE International Conference on Automatic Face and Gesture Recognition
Segmentation given partial grouping constraints
Publisher IEEE Transactions on Pattern Analysis and Machine Intelligence
Multiclass spectral clustering
Publisher ICCV
Steady-State Feedback Analysis of Tele-Graffiti
Publisher Proceedings of the IEEE International Workshop on Projector-Camera Systems
Publisher International Journal of Computer Vision
Object-specific figure-ground segregation
Publisher CVPR
News
Faculty News
Profs at Penn’s GRASP Lab snagged Toyota Research Institute funding for robotics research
February 1, 2021
Faculty News
Penn Engineers Join Honda Research Institute’s ‘Curious Minded Machine’ Project
October 25, 2018
GRASP News
“Penn Engineering Offers an Online MicroMasters in Robotics” in Penn Engineering
February 28, 2017
GRASP News
Why 10 District teachers spent their summer doing grad-level STEM research
August 24, 2016
GRASP News
July 14, 2015
Ph.D. Students
Alumni
PhD, CIS '13 - JPMorgan Chase Assistant Professor, Machine Learning Department, CMU