- This event has passed.
GRASP Special Seminar: Zhengyou Zhang, Microsoft Research, “3D Computer Vision and Multimodal Interaction for Large Touch Displays”
May 19, 2014 @ 1:30 pm - 2:30 pm
Abstract: We describe a system that
augments touch input with visual understanding of the user to improve
interaction with a large touch-sensitive display. A commodity color plus depth
sensor such as Microsoft Kinect adds the visual modality and enables new
interactions beyond touch. Through visual analysis, the system understands
where the user is, who the user is, and what the user is doing even before the
user touches the display. Such information is used to enhance interaction in multiple
ways. For example, a user can use simple gestures to bring up menu items such
as color palette and soft keyboard; menu items can be shown where the user is
and can follow the user; hovering can show information to the user before the
user commits to touch; the user can perform different functions (for example
writing and erasing) with different hands; and the user’s preference profile
can be maintained, distinct from other users. User studies are conducted and
the users very much appreciate the value of these and other enhanced
interactions.
augments touch input with visual understanding of the user to improve
interaction with a large touch-sensitive display. A commodity color plus depth
sensor such as Microsoft Kinect adds the visual modality and enables new
interactions beyond touch. Through visual analysis, the system understands
where the user is, who the user is, and what the user is doing even before the
user touches the display. Such information is used to enhance interaction in multiple
ways. For example, a user can use simple gestures to bring up menu items such
as color palette and soft keyboard; menu items can be shown where the user is
and can follow the user; hovering can show information to the user before the
user commits to touch; the user can perform different functions (for example
writing and erasing) with different hands; and the user’s preference profile
can be maintained, distinct from other users. User studies are conducted and
the users very much appreciate the value of these and other enhanced
interactions.