Abstract: There is increasing interest, activity, and urgency to build robots capable of performing manipulation tasks with human-like competency in human environments. I will present our work towards this objective in the areas of motion planning, perception, and manipulation. Our work focuses on challenges that arise when scope and complexity of tasks cannot be conceived by a programmer a priori but instead have to be assessed and addressed by the robot during task execution in an autonomous fashion. I will conjecture that to achieve this level of competency and autonomy, we – as a research community – may have to change our ways, slightly.