Given the current capabilities of autonomous vehicles, one can easily imagine autonomous vehicles being released on the road in the near future. However, it can be assumed that this transition will not be instantaneous, suggesting two key points: (1) levels of autonomy will be introduced incrementally (e.g. active safety systems as currently released), and (2) autonomous vehicles will have to be capable of driving in a mixed environment, with both humans and autonomous vehicles on the road. In both of these cases, the human driven vehicle (or generally the human-in-the-loop system) must be modeled in an accurate and precise manner that is easily integrated into control frameworks. In this talk, the driver modeling framework that estimates the empirical reachable set will be presented, which is an alternative look at a classic control theoretic safety metric. This method allows for us to predict driving behavior over long time horizons with very high accuracy. This modeling framework has been applied to intervention schemes for semi-autonomous vehicles and to nuanced interactions between humans and autonomy in cooperative maneuvers. By taking a human-centered approach, we observe improved predictability and trustworthiness of the automation from the user’s perspective. Additionally, we will discuss lessons learned when working with and designing control for humans as well as some future directions for human-robot interaction and shared control.