Semi-Autonomous Mobile Device Driving with Obstacle Avoidance

The subject disclosure is directed towards driving a robot or other mobile device safely through an environment by using a depth camera to obtain depth data, and then using the depth data for collision avoidance. Horizontal profile information may be built from the depth data, such as by collapsing a two-dimensional depth map into one-dimensional horizontal profile information. The horizontal profile information may be further processed by fusing the depth camera-detected obstacle data with any closer obstacle data as detected via infrared-based sensing and/or sonar-based sensing. Driving suggestions from a user or program are overridden as needed to avoid collisions, including by driving the robot towards an open space represented in the horizontal profile information, or stopping/slowing the robot as needed.