Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Making

Aerial cinematography is revolutionizing industries that require live and\ndynamic camera viewpoints such as entertainment, sports, and security. However,\nsafely piloting a drone while filming a moving target in the presence of\nobstacles is immensely taxing, often requiring multiple expert human operators.\nHence, there is demand for an autonomous cinematographer that can reason about\nboth geometry and scene context in real-time. Existing approaches do not\naddress all aspects of this problem; they either require high-precision\nmotion-capture systems or GPS tags to localize targets, rely on prior maps of\nthe environment, plan for short time horizons, or only follow artistic\nguidelines specified before flight.\n In this work, we address the problem in its entirety and propose a complete\nsystem for real-time aerial cinematography that for the first time combines:\n(1) vision-based target estimation; (2) 3D signed-distance mapping for\nocclusion estimation; (3) efficient trajectory optimization for long\ntime-horizon camera motion; and (4) learning-based artistic shot selection. We\nextensively evaluate our system both in simulation and in field experiments by\nfilming dynamic targets moving through unstructured environments. Our results\nindicate that our system can operate reliably in the real world without\nrestrictive assumptions. We also provide in-depth analysis and discussions for\neach module, with the hope that our design tradeoffs can generalize to other\nrelated applications. Videos of the complete system can be found at:\nhttps://youtu.be/ookhHnqmlaU.\n