We propose and demonstrate a framework called perception as prediction for\nautonomous driving that uses general value functions (GVFs) to learn\npredictions. Perception as prediction learns data-driven predictions relating\nto the impact of actions on the agent's perception of the world. It also\nprovides a data-driven approach to predict the impact of the anticipated\nbehavior of other agents on the world without explicitly learning their policy\nor intentions. We demonstrate perception as prediction by learning to predict\nan agent's front safety and rear safety with GVFs, which encapsulate\nanticipation of the behavior of the vehicle in front and in the rear,\nrespectively. The safety predictions are learned through random interactions in\na simulated environment containing other agents. We show that these predictions\ncan be used to produce similar control behavior to an LQR-based controller in\nan adaptive cruise control problem as well as provide advanced warning when the\nvehicle behind is approaching dangerously. The predictions are compact\npolicy-based predictions that support prediction of the long term impact on\nsafety when following a given policy. We analyze two controllers that use the\nlearned predictions in a racing simulator to understand the value of the\npredictions and demonstrate their use in the real-world on a Clearpath Jackal\nrobot and an autonomous vehicle platform.\n