ESTIMATING GROUND HEIGHT BASED ON LIDAR DATA

Techniques for controlling a vehicle based on height data and/or classification data being determined utilizing multi-channel image data are discussed herein. The vehicle can capture lidar data as it traverses an environment. The lidar data can be associated with a voxel space as three-dimensional data. Semantic information can be determined and associated with the lidar data and/or the three-dimensional voxel space. A multi-channel input image can be determined based on the three-dimensional voxel space and input into a machine learned (ML) model. The ML model can output data to determine height data and/or classification data associated with a ground surface of the environment. The height data and/or classification data can be utilized to determine a mesh associated with the ground surface. The mesh can be used to control the vehicle and/or determine additional objects proximate the vehicle.