Toy-GS: Assembling Local Gaussians for Precisely Rendering Large-Scale Free Camera Trajectories

Currently, 3D rendering for large-scale free camera trajectories, namely,\narbitrary input camera trajectories, poses significant challenges: 1) The\ndistribution and observation angles of the cameras are irregular, and various\ntypes of scenes are included in the free trajectories; 2) Processing the entire\npoint cloud and all images at once for large-scale scenes requires a\nsubstantial amount of GPU memory. This paper presents a Toy-GS method for\naccurately rendering large-scale free camera trajectories. Specifically, we\npropose an adaptive spatial division approach for free trajectories to divide\ncameras and the sparse point cloud of the entire scene into various regions\naccording to camera poses. Training each local Gaussian in parallel for each\narea enables us to concentrate on texture details and minimize GPU memory\nusage. Next, we use the multi-view constraint and position-aware point adaptive\ncontrol (PPAC) to improve the rendering quality of texture details. In\naddition, our regional fusion approach combines local and global Gaussians to\nenhance rendering quality with an increasing number of divided areas. Extensive\nexperiments have been carried out to confirm the effectiveness and efficiency\nof Toy-GS, leading to state-of-the-art results on two public large-scale\ndatasets as well as our SCUTic dataset. Our proposal demonstrates an\nenhancement of 1.19 dB in PSNR and conserves 7 G of GPU memory when compared to\nvarious benchmarks.\n