Inferring representations of 3D scenes from 2D observations is a fundamental\nproblem of computer graphics, computer vision, and artificial intelligence.\nEmerging 3D-structured neural scene representations are a promising approach to\n3D scene understanding. In this work, we propose a novel neural scene\nrepresentation, Light Field Networks or LFNs, which represent both geometry and\nappearance of the underlying 3D scene in a 360-degree, four-dimensional light\nfield parameterized via a neural implicit representation. Rendering a ray from\nan LFN requires only a single network evaluation, as opposed to hundreds of\nevaluations per ray for ray-marching or volumetric based renderers in\n3D-structured neural scene representations. In the setting of simple scenes, we\nleverage meta-learning to learn a prior over LFNs that enables multi-view\nconsistent light field reconstruction from as little as a single image\nobservation. This results in dramatic reductions in time and memory complexity,\nand enables real-time rendering. The cost of storing a 360-degree light field\nvia an LFN is two orders of magnitude lower than conventional methods such as\nthe Lumigraph. Utilizing the analytical differentiability of neural implicit\nrepresentations and a novel parameterization of light space, we further\ndemonstrate the extraction of sparse depth maps from LFNs.\n