Useful fault-tolerant quantum computers require very large numbers of\nphysical qubits. Quantum computers are often designed as arrays of static\nqubits executing gates and measurements. Photonic qubits require a different\napproach. In photonic fusion-based quantum computing (FBQC), the main hardware\ncomponents are resource-state generators (RSGs) and fusion devices connected\nvia waveguides and switches. RSGs produce small entangled states of a few\nphotonic qubits, whereas fusion devices perform entangling measurements between\ndifferent resource states, thereby executing computations. In addition,\nlow-loss photonic delays such as optical fiber can be used as fixed-time\nquantum memories simultaneously storing thousands of photonic qubits. Here, we\npresent a modular architecture for FBQC in which these components are combined\nto form "interleaving modules" consisting of one RSG with its associated fusion\ndevices and a few fiber delays. Exploiting the multiplicative power of delays,\neach module can add thousands of physical qubits to the computational Hilbert\nspace. Networks of modules are universal fault-tolerant quantum computers,\nwhich we demonstrate using surface codes and lattice surgery as a guiding\nexample. Our numerical analysis shows that in a network of modules containing\n1-km-long fiber delays, each RSG can generate four logical distance-35\nsurface-code qubits while tolerating photon loss rates above 2% in addition to\nthe fiber-delay loss. We illustrate how the combination of interleaving with\nfurther uses of non-local fiber connections can reduce the cost of logical\noperations and facilitate the implementation of unconventional geometries such\nas periodic boundaries or stellated surface codes. Interleaving applies beyond\npurely optical architectures, and can also turn many small disconnected\nmatter-qubit devices with transduction to photons into a large-scale quantum\ncomputer.\n