Summary
This paper proposes a new INR architecture to admit easy cropping of the target datum to a certain partition, allowing one to save memory and inference cost without any retraining. Comparing with training a new INR for the target partition, the approach lets one utilize the global context as well. The idea is to train multiple local networks, and modulate their intermediate features with the features of a global network. Through experiments on image/audio/video encoding, the paper demonstrates that the method enjoys faster and more accurate fitting than training one INR per partition. Also, the paper shows that the method works when combined with many INR architectures, e.g., SIREN and INCODE.
Strengths
- A notable strength of the proposed method is that it is very simple and easy to use, making it likely to be scalable and generally applicable. I believe that the method can also be combined well with the triplane-based neural fields or instant-ngp.
- The method is also very clearly presented. In particular, Figure 3 is very effective in delivering how the proposed architecture works.
- Lastly, I appreciate the fact that the paper provides experiments on many different modalities, from image to video.
Weaknesses
- **Utility of croppability?** The key weakness of this paper is the motivation. Apart from "saving storage & compute," the practical utility of having a croppable INR is unclear; will local-global INRs also be useful in performing any subsequent "editing" operations? I suspect that this is why authors provide section 4.4, where the authors "extend" the LGS to parameterize the larger image than the one originally considered. However, for such purposes, there are already other good meta-learning-based solutions such as [26]. I am not sure why the model should be croppable for such applications.
- **Novelty & Ablations.** The proposed method bears much similarity with [26], which modulates the local model with another global model. The key difference here is how we modulate; this paper uses an extra linear layer to process the local+global features, while [26] uses multiplications (later works, such as functa, used addition). To fully understand what this paper contributes, there should be an explicit comparison with these similar methods as a baseline.
- **Evaluation.** If I understood correctly, the evaluation is mostly based on how the model fits the seen coordinates. I wonder how these affect the generalizability of the learned signal to unseen coordinates. As this is one of the key strengths of having a global context, I have enough reasons to believe that LGS will work well. However, an explicit verification is needed.
- **Hyperparameter tuning.** I wonder how the hyperparameters for the models and the baselines are selected. In particular, how were the values of "omega" and the learning rate selected? These two are quite critical in determining the fitting speed, so this point should be crystal clear.
Questions
Please see the "weakness."