Detailed Answer
> W1: Use of linear networks and comparison to non-linear networks.
As mentioned in the answer to reviewer qEL8, the dynamics of linear networks are non-linear with a non-convex landscape, resembling the dynamics of more complex non-linear neural networks. The part that is linear is the forward path, but the control signal is dealing with essentially a non-linear dynamical system due to the nonlinear loss function and weight coupling between the first and second layers as described in Sec 3. We run an additional experiment to show this, please see the answer to reviewer qEL8.
> W2: It's likely one will need to find a solution for every novel considered intervention.
We agree with your statement. However, as shown in the appendix for each of the models, the math necessary to write the differential equations describing learning as a function of the control signal, in most cases, is standard gradient computation. Your intuition is right, changing the intervention of the control signal or the learning system, will change these equations. This is part of the challenge when extending to non-linear networks. Techniques to describe learning dynamics in non-linear networks exist (see paragraph 3, appendix A), but they have their own drawbacks in terms of mathematical and computational tractability, or required limits in input or hidden dimensionality to obtain closed-form differential equations describing the dynamics. This is a direction we are currently exploring and we can definitely see a path forward, but this is beyond the scope of the paper, since the general framework (Sec 2) presented here is the same.
> W3, Q1 and Q2: Other interventions and learning rules as a control signal
As you mentioned in your review, the experiments presented in this work are vast, and due to the flexibility of our framework, we do not believe it is possible to cover all possible interventions in a single manuscript. The tractability of each control intervention will largely depend on the learning system, data distribution, and degrees of freedom of the control signal. This is all a design choice, for example, in App. H.1.1 we discussed the possibility of using a low-rank basis for the control signal, which might simplify computation and search space. We acknowledge that not all meta-learning phenomena can be instantiated with our framework. There is a literature in meta-learning that we are not covering with our framework, where there is no explicit variable that is meta-optimized. We discussed this in App. F4.
Thank you for the suggestion, using learning rules as control signals is possible, and we now include an experiment demonstrating one approach. In particular, we have considered the setting of “Cao, Summerfield, and Saxe et al. 2020, NeurIPS”, which describes a space of possible learning rules using two parameters, spanning gradient descent, Hebbian contrastive learning, quasi-predictive coding, Hebbian and anti-Hebbian rules. These rules instantiate several popular ideas in theoretical neuroscience. Concretely, for the deep linear network setting, the weight updates are
$$ \Delta W_{1} = \Delta W_{1}^{C}(\gamma) + \Delta W_{1}^{H}(\eta), \hspace{0.2cm} \Delta W_{2} = \Delta W_{2}^{C}(\gamma),$$
with $\Delta W_{i}^{C}$ being a contrastive learning rule, and $\Delta W_{i}^{H}$ a Hebbian learning rule, which are controlled by the parameters $\gamma$ and $\eta$ respectively.
Using our framework, we can find the optimal $\gamma(t)$ and $\eta(t)$ that maximizes value as described in Sec 2. By inferring these parameters, the system is deciding which learning rule to employ within this space over learning. We found that the optimal learning rule is consistent with the original Cao et al. 2020 paper. We provide a new experiment in App F3 which will be referenced in the main text. These methods could offer a route to examining how a learning agent might adapt its learning algorithm to better learn different tasks, for instance. We believe this additional experiment further shows the flexibility of our framework to accommodate different meta-learning settings.
> W3: Switching without knowing the next task in advance
This is an important future direction. We note that in many cognitive science experiments, task switches happen at predictable times (after every 50 trials, say), and our analysis is directly applicable to this setting. To consider the case where task switches are unpredictable would require extensions: fundamentally, one could compute the expected loss over possible next tasks, either analytically in certain cases or through sampling. In essence, while we roll out a single learning trajectory corresponding to a predictable sequence of tasks, a sampling approach would roll out many trajectories with possible next tasks, and construct the loss function as the sum of these tasks. The control signal would then be optimized to perform well for the distribution of switches.