Scalarization for Multi-Task and Multi-Domain Learning at Scale

Summary

The paper studies scalarization for multi-task and multi-domain learning, which is a method to combine the losses of different tasks/domains. The authors conduct substantial experiments to draw insights into the effect of scalarization weights on multi-task/domain learning. They also propose an efficient method to search for a good set of weights.

Strengths

The authors make a valuable attempt to understand the scalarization of multi-task/domain learning. It is an important problem in the literature as despite the existence of various automatic weight selection methods, it is unclear under what circumstances these methods will outperform a static scalarization. The insights are critical and novel, especially the one about conflict gradient and scalarization v.s. dynamic weight update, which is different from common belief.

Weaknesses

The writing can be improved. The goal of this work is ambitious because the authors attempt to study the effect of scalarization from a wide range of aspects. However, such ambition also makes the paragraphs very condensed and jumping. In many cases, the authors bring up several insights in one paragraph with little logical flow (e.g. sec 4.1). It is better to use latex paragraph if many parallel information needs to be conveyed. Some conceptual analysis is lacking. Since this paper is purely empirical, the generalizability of the insights is questionable if proper conceptual analysis is missing. For instance, in fig. 3, based on the two plots, it is difficult to reach any conclusions about the influence of conflicting gradient on performance because the two experiments themselves have conflicting results. For this part, an important question remained to be answered is that how conflicting gradient affect training and why in some cases it has less effects. There are a few “jump to conclusions” situations especially in sec. 4.1 and 4.2.

Questions

1. In section 4.1, to obtain the optimal scalarization weight, do you perform grid search? I wonder how you are able to select a single “optimal” weight since firstly, it is impossible to search for the weight exhaustively, and secondly, it is likely that several sets of weights have undistinguishable results. Could you clarify this? 2. In fig. 4(b), it seems that training on a single task is always better than training the two tasks jointly regardless of model selection and weight tuning. Any insights on that? 3. For sec. 5, if possible, it would be interesting to see how much gap PBT closes compared to some oracle (more exhaustive grid search), even on datasets with few tasks/domains. It is unclear how good a performance a static weighting strategy can achieve because the only static baseline is uniform.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have adequately addressed the limitation.

Summary

This paper seeks to better understand the complexities of unitary scalarization for multitask and multi-domain learning. The paper explores the impact of model size, degree of gradient conflict and variations in scalarization weights in order to derive a set of guiding principles for MTL/MDL. The authors also propose to leverage existing population based HP optimization procedures to efficiently search for the best set of scalarisation weights

Strengths

1. The paper presents an expansive set of experiments to understand the impact of model size, multi-task setting (MTL vrs MDL) and task affinity on the performance of the unitary scalarization approach 2. The paper is actionable -- it proposes to leverage pre-existing population based HP optimization approaches to search for the best scalarization weights. 3. The paper is well written and the experimental methodology is described in sufficient (reproducible) detail

Weaknesses

My main issue with the paper is that I am hesitant about specific parts of the experimental methodology. Primarily, in section 4, the experimental procedure is described as tuning HP for single task and then using the best single task HP for all followup multitask experiments. This creates an unfair comparison since it assumes that the HP setting that is best for the single task is also optimal for the MT setting thus bringing the robustness of the results to question. (This is especially considering that, forcing the scalarization weights to sum to 1, means that the effective per-task learning rate is always smaller than for the single task setting) Also, there exists confounding variables for the the experiments on **model capacity and gradient conflict** that are not addressed. 1. Are the models used ResNet models used for Figure 3 pre-trained or learned from scratch ? 2. How were the learning rates for this experiment chosen ? In general, I suspect that there is also a dependence of the degree of conflict (after a reasonable number of epochs like 1 in your case) on the learning rate used. It would be important to see if the degree of conflict (at a fixed model size) -- varies substantially with learning rate and whether this variance is smaller or larger than the variance that comes changing model size at fixed LR. ----- Update ------ Updated score after rebuttal. Thanks for the responses @ Authors I am willing to raise my score if these concerns are addressed. Missing relevant citation 1. Exploration of model capacity and scalarization weight https://arxiv.org/pdf/2302.09650.pdf

Questions

Questions 1. Is the sweep to find the best single dataset HP performed for each model size (For figure 2) or is it performed for 1 model size and then used across all sizes ? 2. For figure 3, how do the hyper-parameters like learning rate differ from for model size 1. Do you use a fixed learning rate across all model sizes ? Or use a pre-determined best learning rate for each model size ? 3. Are the resnets in Figure 3 pre-trained resnets ? 1. It would be interesting to see if the effect still holds for resnets trained from scratch vrs pre-trained on say Imagenet

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

Authors have addressed limitations.

Summary

This work is interested in analyzing the extent to which scalarization is an effective strategy against negative transfer in multi-task and multi-domain learning. Scalarization focuses on selecting an adequate weights for a convex combination of task losses, rather than employing expensive or complex conflict mitigation strategies. Although the search space for scalarization weights grows exponentially with the number of tasks, during training minimizing a weighted sum of task losses is fast and simple compared to many complex optimization methods and has recently been shown to be just as good. This work therefore attempts to better understand the dynamics of scalarization in multi-task models by studying multi-task generalization under scalarization alone. They find a few findings which appear consistent across the settings they consider: the first is that scalarization is more effective as model capacity grows; the second is that uniform scalarization is rarely optimal, implying that for each MTL setting the scalarization weights must be tuned; the final observation is that while gradient conflict can predict e.g. task affinity, it does not predict generalization because model capacity does not observably affect gradient conflict. Using these observations, the authors posit that a strong scalarization approach to MTL can be extremely effective, but the search for the optimal scalars makes it more expensive than other proposed methods. To this end, the authors leverage population based training to efficiently explore the parameter space of task weights. They find that models trained with scalarization weights from PBT outperform the uniform scalarization baseline, as well as several other sota optimization methods.

Strengths

- The paper is well written. It conveys its core ideas and motivation clearly. - The in-depth, rigorous exploration into multi-task learning dynamics is important as recent work has shown that most prior optimization work is not actually beneficial for standard MTL problems. - The observation that uniform weighting is rarely optimal is useful, even if not surprising. - The findings w.r.t. task conflict and generalization are very interesting, and helpful to consider in further development of MTL methods. - In total, the analysis of section 4 could be helpful for the design of future methods which aim to target scalarization, and they serve to motivate the proposed method in section 5. - The proposed method is clearly useful empirically, and demonstrates the effectiveness of scalarization vs. other, much more complex, optimization methods.

Weaknesses

- All 3 key conclusions come from experiments which study only 2 tasks at once. While it is not unreasonable to extrapolate some conclusions from this setting, some conclusions could be at least verified for larger task settings, even up to 3 or 4 simultaneous tasks just to ensure the trends still hold. For example, does model capacity really not affect gradient conflict levels when considering all 40 tasks of CelebA? - The models all use the optimal single-task parameters but this might be unfair to the MTL models, e.g. [1] suggested that the learning rate should scale with the number of tasks if all else is fixed. - The final results should probably use random scalarization [2] as an additional baseline. I find this to be especially true given that the uniform models almost uniformly outperform the other optimization methods, so it is not extremely surprising that additional tuning of the task weights will result in the best performance on the tables. - The comparison to previously considered SOTA methods only goes up to 7 or 8 tasks, whereas many of the methods were tested on e.g. up to all 40 tasks on CelebA. It’s not clear if PBT can efficiently scale up to 40 tasks. To that end, a comparison of the overall compute used by the tested methods would be really helpful. [1] the importance of temperature in multi-task optimization, Mueller et al., 2022 [2] reasonable effectiveness of random weighting: a litmus test..., Lin et al., 2021

Questions

- Do you know how PBT scalarization compares to random scalarization? - How does the entire procedure of PBT compare to other optimization methods w.r.t. total compute time or flops? - Finally, I’m particularly interested in whether or not the trends w.r.t. model capacity and gradient conflict hold as the number of tasks increases. Do you happen to know if they do?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

yes

Summary

This paper analyzes multi-task learning (MTL) and multi-domain learning (MDL) setting. The paper observes several points: 1. MDL/MTL improvements are more significant with bigger network capacity, 2. Gradient conflicts are not necessarily well correlated with MDL/MTL performances, 3. Tuning scalarization weight is important for MDL/MTL performance, 4. population-based training (PBT) can be an efficient way to tune the scalarization weights when there are many tasks. The experimental results support each of their claims, and lastly this paper show that PBT can be even competitive with memory-expensive gradient-based methods, such as PCGrad.

Strengths

- Paper was easy to read.

Weaknesses

[major comments] - overall, almost all observations found by this paper seem quite trivial to me. Currently, I don't think this paper provides very useful insights or something that has not been investigated before. - For instance, in page 2, (C1) is already quite trivial - we already know that larger network capacity can mitigate negative interference because larger capacity means it can accommodate more diverse information. See [1] for the reference. - (C3) is also trivial - we already know that tuning scalarization weights is important and they should be tuned differently for each task/domain/architecture, and so on. - The conclusion of (C2) is misleading, in my opinion. The authors observed that MTL/MDL performance improves with bigger network capacity while the degree of gradient conflict remains the same, and they conclude that gradient conflict do not correlate well with the actual MTL performance in practice (L219-220). This conclusion sounds weird because the network architecture is different. What if the network architecture remains the same and we resolve the gradient conflict, which is the usual assumption of other papers, such as PCGrad? - (C4) is simply an application of the existing technique (PBT) to scalarization weights, which is not very surprising. And the authors did not provide other baselines than Uniform, although there should be many existing methods that allows to carefully tune the scalarization weights. - The same for the conclusion in section 4.1. All (C1), (C2), and (C3) sound obvious to me. - (C1): we already know that larger network capacity can mitigate negative interference effect, as mentioned above. - (C2): Of course the best-performing scalarization weight would not be p1=p2=0.5. - (C3): Of course MTL/MDL has a regularization effect, so the training loss converges slower but the test accuracy is higher. [minor comments] - in (1), the notation $\frac{\nabla}{\nabla}$ looks very weird. It should be either $\frac{\partial}{\partial}$ or simply $\nabla_\theta \mathcal{L}$. - in (2), the definition of $f$ is missing. What is it? (I assume that it's $\nabla_{\theta_i} \mathcal{L}_t(x_t,y_t)$?) - missing baseline - Sequential Reptile [2], which can resolve gradient conflict issue without heavy memory overhead. [reference] [1] Wang et al., On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment, 2020 [2] Lee et al., Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning, 2022

Questions

See the comments above. Overall, I don't think this submission is above the acceptance bar. Most of the observations seem obvious and not very informative. In order for such an analysis-style paper to be accepted, the analysis should be 1. better organized with clear insight, 2. providing novel and useful insights that have not been found by other researchers. ------------------------------------------------------------------ [After rebuttal] I read the author's rebuttal and other reviewer's comments. Unfortunately, I'm still not convinced of the rebuttal, thus I maintain my current score.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

1 poor

Limitations

The authors have properly addressed the limitations of this paper in Sec 6.

Question about PDF uploaded

Hi Authors, Thank you very much for your responses. I have quick question about Fig 1(a) in the PDF that you uploaded. Since the model sizes are not marked on the line, it is really hard to see what is going on here. I guess the question I was trying to have answered with the learning rate vrs capacity question is this : at any fixed point in training (say 50%), if we consider two learning rates $l_1$, $l_2$ that are sufficiently different but not divergent (w.r.t the model), and we consider $\mathrm{model}_1$, $\mathrm{model}_2$ where size(model1) < size(model2), could $$\text{gradconflict}(\mathrm{model}_1, l_1) < \text{gradconflict}(\mathrm{model}_2, l_2)$$ but $$\text{gradconflict}(\mathrm{model}_2, l_2) < \text{gradconflict}(\mathrm{model}_1, l_2)$$ This would mean that the conclusion that larger models have higher conflict would be invalid except when conditioned on a specific choice of learning rate.

Thanks for the rebuttal

Although the review did not engage, I'll carefully read and consider it during the decision period. AC

Thanks for the rebuttal.

Although the review did not engage, I'll carefully read and consider it during the decision period. AC