Linear scalarization, i.e., combining all loss functions by a weighted sum, has been the default choice in the literature of multi-task learning (MTL) since its inception. In recent years, there is a surge of interest in developing Specialized Multi-Task Optimizers (SMTOs) that treat MTL as a multi-objective optimization problem. However, it remains open whether there is a fundamental advantage of SMTOs over scalarization. In fact, heated debates exist in the community comparing these two types of algorithms, mostly from an empirical perspective. To approach the above question, in this paper, we revisit scalarization from a theoretical perspective. We focus on linear MTL models and study whether scalarization is capable of fully exploring the Pareto front. Our findings reveal that, in contrast to recent works that claimed empirical advantages of scalarization, scalarization is inherently incapable of full exploration, especially for those Pareto optimal solutions that strike the balanced trade-offs between multiple tasks. More concretely, when the model is under-parametrized, we reveal a multi-surface structure of the feasible region and identify necessary and sufficient conditions for full exploration. This leads to the conclusion that scalarization is in general incapable of tracing out the Pareto front. Our theoretical results partially answer the open questions in Xin et al. (2021), and provide a more intuitive explanation on why scalarization fails beyond non-convexity. We additionally perform experiments on a real-world dataset using both scalarization and state-of-the-art SMTOs. The experimental results not only corroborate our theoretical findings, but also unveil the potential of SMTOs in finding balanced solutions, which cannot be achieved by scalarization.
Paper
References (53)
Scroll for more · 38 remaining
Similar papers
Peer review
Summary
In recent years, there has been a surge in papers suggesting Specialized Multi-Task Optimizers (SMTOs). These papers show the empirical advantage of using SMTOs compared to linear scalarization (LS). However, recently, there have been several papers that suggested that LS with proper tunning can match SMTOs performance. The paper analyzes scalarization from a theoretical perspective, and studies whether scalarization is capable of fully exploring the Pareto front in linear MTL. It reveals some inherent limitations of LS for MTL optimization.
Strengths
1. The paper addresses an important open question in MTL optimization. 1. The paper is (mostly) well-written and well-structured. 1. The paper provides an in-depth theoretical analysis with implications for both MTL researchers and practitioners.
Weaknesses
1. A primary argument is that LS cannot reach a point in the intersection of surfaces. However, it is not clear from Section 3.2 why this is the case. Please elaborate on this point and revise the manuscript accordingly. 1. Please discuss the main results and novelty w.r.t the known result ([1]), which states that LS cannot reach non-convex parts of the Pareto front. 1. It would be beneficial to revise Section 3, by either breaking it into two sections or summarizing the main results at the beginning of the Section. 1. Please add an experiment with the proposed randomization approach to empirically verify that it can explore the entire Pareto front. 1. Missing citations in related work: 1. SMTOs: - Multi-Task Learning as a Bargaining Game, ICML 2022. - Towards Impartial Multi-task Learning, ICLR 2021. - Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models, ICLR 2021. 1. Exploration of the Pareto front: There’s a line of work for methods trying to learn the entire Pareto front, so it is important to mention it. The two pioneering papers for Pareto front learning are: - Learning the Pareto Front with Hypernetworks, ICLR 2021. - Controllable Pareto Multi-Task Learning, 2021. [1] S. Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimization. 2004.
Questions
1. How realistic is the case $q<k$ for real-world applications?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
Addressed.
Summary
This paper is related to multi-objective optimization area. They post a research question: if linearly weighting multiple objectives can fully explore the Pareto front? Through theoretical analysis and a simple experiment, the answer is negative. Hence, this might prove that multiple gradient descent algorithm is inherently better than linear objective weighting.
Strengths
Recently, many report that linear weighting is no worse and even better than gradient balancing algorithms like multiple gradient descent algorithm (MGDA). However, these findings are empirical, This paper targets on an important question: if multiple gradient descent algorithm is inherently better than simple linear weighting when conflict gradients are presenting. Through theoretical analysis and a simple experiment, the answer is yes, which lay a good foundation for this area.
Weaknesses
The MTL model is this paper is very simple. I am not sure if this could reflect the real problem in the industry system. This model only have one layer of shared layer and one layer of task-head. And this is a linear neural network. (I am not sure if pure linear can be called neural network). So, seems this MTL model is a convex optimization. However, in real industry systems, there are many layers (maybe millions of parameters) with non-linear transformation and hence a non-convex optimization.
Questions
1. if MTL has different objectives like one regression task and one classification task, will your conclusion hold? 2. If MTL has non-linear transformation like Sigmoid, will your conclusion hold? 3. Do we have practical usage of randomization for linear scalarization?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
No negative social impact
Summary
This paper revisits linear scalarization from a theoretical perspective. The authors study multi-task learning with a two-layer linear network and reveal a multi-surface structure of the feasible region. They show the necessary and sufficient conditions for full exploration in the under-parameterized regime. Their theoretical results imply that linear scalarization has fundamental limitations and that scalarization tends to overfit a small fraction of tasks. They answer some open questions proposed in Xin et at.(2022) and also provide experiments on a three-task learning problem to verify the theoretical results.
Strengths
- It is a novel and interesting problem to study whether linear scalarization can fully explore the Pareto front. The authors provide necessary and sufficient conditions for this in the under-parametrized regime, which is the main contribution of this paper. - The notations and presentations in this paper are clear and the proofs seem to be correct. - They empirically study the validity of the theoretical results, and the experiments are reasonable, and the results are convincing to me.
Weaknesses
- The authors study only two cases (q = 1 and q = k - 1) in the under-parametrized regime, but are these two extreme cases representative for other cases in the under-parametrized regime? - This paper discusses linear scalarization in the under-parametrized regime. However, over-parameterization is common in deep learning, where the network has sufficient capacity to adapt to the target tasks. Does this mean that linear scalarization in the overparameterized regime does not have the fundamental limitations discussed in this paper? - I commend the authors for attempting to answer some open questions in Xin et al. (2022), but it lacks an explanation for why SMTOs have no improvement over linear scalarization in deep multitask learning, which is found in Xin et al. (2022).
Questions
Please see the weaknesses.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The theoretical analysis in this paper is only applied to the linear network and under-parametrized regime, which is a limitation of this paper.
Summary
This paper studies the linear scalarization approach in multi-task learning (MTL). It shows theoretically that the linear scalarization is not able to fully capture the Pareto optimal (PO) solutions. It also identifies necessary and sufficient conditions of full exploration of the PO solutions for under-parameterized two-layer linear MTL models. This explains why scalarization fails in certain cases. Experiments are performed to further verify the theoretical findings.
Strengths
1. The paper studies an important and timely problem, focusing on whether the popular approach -- linear scalarization for MTL are able to fully capture the Pareto front. 2. The perspective of this paper and techniques to develop the theory under the specific two-layer linear models are unique. 3. Paper is well written and easy to follow.
Weaknesses
1. There is a lack of discussion of related works that also identifies the failure cases of linear scalarization fully capturing the Pareto front. See [1,2,3,4]. What is the unique contribution of this paper compared to these prior works? Some discussions need to be provided to distinguish with the prior works. --- [1] Lin, Jiguan G. “Three Methods for Determining Pareto-Optimal Solutions of Multiple-Objective Problems” [2] Zadeh LA, "Optimality and non-scalar-valued performance criteria" [3] Goicoechea A, Hansen DR, Duckstein L, "Multiobjective decision analysis with engineering and business applications". [4] R. Timothy Marler, Jasbir S. Arora, "The weighted sum method for multi-objective optimization: new insights" [5] Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, Qiang Liu, "Conflict-Averse Gradient Descent for Multi-task Learning" --- 2. In line 56-57, it is too restrictive to claim that "if scalarization cannot fully explore the Pareto front, there is no inherent advantage of SMTOs over scalarization." In fact, there are other benefits of SMTOs beyond this. For SMTO methods like MGDA, the motivation is to find the steepest common descent direction for all objectives at each iteration. And the benefit sometimes lies in the optimization procedure besides the solutions they find. See the toy example in [5] 3. Even though the paper shows that linear scalarization approach is not able to fully explore the Pareto set in some settings, there is no discussion on whether SMTOs are able to fully explore the Pareto set in the same settings. 4. Some questions need to be clarified. See **Questions** -----------------------------update post rebuttal------------------------------- I have read the rebuttal and participated in the discussion. The rebuttal partially addressed my concerns so I remained relatively positive about this work. It could further improve with a more thorough discussion of the classical works and more precise positioning of their contributions. --------------------------------------------------------------------------------------
Questions
1. I do not agree with the claimed contribution 2, line 63-65, where the authors claimed to give the "first guarantee for full exploration in the presence of non-convexity." - Please see reference [1], where necessary conditions of Pareto optimality are given for the linear scalarization method with "p-directionally convex" requirement of the criterion space, which is not convex. - Please see reference [2,3], where sufficient conditions of (weak) Pareto optimality are given for the linear scalarization method. - Please discuss the difference and relations with the listed prior works. Perhaps you need to add the specific settings, such as "for under-parameterized two-layer linear networks". 2. Since this paper is focused on the underparameterized model with $q < k$, how does the theory in this paper helps understanding the question posed in Xin et al. (2022), which is for deep models usually overparameterized? 3. What are the implementation details for visualization in Figure 1? 4. In line 170, what does "gradients disagree" mean? Does it mean they are not in the exactly same direction or the angle between the two gradients is larger than a threshold? There should be a formal definition. --- [1] Lin, Jiguan G. “Three Methods for Determining Pareto-Optimal Solutions of Multiple-Objective Problems” [2] Zadeh LA, "Optimality and non-scalar-valued performance criteria" [3] Goicoechea A, Hansen DR, Duckstein L, "Multiobjective decision analysis with engineering and business applications". --- ### Minor Typos and grammar errors: Line 2: "since its inception" -> "because of its inception" Line 96: "vecotrs" -> "vectors"
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
The authors have adequately addressed the limitations in Section 5. I expect the authors to conduct a thorough review with the most relevant works, see Weaknesses-1. Overall, this paper studies an important problem with a unique perspective. I am willing to increase my score if the authors can address these questions.
Summary
This paper revisits linear scalarization in multi-task learning from a theoretical perspective. The authors reveal that scalarization is, in general, incapable of tracing out the Pareto front. Specifically, when the model is under-parametrized, a multi-surface structure of the feasible region is revealed by the authors and it can be used to identify necessary and sufficient conditions for full exploration. Experimental results further verify the theoretical findings.
Strengths
1. This paper provides rigorous analysis to answer an important question in multi-task learning: whether linear scalarization is capable of fully exploring the Pareto front? The authors provide insightful remarks for the theories, and the figures provide intuitive explanations for the multi-surface structure of the feasible region. 2. The paper is well-written and easy to follow. Sufficient preliminaries and explanations are provided.
Weaknesses
1. The paper only studies the under-parametrized regime. It remains unclear whether the conclusion still holds for over-parametrized regimes where the model capacity is large enough and task competence does not exist. 2. The paper mainly focuses on a two-layer neural network. Therefore, it remains unclear whether the conclusion still holds for non-convex functions.
Questions
Please check the weakness.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Yes.
Summary
In this paper, the authors introduce a novel mathematical framework to answer the problem “Under what conditions can (or cannot) linear scalarization recover the Pareto front of a given multi-objective optimization problem?”. To this end, the paper considers the multi-task learning setting using a simple linear multi-task learning model in the under-parameterized regime, and introduce new concepts like “feasible regions” and "gradient disagreement", and state that the intersection points of aforementioned feasible regions are unattainable by optimizing any linear scalarization of the multiple objectives. Accordingly, the paper derives necessary and sufficient conditions as to when optimizing any linear scalarization of the multiple objectives can recover the Pareto front by checking for situations where the Pareto front of the problem lies entirely in one so called feasible region. The paper then provides some empirical validation of the theory that suggest linear scalarization indeed fail to cover the Pareto front, especially the “balanced” Pareto optimal points. The empirical results further suggest that Specialized Multi-Task Optimizers (SMTOs) like MGDA and MGDA-UB can achieve a more “balanced” Pareto optimal points compared to linear scalarization.
Strengths
* The paper is aimed at finding necessary and sufficient conditions for when scalarization can not recover the Pareto front, which is an important and pertinent problem in the field of multi-objective optimization. * This paper provides a mathematical framework for analyzing the aforementioned problem, which may provide insights into how to select a method of solution for a multi-objective optimization problem based on the nature of the problem.
Weaknesses
* The necessary and sufficient conditions derived for linear scalarization to recover the Pareto front in this paper requires Pareto optimal objective vectors to be “on the same feasible surface”, rather than being at an “intersection point of feasible surfaces”, as described in Lemmas 3.4, 3.5. Yet, it is unclear why these intersection points cannot be attained by some scalarization of the objectives. Specifically, the “gradient disagreement” concept is unclear; for example how to mathematically characterize this gradient in the objective space, and why this gradient is related to the optimization of the multiple objectives. * It is unclear what is the origin ((0, 0, 0) point ) of the objective spaces used in the Figures 1 and 3. Thus, it is a bit hard to understand the provided illustrations of feasible regions in the objective space. * While the paper provide some weakness (i.e. the inability to recover the whole Pareto front of a problem) of scalarization in the under-parameterized regime, it does not give theoretical validation whether SMTOs can overcome this weakness. The empirical validation for showing that SMTOs can overcome the aforementioned weakness have some limitations, as described in following points. Thus, the conclusion of the paper seems to be not justified from the theoretical and empirical results. * The experiments done to verify theory seems inconsistent for SMTOs and linear scalarization methods. Specifically, SMTOs are optimized using iterative methods for 100 epochs, while scalarization solutions are obtained from the corresponding closed form expression for the optimum solution. A fair comparison of methods would be either running SMTO methods for a very large number of iterations, or using an iterative updating scheme for scalarizations for the same number of epochs. * By nature of the MGDA algorithm, if deterministic gradients are used, the algorithm should terminate at the first encounter of a Pareto stationary point (a notion of stationarity for multiple objectives similar to stationarity in single objective optimization), which is often a point either at least one objective have achieved stationarity, or the point where the objectives begin to conflict with one another (see some discussion on this here [1]). Thus, this kind of point is usually a point that will favor one objective over the others (unless the objectives are perfectly aligned). In this sense, it is unclear how MGDA have achieved an interior point of the Pareto front, as implied by Figure 3. * At the beginning of the paper (such as in the abstract) the authors pose the weakness of scalarization over SMTOs as the inability to “fully recover” the Pareto front, while the empirical evaluations suggests that SMTOs can also only recover a single Pareto optimal point, which seems to not align with the initial message. Minor comments * Definition of Pareto front introduced in the paper seems not standard (line 119). Specifically, in this paper it is referred to as the collection of Pareto optimal points (PO) $\theta^*$, while usually Pareto front is the set of objective values $\\{L_i(\theta^*)\\}_{i\in[k]}$ corresponding to Pareto optimal points. [1] Liu, Xingchao, Xin Tong, and Qiang Liu. "Profiling pareto front with multi-objective stein variational gradient descent." Advances in Neural Information Processing Systems 34 (2021): 14721-14733. Edit: Added missing reference.
Questions
* What is the mathematical definition of the gradients depicted in Figure 3, and what is the precise relationship between the “gradient disagreement” and the inability to recover this point by optimizing a linear scalarization of the objectives? * Could the authors provide some intuition or some physical meaning of the feasible regions described in the paper, and how they relate to the multiple objectives? * Would an implementation change of the experiments as described in the previous section (point 4) change the solution distribution of each algorithm compared to Figure 3? * Given the nature of MGDA algorithm as described in previous section (point 5), could the authors provide the reason for the convergence of MGDA like algorithms for the interior point of the Pareto front of the problem?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
2 fair
Contribution
2 fair
Limitations
Limitations of the proposed framework are discussed discussed in the paper. In addition to these limitations, the paper does have limited validation as to SMTOs can overome the weaknesses of linear scalarization, as described in previous sections.
Thanks for the response, I believe it is an interesting paper, and I still have some concerns: 1. Is the under-parametrized regime commonly used in recent works on multi-task learning? Is it meaningful to show that linear scalarization is not sufficient in this regime? 2. The authors study only two cases ($q = 1$ and $q = k-1$) in the underparametrized regime. I understand that it is difficult to study the case where $q$ is not equal to $1$ or $k-1$ in general, so the authors assume that the result changes slightly when $q$ varies. However, can you state this assumption explicitly in your theoretical analysis? Why is this assumption reasonable?
Thank you for the response. Regarding your first concern, we have two comments: - From a theoretical perspective, studying the under-parametrized regime is necessary for linear MTL. Otherwise, the model will have sufficient capacity to fit all tasks perfectly, and there is essentially no conflict between different tasks. In other words, it is only within the under-parametrized regime that the 'competence among difference tasks' can be accurately reflected, which we believe to be a core of MTL. We refer to the reviewer to a prominent work [1], whose main result (Proposition 3) is derived within the under-parametrized regime. - On the empirical side, it is pointed out in the literature (see [2]) that under-parameterized models are actually favorable in MTL, as they can help with information transfer and lead to better generalization. Based on the above points, we believe our study will be of interest to both theorists and practitioners in MTL. Regarding your second concern, we will elaborate further on our previous response: - We believe a larger $q$ (which implies larger model capacity) should intuitively help in our problem. We have two observations to support this argument: 1) when the model has sufficient capacity to fit all tasks (i.e., $q \ge k$), the Pareto front reduces to a singleton and can be explored by scalarization; 2) the necessary and sufficient condition under $q=1$ ($C_1$) is more restrictive than the condition under $q=k-1$ ($C_{q-1}$). Specifically, under the simplified probabilistic model discussed in Section 3.3, we can prove that the probability such that $C_1$ holds is strictly smaller than that of $C_{q-1}$. - Therefore, for the case of general $q$, we expect the necessary and sufficient condition to be more restrictive than $C_{q-1}$ while being less restrictive compared to $C_1$. Since $C_{q-1}$ still does not hold in general, we expect the hardness of full exploration to be true for all $q \in [1,q-1]$. - Finally, we believe it is more important to reveal 'why' scalarization fails in terms of full exploration. Our key observation is that the multi-surface structure of the feasible region, along with the associated phenomenon of 'gradient disagreement', leads to the failure of full exploration. While it may not be obvious that the conclusion should change smoothly with $q$, we believe it is reasonable to expect a smooth change of the feasible region from a geometric perspective (i.e., that the feasible region will not abruptly collapse into a single surface as we increase $q$). This hypothesis, combined with our observation that gradient disagreement leads to the failure of full exploration, yields the conclusion for general $q$. We leave the rigorous analysis to future work. Again, we would like to thank Reviewer 2hHW for engaging in the discussion. We will make sure that the above points are adequately addressed in the revision, and will be happy to answer further questions. References [1] Wu, Sen, Hongyang R. Zhang, and Christopher Ré. "Understanding and improving information transfer in multi-task learning." In ICLR 2023 [2] Wang et al., "Can Small Heads Help? Understanding and Improving Multi-Task Generalization". In WWW 2022
Thank you for the time and effort in reviewing our paper!
I thank the authors for the clarifications, specially on the concepts of the feasible regions and the gradient disagreement. I have few follow up questions and comments based on these responses. * In the 2D objective space figure provided in the link, is it possible to point out the corresponding feasible regions and the corresponding intersection point of these feasible regions? * As mentioned in point 4 of the response, did the authors consider “artifacts of optimization procedure” might also affect (possibly in a favorable way) the SMTO methods? * In the caption of Figure C in pdf, it says “Both algorithms tend to find solutions located in the interior of the feasible region”? Does this mean these are not necessarily in the Pareto front? * Regarding point 5 of the response, the reference is not for some theoretical results on why MGDA can’t converge to the interior point on Pareto front, rather for empirical results, similar to what this paper is trying to provide. Some of the locations in the paper that discuss and demonstrate this issue are: Figure 1 and Example 1 (a simple toy example), Figure 2 (b) (stochastic version of MGDA, applied for several datasets). * I feel the setting used in this paper is significantly incomparable to that of Xin et al. (2022) to imply that results in this paper can dispute the claims in Xin et al. (2022), as suggested in point 6.
Thanks for your response. We will answer your questions in a point-by-point manner: - **Feasible region.** As stated in the caption of the figure, the shaded region (denoted as $\mathcal{O}$) is the set of achievable values. Rigorously speaking, the feasible region that we have defined is a subset of $\mathcal{O}$, but these two can be treated equally in our setting. The reason is that the feasible region contains the entire Pareto front (PF) (explained at the beginning of Section 3.1), and since our interest lies in the exploration of PF, ignoring the points that are Pareto dominated will not affect our analysis. - **Intersection point.** The figure comes from a classical textbook [1] and demonstrates a *hypothetical* failure case of scalarization. In contrast, the intersection point is a new discovery in a concrete setting and hence cannot be reflected in the figure. As stated in the paper, what we have done partly is to 'penetrate through the surface and gain a holistic view of the feasible region'. By solely examining the surface, it is not possible to identify the intersection points or the phenomenon of gradient disagreement. We believe this is one of our core contributions: our work advances previous studies by offering new insights into the failure modes of scalarization, going beyond mere hypothetical examples. - **Artifacts of optimization procedure.** This is not likely, especially in light of the additional experiments that we have performed. Specifically, we stick to the original implementation of MGDA and MGDA-UB with no modifications at all. To eliminate the effect of random initialization, we performed additional experiments and observed similar results (see Figure C of the attached PDF). Finally, we observed consistent results when varying the tasks, further consolidating the effectiveness of MGDA and MGDA-UB in our setting. - **Interior point.** We are sorry for the confusion. We intend to claim that the two algorithms tend to find solutions located at the interior of the PF, which is a part of the *surface* of the feasible region. As stated previously, the feasible region contains the entire PF, and the PF must lie on the surface of the feasible region. Furthermore, MGDA and MGDA-UB are guaranteed to converge to Pareto optimal solutions (Theorem 2.2 of [2] and Theorem 1 in [3]). Therefore, the two convergent points must lie on the surface of the feasible region and belong to the PF. In our experiments, we also plot non-negative orthants at the two convergent points and observed that they do not intersect with the feasible region, meaning that they are indeed Pareto optimum. - **Convergence of MGDA.** We agree it is possible to construct examples where MGDA converges to bad solutions. As stated in our previous response, we don't think one SMTO algorithm can work in every scenario, and the choice of algorithm should depend on the problem structure and specific requirement. Nevertheless, since our experiments are performed in a more practical scenario using real (despite simple) neural networks, and the experimental results are consistent across difference random seeds and tasks, we believe it is reasonable to claim that 'certain SMTO methods have the potential to find balanced solutions'. After all, it is not our goal to advocate for a particular algorithm; what we hope is to bolster research in SMTO and promote a healthier and more balanced developement in MTL. - **Xin et al. (2022).** We do realize that the difference of settings makes the results in these two works not directly comparable. Nevertheless, we would like to clarify a few points: - We do not intend to refute all the claims and conclusions in Xin et al. (2022), but we do feel that there are *some* reasoning gaps in their work that we would like to point out and clarify. Specifically, it is marked bold in their paper that 'in the convex setting it is provable that no algorithm can outperform scalarization', which the authors used to set the tone for their paper. However, their experiments are performed with deep neural networks, and the authors simply connect the theory and experiments by listing a number of open questions. Through our theoretical analysis, we found that the conclusions in the convex setting do not transfer. Therefore, the theoretical results in Xin et al. (2022) cannot be used to support their empirical findings. Instead, the community needs to develop a new theory to explain the empirical success of linear scalarization. - We see our work as a complement to Xin et al. (2022). Putting together, they provide a more comprehensive view on the strengths and weaknesses of linear scalarization and SMTO. We thank the reviewer for pointing this out and will tune down our tone in drawing comparison with Xin et al. (2022). The remaining points will also be addressed accordingly. We appreciate the reviewer's constructive feedback for our work, and will be happy to answer any further questions.
Thanks for the response and for providing additional results. - **Comparison with [1]:** I meant the results in Ch 4.7 of [1], specifically, please see Ch. 4.7.4. Note that Linear Scalarization is referred to as Scalarization. - **Point raised by Reviewer nBcV w.r.t Xin et al. (2022):** I agree with the point raised by Reviewer nBcV regarding the results w.r.t Xin et al. (2022). The theoretical results here are limited in terms of applicability to real-world scenarios and modern, over-parametrized MTL networks. This does not diminish or damage the value and importance of the analysis in the paper, but the limitations in immediate empirical implication should be made clear. [1] S. Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimization. 2004.
Thanks for your response. Regarding the first point, we believe the results in Ch. 4.7.4 have been well summarized in Xin et al. (2022) (see Theorem (Informal) in P3). Specifically, 1) any solution to the scalarization objective with positive coefficients is Pareto optimal; 2) when the objective functions are convex, the Pareto front can be fully explored by linear scalarization. In contrast, our theoretical results differ significantly from prior works because 1) they are established in the non-convex setting, which demands fundamentally different techniques compared to those used in convex analysis; 2) the conditions for full exploration that we have uncovered are both necessary and sufficient, which helps us understand the weakness of scalarization; 3) finally, we build up a theory (the multi-surface structure and the phenomenon of gradient disagreement) to explain why scalarization fails. This significantly advances previous studies by offering new insights into the failure modes of scalarization, going beyond mere hypothetical examples (e.g., Figure 4.9 in the textbook). We agree with the reviewer that it is important to contrast our results with prior art in order to showcase their significance and novelty. We will make sure that the above points are adequately discussed in the revision. Regarding the second point, we would like to clarify that we do not attempt to completely refute the claims and results in Xin et al. (2022), and we understand the difference in the settings makes the results in these two papers not directly comparable. Instead, below are our standing points: - We feel there are some reasoning gaps in Xin et al. (2022) that we would like to point out and clarify. Specifically, as shown in our paper, the positive results in the convex setting do not transfer. Therefore, the theory they developed cannot be used to support their experiments, which are performed with deep neural networks. By pointing this out, we call for a need from the research community to develop a new theory to explain the empirical success of linear scalarization. - We don't think one single SMTO algorithm can fix the issue of linear scalarization, and it is not our goal to advocate for a particular algorithm like MGDA. Instead, we hope to refute the claim that linear scalarization is sufficient for MTL, and bolster research in the development of SMTO algorithms. One of our central goals is to promote a healthier and more balanced algorithmic developement in the field of MTL. - We view our work as a complement to Xin et al. (2022) (as well as a few other works). Putting together, they provide a more comprehensive view on the strengths and weakness of linear scalarization and SMTO. We thank the reviewer for emphasizing this point. We will make sure to discuss the above points as well as the limitation of our empirical evaluations in the revision. Again, we would like to thank the reviewer for taking the time to review our paper and joining the discussion. We will be happy to answer any further questions.
reply to rebuttal
Thanks for the reply. I agree, although the network is simple in this paper, this is still an important step towards explaining the advantage of SMTO over linear weighting
So I raised the rating from 4 to 5
Thank you for taking the time to review our paper!
Thank you for the additional clarification. While the paper may not exactly answer the practical issues in MTL in recent literature, I acknowledge the theoretical framework introduced in the paper is interesting, albeit the setting considered might be very restrictive. Thus, I raise my score.
Thank you for the recognition in our work, and again, for taking the time to review our paper and providing constructive feedback. We will soon compile a list of changes according to yours and the other reviewers’ suggestions in the general response.
Discussion of related works
Thanks for the rebuttal. Regarding the concern of **Comparison with prior works (Weakness 1 and Question 1)**, there seems to be some confusion in definitions. I will first clarify some definitions of **sufficient and necessary conditions for (weak) Pareto optimality (PO)** that are used in prior works. For easy reference, there is a survey paper [5, Section 2.3] that provides all the definitions. 1. If a formulation (e.g. linear scalarization) provides a **sufficient condition for PO**, its solutions (given all possible scalarizations) are always Pareto optimal, though it may not cover all Pareto optimal points. The solution set is a subset of the Pareto optimal set. 2. If a formulation provides a **necessary condition for PO**, then a Pareto optimal point must be a solution to the formulation, though some solutions may not be Pareto optimal. The Pareto optimal set is a subset of the solution set. To ensure linear scalarization fully explores the Pareto front (sufficient condition for full exploration), it means the PO set is a subset of the solutions of linear scalarization, the **necessary condition for PO**. [1] provides necessary conditions of PO, or sufficient condition for full exploration, and [2,3] provide sufficient conditions for PO. So I think [2,3] is not the same as the condition studied in this paper. Thank you for your response! And it would be better if you can have more discussion on these prior works. --- [5] R.T. Marler and J.S. Arora, "Survey of multi-objective optimization methods for engineering"
Thank you for the clarification. We now have a better understanding of what "sufficient/necessary conditions for PO" mean. Apart from [1] which was already included, we will further add these results to the related work section---the discussion below Eq. (3) in [2], Theorem 1 in [5], and the discussion below Eq. (4.60) in [6] (another sufficient condition for PO we are aware of). Unfortunately, we failed to access [3] online; we appreciate it if the reviewer could point us to an accessible source. We thank the reviewer once more for bringing these classical results to our attention, and will make sure to give proper credits to them in our revision. Among the vast literature that concerns Pareto optimality in multi-task learning, one thing that we find missing is a systematic and rigorous study of *when and why scalarization fails to achieve a specific Pareto optimum*. Prior work typically resorts to *hypothetical examples* (e.g., Figure 4.9 in [6]), or applies intuitive yet theoretically-unjustified descriptions (e.g., "the non-convex regions of Pareto fronts" in [Zhang 2023](https://openreview.net/pdf?id=M8rwWdaGa6x)) to demonstrate the failure modes of scalarization. Our study---identifying the multi-surface structure of the feasible region and uncovering the phenomenon of gradient disagreement---serves as a first step towards filling this important gap in the literature. We hope this further underscores the significance of our work, beyond providing a more balanced and comprehensive view on the strengths and weaknesses of scalarization and SMTO. Reference [6] Boyd, Stephen P., and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Plan of revision
Dear reviewers, We are grateful to all of you for taking the time to review our paper, for engaging in the subsequent discussions, as well as for providing constructive feedback. We believe the peer-review process has substantially improved the quality of our work. In what follows, we will list the changes that we intend to make on the current manuscript: - When answering the open questions from Xin et al. (2022) in Section 3.3, we will include a paragraph to clarify that: 1) we do not intend to refute all their claims; rather, we would like to point out the reasoning gap between their theory and experiments, and call for a new theory from the research community to explain the empirical success of linear scalarization; 2) the difference in settings makes the results from these two papers not directly comparable; nevertheless, they complement each other by providing a more comprehensive view on the strengths and weaknesses of scalarization and SMTO. - For Section 4, we will add the experiments for randomization and MGDA with different initializations. We will also clarify that 1) it is not our goal to advocate for a particular SMTO algorithm, and we don’t believe one single algorithm can work in every scenario; 2) instead, by revealing the limitation of scalarization and potentials of some SMTO algorithms, we hope to bolster the research of SMTO, to refute the claim that linear scalarization is sufficient for MTL, and to promote a healthier and more balanced development within the field of MTL. - We will clearly define and explain the concept of gradient disagreement, and will provide detailed reasoning steps on why and how it leads to the failure of exploration at intersection points. - We will include more discussions on the limitation of our work (mostly from a technical perspective), specifically on the under-parametrized regime and general $q$. - We will make sure all the relevant references that the reviewers have mentioned are properly cited and discussed. - We will address other minor points, e.g., issues with figures (implementation details and captions), overclaiming, structure of Section 3, etc. We kindly request the reviewers to let us know if there are other changes that you would like to see in the revision. Sincerely, Authors of Submission 5214
Thank the authors for addressing my concerns. Although the results may not hold in the over-parameterized regime, this paper still provides some new insights into the weakness of linear scalarization and the strength of SMTO methods. I prefer to increase my score.
Thanks for your acknowledgment of our response and taking the time to review our paper!
Decision
Accept (poster)