A Unified Framework for Rank-based Loss Minimization

The empirical loss, commonly referred to as the average loss, is extensively utilized for training machine learning models. However, in order to address the diverse performance requirements of machine learning models, the use of the rank-based loss is prevalent, replacing the empirical loss in many cases. The rank-based loss comprises a weighted sum of sorted individual losses, encompassing both convex losses like the spectral risk, which includes the empirical risk and conditional value-at-risk, and nonconvex losses such as the human-aligned risk and the sum of the ranked range loss. In this paper, we introduce a unified framework for the optimization of the rank-based loss through the utilization of a proximal alternating direction method of multipliers. We demonstrate the convergence and convergence rate of the proposed algorithm under mild conditions. Experiments conducted on synthetic and real datasets illustrate the effectiveness and efficiency of the proposed algorithm.

Paper

References (55)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer kXbu4/10 · confidence 2/52023-07-05

Summary

This paper presents rank-based loss as a popular replacement for empirical loss. The work develops how optimization of rank-based loss can be done by a proximal alternating direction method of multipliers. The authors also demonstrate the algorithm's features in terms of convergence under certain conditions. Experimentation which includes synthetic and real datasets is also added in the paper with some numerical simulations showing how the framework behaves. It supports the theoretical results.

Strengths

The paper is mathematically strong

Weaknesses

The contribution doesn't seem significant enough The applicability of the result is not clearly shown

Questions

I would like the authors to compare the method with similar-purpose frameworks, even if not using rank-based loss, and see their differences in performance. I would appreciate the inclusion of the applicability of the result to a real machine-learning problem.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

N.A.

Reviewer G6DJ7/10 · confidence 3/52023-07-05

Summary

This submission focuses on efficient minimization of a group of loss functions called rank-based losses. It proposes to consider several related losses from the perspective of a genral unified framework with a regularizer. Then, focusing on the case of monotone increasing loss functions and wealy convex regularizers, it proposes an ADMM-based algorithm and shows its convergence rate under common assumptions. Furthermore, when the regularizer is non-smooth, it then extends the proposed algorithm with weaker assumptions and also shows its convergence rate. Sufficient numerical verification shows the satisfying empirical performance of the proposed algorithm compared with several existing methods.

Strengths

Originality: - The task provides a new perspective on an important problem and the proposed methods improving ADMM are novel. - Authors clearly address where exactly the improvements from existing methods are. - Related literature review is adequate and detailed. Quality: - The submission is technically rigid. - Claims are well-supported by its clear presentation. Adequate theoretical and empirical results are presented. Clarity: - The submission is clearly written and easy to follow. It is very well organized and the story line expands naturally. Significance: - The results are important to the field. Others are very likely to use the results as a baseline method or build extensions upon it.

Weaknesses

- It would be more pursuative to elaborate on potential practical limitations of the proposed method. For example, under what case the proposed method may not function efficiently.

Questions

- Is equation (1) first proposed by this paper, or it has been somehow mentioned in related publications? - In Figure 1 (h), it seems like existing methods can take advantage of more data at the beginning of optimization. How this trend changes when more samples are available? Is there some theoretical explainable available for the phenomenon?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

No potential negative societal impact needs to be addressed.

Reviewer GkCz5/10 · confidence 2/52023-07-06

Summary

This paper presents a new ADMM algorithm that focuses on three specific cases of rank-based losses. The algorithm's convergence is theoretically analyzed in the paper. Additionally, the authors conducted comprehensive experiments to compare the new algorithm with traditional approaches. The results indicate that the proposed algorithm outperforms traditional methods in terms of both efficiency and effectiveness.

Strengths

1. The authors proposed a new algorithm that can address three specific cases of rank-based losses, whereas traditional algorithms like SGD, DCA, and LSVRG can only handle one. Additionally, the new algorithm permits the regularization term to be a weakly convex function. 2. The authors' theoretical analysis of the new algorithm's convergence is a significant contribution that has not been achieved in previous work. 3. To demonstrate the advantages of the new algorithm, the authors conducted comprehensive experiments. The results show that the new algorithm generally outperforms existing methods in terms of both efficiency and effectiveness. Overall, this study provides valuable insights into the development of more efficient and effective algorithms for rank-based losses.

Weaknesses

Significant Issues (Detailed Explanation Required) 1.The representation of the loss vector function in line 23 is unclear. If function l maps an n-dimensional column vector to a real number, how a comparison of magnitude can be made later on? 2.As is widely known, due to the unknown underlying distribution of the data, we can only optimize the discrete form of risk expectation, which is the arithmetic mean of n observations. The theoretical basis for doing so is that the arithmetic mean converges in probability to the expectation. Therefore, could you please explain the theoretical basis for the discrete form of spectral risk that you employed in lines 78-79. 3.Could you please clarify how the conclusion stated in line 116 was derived? If it is a result obtained from citing other papers, please provide the source. If it is a result you have proven yourself, please provide a detailed proof. 4.In line 159, definitions for two consecutive blocks disorder are provided, but definitions for three or more consecutive block disorder are missing. 5.In line 6 of Algorithm 2, it is not possible to select the disordered blocks because definitions for three or more consecutive blocks disorder have not been provided. 6.In line 6 of Algorithm 2, you assume that the optimal solutions for are equal to each other. Could you please a more detailed proof or a citation if the conclusion is derived from another paper? 7.In line 225, you stated that Assumptions 4 and 5 are weaker than Assumption 2. Please provide a detailed explanation for this claim. 8.Could you please provide specific information on the loss functions and regularizers used in the experiments in Sections 5.2 and 5.3? Minor issue (needs improvement). 1.The "l" in line 23 and the "l" in line 71 have different meanings. It’s suggested to use distinct notations to indicate this difference. 2.In Equation (1), the independent variable of is , where is a d-dimensional column vector. However, the independent variable of is written as an n-dimensional column vector in line 24. 3.Line 71 should use lowercase "d" instead of uppercase. 4.In line 85, please provide a brief explanation of the term "fairness". 5.There is a missing negative sign in line 127. 6.In line 154, "" should be in lowercase. 7.The explanations in lines 169-170 appear to contradict the explanations in lines 187-188. 8.To avoid confusion among readers, please use a consistent notation for the loss function "" in lines 71, 125, and 194. 9.Please provide an explanation for "dist" as used in line 249 at its initial occurrence in line 212. 10.The placement of "Dataset" in Tables 1 and 2 is inconsistent. 11.Please use the notation "new-ADMM" to distinguish the modified version of the traditional ADMM algorithm.

Questions

Please see the above comments about the Weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

None

Reviewer Gyag6/10 · confidence 1/52023-07-06

Summary

This paper proposes a unified framework for rank-based loss minimization based on the ADMM algorithm. The paper proposes to apply a pool adjacent violators (PAV) algorithm to solve one of the subproblems of ADMM. Numerical experiments show that the proposed algorithm outperforms the existing ones.

Strengths

+ The problem of rank-based loss minimization is very important. + The proposed PAV algorithm looks interesting. + Convergence of the algorithm is theoretically analyzed.

Weaknesses

- It seems that Eq.(1) assumes a linear model and it is unclear how to generalize to non-linear settings.

Questions

- Can the proposed method be applied to nonlinear models?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Limitations are not discussed in the paper.

Reviewer kXbu2023-08-16

acknowledgement of having read the authors' reply

I thank the authors for addressing all the points I indicated and providing a reply to them.

Reviewer G6DJ2023-08-16

I thank authors for providing detailed replies for all my and other reviewers concerns. Authos clarified my concerns over the difference of similar equations to Eq(1) in related literatures. The consideration for faster update of existing methods in experiments is very thoughtful.

Reviewer Gyag2023-08-18

Thank you for the response

Thank the authors for the response. The nonlinear extension looks interesting and with this incorporated, the paper overall looks solid to me. I am willing to raise the rating to 6.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC