PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation

Softmax Loss (SL) is widely applied in recommender systems (RS) and has demonstrated effectiveness. This work analyzes SL from a pairwise perspective, revealing two significant limitations: 1) the relationship between SL and conventional ranking metrics like DCG is not sufficiently tight; 2) SL is highly sensitive to false negative instances. Our analysis indicates that these limitations are primarily due to the use of the exponential function. To address these issues, this work extends SL to a new family of loss functions, termed Pairwise Softmax Loss (PSL), which replaces the exponential function in SL with other appropriate activation functions. While the revision is minimal, we highlight three merits of PSL: 1) it serves as a tighter surrogate for DCG with suitable activation functions; 2) it better balances data contributions; and 3) it acts as a specific BPR loss enhanced by Distributionally Robust Optimization (DRO). We further validate the effectiveness and robustness of PSL through empirical experiments. The code is available at https://github.com/Tiny-Snow/IR-Benchmark.

Paper

References (65)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer m2Qo5/10 · confidence 4/52024-06-27

Summary

This paper examines the limitations of softmax loss in recommender systems, focusing on its weak correlation with ranking metrics like DCG and sensitivity to false negatives. The authors propose pairwise softmax loss (PSL), which replaces the exponential function in softmax loss with activation functions such as ReLU, Tanh, and Atan to improve alignment with ranking metrics and enhance robustness. PSL optimizes the score gap between positive and negative item pairs, aiming for a tighter correlation with ranking metrics and improved noise resistance. Empirical evaluations on various datasets under different conditions (IID, OOD, and noisy) indicate that PSL generally outperforms SL and other baseline loss functions in recommendation accuracy and robustness.

Strengths

* The paper is well-organized with clear explanations and presentation of theoretical and empirical results. * The authors provide strong theoretical analysis and comprehensive empirical evaluations on multiple datasets and testing conditions. * The proposed PSL method addresses key limitations of softmax loss, potentially leading to more accurate and robust recommender systems.

Weaknesses

* The proposed activation functions may interact differently with various underlying recommendation models. While the paper tests on a few models (MF, LightGCN, XSimGCL), the results might not hold for other models or architectures, limiting the generalizability of the method. * The method may introduce biases due to the specific choice of activation functions, potentially favoring certain types of data distributions or interaction patterns. * The reliance on temperature scaling for robustness could lead to parameter sensitivity, requiring careful tuning that might not be practical in real-world scenarios.

Questions

* The proposed activation functions (ReLU, Tanh, Atan) may have been tested on models like MF, LightGCN, and XSimGCL, but these models do not cover the entire spectrum of recommendation systems. Different models may have unique characteristics and architectural dependencies that interact differently with these activation functions. As a result, the performance improvements observed in the tested models may not generalize to other recommendation models or architectures. - Have the authors considered testing the activation functions on a broader range of recommendation models? How do the activation functions perform on models with different embedding sizes or layer structures? - The authors could extend their experiments to include a wider variety of recommendation models to validate the generalizability of the proposed activation functions. Additionally, conducting ablation studies to understand the interaction between the activation functions and various model components could provide deeper insights. * The method might introduce biases because the chosen activation functions could favor specific types of data distributions or interaction patterns. For instance, ReLU may perform better on sparse data, while Tanh might be more effective in scenarios with more uniformly distributed interactions. This bias can limit the applicability of the method to a narrower range of scenarios. - What criteria were used to select the specific activation functions (ReLU, Tanh, etc.)? Providing a rationale for the selection of each activation function based on empirical evidence would also strengthen the validity of their choices. * The method relies on temperature scaling to control the influence of different data points, which introduces parameter sensitivity. Finding the optimal temperature setting is crucial for achieving the desired robustness and performance, but this process can be labor-intensive and impractical in real-world scenarios. Moreover, the optimal temperature may vary across different datasets, requiring frequent re-tuning. - I wonder how sensitive are the results to variations in the temperature parameter? The authors should provide guidelines or automated methods for tuning the temperature parameter effectively?

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

* The proposed activation functions have been tested on a limited number of recommendation models, which may not represent the full diversity of models used in practice. This limits the generalizability of the findings. * The specific choice of activation functions may introduce biases, favoring certain data distributions or interaction patterns, which could reduce the method's applicability across diverse datasets. * The reliance on temperature scaling introduces sensitivity to the temperature parameter, requiring careful and potentially impractical tuning for different datasets and scenarios.

Authorsrebuttal2024-08-06

Addtional Rebuttal (Part 2/2)

# Response to Reviewer `m2Qo` (2/2) > **[C2] The method may introduce biases due to the specific choice of activation functions.** **[Response]** We appreciate your concern. We would like to clarify that PSL does not introduce biases. Although we offer three choices of activation functions, PSL consistently outperforms the compared methods with any choice, with few exceptions, as evidenced in Table 1, Table 2, Table A1, Figure 2, Figures C.1-C.4. In practice, we recommend using PSL-relu, as it achieve superior performance in most cases. Moreover, the linear ReLU function is simpler than the others, making it potentially easier and more efficient to optimize. > **[C3] The reliance on temperature scaling for robustness could lead to parameter sensitivity, requiring careful tuning that might not be practical in real-world scenarios.** **[Response]** Thank you for raising this concern. However, it is important to note that PSL does not introduce additional hyper-parameters compared to the classic SL, which also utilizes the temperature parameter. SL has been widely applied in practical recommender systems like Taobao and TikTok. Besides, our empirical experiments on the temperature (presented in the following Table A4) indicate that PSL achieves strong performance requiring a narrow search range for the temperature of $[0.025, 0.05, 0.1, 0.15]$ across four datasets. More importantly, PSL consistently outperforms basic SL at any temperature within this range. This suggests that PSL could serve as a better DCG surrogate loss than SL in practice, without necessitating additional tuning efforts. In practice, when facing scenarios with significant resource constraints that preclude extensive hyperparameter tuning, we recommend limiting the search for the temperature to only two values $\{0.05, 0.1\}$. At this setting, the performance of PSL may be slightly below the optimum but is still significantly better than BPR and SL. In more extreme scenarios, we may even opt to simply set $\tau=0.05$, as this value yields decent performance in the majority of cases. It is important to note that other advanced loss functions, such as BSL and InfoAdvNCE, are less effective in these scenarios because they require the tuning of more hyperparameters to maintain their performance. **Table A4.** Recommendation performance (NDCG@20) with varying temperature $\tau$. |Gowalla|$\tau$ = 0.025|$\tau$ = 0.05|$\tau$ = 0.1|$\tau$ = 0.15| |-|:-:|:-:|:-:|:-:| |SL|0.1543|0.1624|0.1248|0.0980| |PSL-tanh|0.1543|0.1646|0.1308|0.1014| |PSL-relu|0.1541|**0.1647**|**0.1323**|**0.1032**| |PSL-atan|**0.1547**|0.1646|0.1308|0.1013| |Movie|$\tau$ = 0.025|$\tau$ = 0.05|$\tau$ = 0.1|$\tau$ = 0.15| |-|:-:|:-:|:-:|:-:| |SL|0.0755|0.0929|0.0885 |0.0755| |PSL-tanh|0.0750|0.0941|0.0910 |0.0771| |PSL-relu|**0.0755**|**0.0945**|**0.0920**|**0.0783**| |PSL-atan|0.0750|0.0941|0.0910 |0.0771| |Electronic|$\tau$ = 0.025|$\tau$ = 0.05|$\tau$ = 0.1|$\tau$ = 0.15| |-|:-:|:-:|:-:|:-:| |SL|0.0274|0.0360|0.0529|0.0516| |PSL-tanh|0.0292|**0.0417**|0.0535|0.0521| |PSL-relu|0.0292|0.0416|**0.0541**|**0.0526**| |PSL-atan|**0.0293**|**0.0417**|0.0535|0.0522| |Book|$\tau$ = 0.025|$\tau$ = 0.05|$\tau$ = 0.1|$\tau$ = 0.15| |-|:-:|:-:|:-:|:-:| |SL|0.1210|0.1174|0.0807|0.0611| |PSL-tanh|0.1225|0.1207|0.0856|0.0641| |PSL-relu|**0.1227**|**0.1214**|**0.0876**|**0.0658**| |PSL-atan|0.1226|0.1207|0.0856|0.0642| --- We sincerely appreciate your feedback and hope these additional experiments and analyses have addressed your concerns. Please feel free to reach out if you have any further questions or suggestions.

Reviewer m2Qo2024-08-12

Thanks for the authors' responses, which have addressed my concerns.

Reviewer ufck8/10 · confidence 4/52024-07-04

Summary

From the unique characteristics of recommendation systems (RS), this paper highlights the incompatibility of the exponential function in the softmax loss when applied to RS. To address these issues, the authors propose the PSL loss, introducing two main modifications: the replacement of the activation function and the adjustment of the temperature factor's position. These changes result in multiple benefits, which are thoroughly discussed in the paper. Compared to several SOTA loss functions in RS, the proposed method demonstrates substantial improvements in user-item recommendation experiments conducted across three different scenarios.

Strengths

1. The paper is well-written and accessible to readers, with the clearly-stated and intriguing motivation. The topic is fundamental to the field of recommendation systems. The proposed method effectively alleviates existing problems, despite the minimal changes. 2. The proposed PSL loss could serve as a tighter surrogate of DCG compared to SL and could be considered as a DRO-empowered BPR loss. The bound relations with AdvInfoNCE and BPR underscore the sound merits of PSL. The logical coherence of several theorems and lemmas contributes to the comprehensiveness of this work. 3. In the experiments, the detailed parameter settings for all compared methods are provided. The inclusion IID setting, OOD setting and noisy setting are thoughtfully designed to evaluate the applicability of PSL. 4. Though the straightforward implementation, it’s encouraging that the proposed method demonstrates substantial improvements on widely-used datasets compared to several SOTA performances in RS.

Weaknesses

1. The proposed PSL loss is derived from the pair-wise form of softmax loss; however, the role of the original softmax loss in the subsequent discussion is not thoroughly addressed, leading to a minor gap in the exposition. 2. While the paper provides some justification for the chosen activation functions, the theoretical rationale for selecting ReLU, Tanh, and Atan over others is not fully developed. The impact of the chosen activation functions on the stability of the training process is not sufficiently discussed. 3. The paper mentions the use of grid search for hyperparameter optimization but does not provide a detailed analysis of the sensitivity of PSL to the temperature factor. 4. There are a few typos in the paper: such as ‘faciliating’ in line73, ‘Scenerio‘ in line 256, ‘outperforms’ in line 314, ‘furture’ in line 325, , mismatch in line 157 158.

Questions

1. Recommendation systems often suffer from data sparsity issues. How does PSL perform under conditions of extreme data sparsity? Have the authors conducted experiments to evaluate the effectiveness of PSL in such scenarios, and what mitigation strategies, if any, are suggested? 2. Generally, contrastive loss is sensitive to noise (false negative instances). Is it feasible to apply PSL to general contrastive learning? 3. In the experiments, PSL-ReLU consistently achieves sound performance across all settings. Can authors provide any rudimentary insights or intuitive understandings for the selection of activation function for broader application? 4. The introduction of new activation functions in PSL may alter the training dynamics of the backbone models. Apart from performance metrics, are there differences in the objective function values or convergence rates between PSL and SL?

Rating

8

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

See weaknesses.

Authorsrebuttal2024-08-06

Addtional Rebuttal (Part 2/2)

# Response to Reviewer `ufck` (2/2) > **[Q1] How does PSL perform under conditions of extreme data sparsity? Have the authors conducted experiments to evaluate the effectiveness of PSL in such scenarios, and what mitigation strategies, if any, are suggested?** **[Response]** Thank you for raising this concern. Note that PSL is theoretically superior to SL and it does not rely on any assumptions on dataset. Empirically, we have conducted extensive experiments on six datasets with varying sparsity (cf. Tab.B.1). The data density ranges from the highest in Amazon-Electronic (0.00208) to the lowest in Amazon-Book (0.00026), with a nearly tenfold difference. We observe that PSL consistently achieved SOTA performance with varying sparsity. To mitigate the impact of extreme data sparsity, we believe leveraging some cold-start recommendation strategies could be useful like the work [a1, a2]. These strategies can be seamlessly integrated with PSL, requiring only the substitution of their loss functions (e.g., BPR, SL) with PSL. **[References]** [a1] Contrastive Learning for Cold-Start Recommendation, MM'21 [a2] A heterogeneous graph neural model for cold-start recommendation, SIGIR'21 > **[Q2] Generally, contrastive loss is sensitive to noise (false negative instances). Is it feasible to apply PSL to general contrastive learning?** **[Response]** Insightful suggestions! PSL has the potential to be adapted as a loss function for contrastive learning (CL). In the context of CL, for a sample $z \sim X$, where the probability distributions of positive and negative samples $z_i^+, z_j^-$ are $P_z^+, P_z^-$, respectively, PSL can be defined as: $ \mathcal{L}_{\mathrm{PSL}}(z) = \mathbb{E} _ {i \sim P_z^+} \left[\log\mathbb{E} _ {j \sim P_z^-}\left[\sigma(f(z, z_j^-)-f(z, z_j^+))^{1/\tau}\right]\right] $ When $\sigma = \exp$ , PSL degenerates into InfoNCE. We can expect PSL equipped with relu or tanh activation functions, could mitigate the noise sensitivity by avoiding gradient explosion (cf. Fig.1(b)). We plan to further explore this promising topic in future research. > **[Q3] In the experiments, PSL-ReLU consistently achieves sound performance across all settings. Can authors provide any rudimentary insights or intuitive understandings for the selection of activation function for broader application?** **[Response]** In practical, we recommend using PSL-relu directly, as it achieve superior performance in most cases. Moreover, the ReLU function is simpler than the others, making it potentially easier and more efficient to optimize. The reason for the effectiveness of PSL-relu could be explained as follows: as illustrated in Fig.1(a), on the negative half-axis, ReLU is closer to the ideal $\delta$ compared to the others. Note that $d_{uij} = f(u, j) - f(u, i) $ and in many cases the negative score $f(u, j)$ could be smaller than the positive score $f(u, i)$, indicating the $d_{uij}$ are more likely to be located at the negative half-axis. Therefore, PSL-relu acts as a *de facto* closer surrogate to DCG than others in practice, leading to better recommendation accuracy. > **[Q4] The introduction of new activation functions in PSL may alter the training dynamics of the backbone models. Apart from performance metrics, are there differences in the objective function values or convergence rates between PSL and SL?** **[Response]** Thanks for your question. As stated in the Response to [W2], PSL can in fact enhance the training stability by moderating the gradient distribution. We have provided the loss curves of SL and PSL in Figure R1 (in Author Rebuttal), which show that the training process of PSL is stable and converges fast.

Reviewer ufck2024-08-13

Official Comment by Reviewer ufck

Thanks for the responses. I will improve my score slightly after reconsidered this work since most of my concerns are addressed.

Reviewer kuk65/10 · confidence 3/52024-07-08

Summary

The authors re-examine the connection between the Softmax Loss (SL) and the evaluation metric Discounted Cumulative Gain (DCG), highlighting the inadequate tightness of SL as a surrogate loss for DCG. They propose minimal yet effective modifications (PSL-tanh/relu/atan) based on the pair-wise formulation of SL, showing provable tightness and controllable weight distributions.

Strengths

1. The authors provide a unified framework that helps to design a tighter and stronger DCG surrogate loss. 2. Extensive experiments have been conducted in IID, OOD, and Noise settings. Specifically, the proposed PSL-relu loss shows superior performance in both OOD and Noise settings.

Weaknesses

1. In the IID setting, PSL demonstrates only marginal improvement even when compared to the standard Softmax Loss (SL). 2. The authors argue that SL is highly sensitive to noise, while PSL with an appropriate activation function can mitigate this sensitivity. I would expect that as the noise ratio increases, an activation function with a moderate weight distribution should perform better. In other words, PSL-tanh/atan shall be a better choice to SL/PSL-relu if the noise ratio is high. However, this is not the case according to Figures C.1-4, in which PSL-relu consistently outperforms the other methods.

Questions

Why does BSL only keep on par with SL in both IID and OOD settings? It appears that the connection to Distributional Robust Optimization (DRO) determines the performance in OOD settings, but it is not the case for BSL?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have addressed the limitations.

Reviewer VFmg6/10 · confidence 3/52024-07-09

Summary

This paper aims to investigate the effectiveness of softmax loss in the recommendation model. To overcome the limitation of SL, the authors propose a new pairwise softmax loss (PSL). Based on the analysis, the authors argue that replacing exponential function with other active functions can benefit the ranking performance. Experiments on both IID, OOD, and noise datasets prove the effectiveness of the proposed PSL.

Strengths

1. The analysis of softmax loss on the DCG metric is impressive and reasonable. The pair-wise softmax combines BPR loss and activation operation, which is easy to incorporate into the current model. 2. The technique is sound based on well-motivated according to the analysis. 3. The experiment design is reasonable and follows the related work, which verifies the effectiveness.

Weaknesses

1. Overall, the softmax loss is utilized in training and the test stage, which means we can utilize the model's output as the probability. The probability can be used in downstream tasks, such as ECPM calculation in online advertisements. So, replace the operation related to ranking metrics and practical use. 2. The distinction between AUC, Recall, and other metrics should be paid more attention. The motivation behind SPL is prioritized to focus on the DCG metrics. 3. The experiment design is reasonable, but the analysis needs to be further enhanced.

Questions

1. How the SPL affects other metrics, for example, AUC is important in pointwise click scenarios. 2. In experiments, only results of NDCG@20 are given. Results on more cutoffs should be given in the recommendation.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes

Reviewer kuk62024-08-12

Thanks for the authors' response and I will keep my rating.

Reviewer VFmg2024-08-13

Thanks for the authors' response and my concerns are resolved, and I raise my scores.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC