Sorting with Predictions

We explore the fundamental problem of sorting through the lens of learning-augmented algorithms, where algorithms can leverage possibly erroneous predictions to improve their efficiency. We consider two different settings: In the first setting, each item is provided a prediction of its position in the sorted list. In the second setting, we assume there is a"quick-and-dirty"way of comparing items, in addition to slow-and-exact comparisons. For both settings, we design new and simple algorithms using only $O(\sum_i \log \eta_i)$ exact comparisons, where $\eta_i$ is a suitably defined prediction error for the $i$th element. In particular, as the quality of predictions deteriorates, the number of comparisons degrades smoothly from $O(n)$ to $O(n\log n)$. We prove that the comparison complexity is theoretically optimal with respect to the examined error measures. An experimental evaluation against existing adaptive and non-adaptive sorting algorithms demonstrates the potential of applying learning-augmented algorithms in sorting tasks.

Paper

References (44)

Scroll for more · 32 remaining

Similar papers

Peer review

Reviewer gjeT7/10 · confidence 3/52023-06-23

Summary

This paper presents sorting algorithms augmented by prediction under two settings: (i) sorting with positional predictions and (ii) sorting with dirty and clean comparisons. In (i), the algorithm can use a predicted position $\hat{p}(i)$ of each element $i$, and in (ii), the algorithm is provided a faster but possibly inaccurate comparison oracle besides a slow but accurate comparison. In both settings, the authors present provably faster algorithms making use of the positional predictions/dirty comparisons. They also show that the complexities of the presented algorithms are asymptotically optimal. The efficacy of the algorithms is validated through numerical experiments on synthetic and real-world datasets.

Strengths

The two settings introduced in this paper are well-grounded in real-world scenarios, demonstrating their relevance and applicability. The algorithms proposed are elegant and straightforward, contributing to their practical utility. I also find the lower-bound results commendable. The paper is well-composed, particularly the introduction, which provides an accessible entry point for readers unfamiliar with algorithm-with-predictions research. Therefore, my overall assessment leans towards a positive evaluation of this paper.

Weaknesses

- The sorting algorithm with dirty comparisons is randomized and takes $O(n \log n)$ comparisons, as explained in Section 5. The contribution of this paper will be stronger if the algorithm is derandomized and/or the worst-case time-complexity bound is improved to $O(\sum_i \log (\eta_i^\triangle + 2))$. - The algorithms cannot be implemented in the "in-place manner" because they use extra binary trees.

Questions

- The height of a binary search tree can be deterministically maintained within a logarithmic order of #nodes by using the "rotation" operation. Can the presented algorithm with dirty comparisons also be derandomized by rotating the tree? - What happens if there are ties in the correct order of elements? Are your sorting algorithms work? If so, are they stable? Can you extend your result to the setting where the dirty comparison for two elements reports that "they are in the same order"? - The insertion sort runs fast if the array is almost sorted. Thus, in the positional prediction setting, the bucket sort (by the prediction) followed by the insertion sort seems to run fast. I'm interested in an experimental comparison.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Yes

Reviewer HoTq6/10 · confidence 3/52023-06-28

Summary

This paper studies learning-augmented sorting, where an algorithm obtains access to a potentially erroneous prediction on the sorting of an array. The paper considers two prediction models. In the positional predictor, every item has a prediction for its place in the sorted array, and the error of item i is the number of indices its predicted location is from its true location in the sorted array. The also consider a one sided error model for that prediction. In sorting with dirty comparisons, every pair of items has a < or > relation between them. Note that the dirty comparisons do not have to satisfy transitivity, thus potentially giving rise to cycles. A query between two elements is called a clean comparison. The authors design sorting algorithms that use these predictions, and the performance of the algorithms depends on the error in the prediction. The performance is measured by the number of comparisons. Two of the main results (given with matching lower bounds) are: - Given access to dirty comparisons, there is a randomized alg sorting in O(n log n) runtime, O(n log n) dirty comparisons, and O(sum_i log(eta_i + 2)) clean comparisons in expectation. - Given a positional predictor, there is an alg sorting in O(sum_i log(|p(i)-\hat{p}_i|+ 2)) runtime and comparisons.

Strengths

- The paper is well written, and I believe the theoretical results to be correct. - The experimental results sufficiently back the theory (for my taste). - Overall, I like the paper well enough. However, I’m concerned about how much new insight into sorting with predictions this is contributing, given that similar problems have been studied and the authors do not thoroughly compare to these prior works. Can the authors comment more on this?

Weaknesses

- While the paper gives a nice background on learning-augmented algorithms, I think several key works are missing that need to be compared against. For example, I need a thorough comparison against The Case for a Learned Sorting Algorithm by Kristo et al. Also, the papers The Case for Learned Index Structures by Kraska et al and SageDB: a Learned Database System by Kraska et al should be at least mentioned. I would also like a few more sentences explaining why the Lu et al and Kristo et al algorithms are not amenable to this setting/ what the notable differences are. - From a cursory glance, I believe the paper from Kristo et al has analysis that depends on the total error of the indices, whereas the paper here considers a more fine-grained analysis by depending on each indices error. I don’t know how/ if the algorithms are different. Can the authors please comment more on this?

Questions

(Please see questions in Weaknesses and Strengths) Many grammatical typos throughout, I highlights a few below but I recommend a full read through before submitting a CR to Neurips or submitting to another venue. Here are some things I noticed below: - (lines 15-17) The first sentence in the intro is oddly phrased, consider rewording. - (lines 24-38) Add citations for the examples of how learning-augmented sorting could be useful. - (line 41) In *the* dirty comparison setting… - (line 42) comparison *bounds* on the error… - (line 126-127) While *the* recurrent noisy… - (Lines 176-179) I think slightly more detail here could be added. The point is that with the dirty comparisons you end up without a total ordering, so the point is the go to the closest subtree that contains something breaking the total ordering and then fix that element’s position in this subtree? So this clean search is what will remove the cycles. - (Line 293) We design *the* decay setting.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

na

Reviewer S6Cj8/10 · confidence 5/52023-07-03

Summary

The paper studies the oracle-augmented sorting problem, in which, in addition to the array, an oracle is provided to return ‘reference answers’ for the sorting of the array. The oracle is presumably efficient – queries are assumed to take $O(1)$ time – but might be noisy, i.e., at times return incorrect answers. The key question is to design algorithms that 1). When the oracle is great, the number of comparisons is significantly smaller than the baseline algorithms; and 2). When the oracle is very bad, the number of comparisons is still comparable to the baseline algorithms. To this end, the paper considered oracles under two settings: a). An oracle that gives ‘dirty comparisons’ – a list of ‘all-pair’ comparisons that may include some errors and b). An oracle that gives a positional list – a list of predicted positions for each element in the array. The paper then designs algorithms under both models. In particular, for the dirty comparison model, the proposed algorithm achieves $O(\sum_{i=1}^{n} \log \eta_{i})$ number of true comparisons and $O(n \log n)$ dirty comparisons, where $\eta_i$ is the number of ‘wrong comparisons’ based on element $a_i$. This demonstrates a smooth tradeoff between the number of true comparisons needed and the prediction error. The paper also obtained similar results for positional list predictions under different notions of error. I read the dirty-clean comparison algorithm and proofs to some details, and they look correct as far as I can tell. I skimmed the claims for the positional list algorithms, but did not verify the proofs (these algorithms seem to have built on more recent ideas). For the dirty-clean comparison, I think a key observation here is that the binary comparison trees are fairly ‘robust’ against perturbations in the sense that 1). The error is consistent – if there is $R<L$ on level $t$, then the error also happens for all $t’>t$ and 2). Thanks to randomization, the probability of the mess-up of the upper and lower bounds becomes exponentially small when the level moves up. As such, the binary search tree should be able to travel sufficiently deep in the tree based only on the dirty comparisons unless a very large chunk of the comparisons is corrupted. These observations, although not technically novel, do seem to be neat and cute for this problem. Overall I think the paper makes a significant contribution to a fundamental problem, and the setting is interesting and well-motivated. The paper is also well-written and easy to follow; the experiments also demonstrate competitive performance for the new algorithms. As such, I have a positive evaluation of the paper.

Strengths

As I mentioned in the general evaluation, I think the paper is in a nice shape both in terms of the contributions and the presentations. To elaborate a bit more: - The paper follows a recent line of work on ‘learning-augmented algorithm’/’algorithm with advice’, and it tackles a very important and fundamental problem. It is surprising that previous work has not considered the problem. - Although the techniques are not complicated, the algorithms are based on several neat observations. The upper bounds are also tight by matching the lower bounds (unfortunately deferred to the appendix). - In addition to the theoretical results, the experimental performance is also quite competitive – one interesting observation is that the dirty-clean sort remains very competitive to baseline even when the corruption rate approaches 1. Furthermore, the experiments are run on large scales, which verifies the efficiency also from the practical time side (not necessarily captured by the theoretical time complexity).

Weaknesses

- One potential criticism of the work is that the techniques are not novel and perhaps easy to come up with. This is true for the dirty-clean sorting algorithm; and although I *personally* do *not* think this is an issue, this perspective is frequently flagged for theoretical work. The positional list with left-right errors seems to require more involved techniques, but I do not really like that definition of error. - It seems the two settings of the oracles in this paper are very natural. However, there could still be other settings that your model does not capture. For instance, one can define a dirty comparison oracle with amortized error only (across ${n \choose 2}$ comparisons), and what will be the optimal bound therein? That being said, I don’t think it’s very fair to ask a paper to address all possible models, so I’m not letting this question affect my evaluation. - Some (minor) presentation issues: In the binary tree search subroutines of algorithm 1, the use of tuples makes the text slightly confusing. I struggled for a minute to parse the value assignment of tuples. Maybe you can say we use tuples to represent (lower-bound, pivot, upper-bound) before pointing the readers to the algorithm. Also, what does it mean for ‘return inorder traversal of B’? (I know what it means, but is this the best way to say it?)

Questions

- In your clean-dirty comparison model, I think you have this ‘deterministic error’ that if the comparison between a and b is wrong, you cannot get the correct answer by querying it again. This crucially separates your model from the classical noisy comparison line of work (your ‘dirty perturbation’ is not random). If what I said is correct, I would suggest adding a discussion about this. - In your experiments, the dirty-clean sorting seems extremely robust even when the damage ratio approaches $1$. Is this an artifact of your experiment setting/your implementation, or is there a reason that the algorithm performs in such a nice way? (Note that your proof shows only *asymptotically* competitive, but it seems even the constant is very competitive when the damage ratio is high.) (More presentation-related) - Does the term BST on page 4 stand for ‘binary search tree’? The abbreviation was never mentioned before. - Line 185, it’s a bit weird to talk about ‘timestep’ here – I think you mean the recursion step.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Most of the limitations are discussed in the ‘weakness’ section. From a broader societal perspective, since the work is of theoretical nature, there is no immediate negative impact, and I could not foresee any (take it with a grain of salt since I do not have experience in ethical/societal review).

Reviewer JwAo4/10 · confidence 4/52023-07-08

Summary

The paper proposes algorithms for sorting a set of items when unreliable extra information is available in the form of positional predictions or comparisons. The paper proposes a randomized algorithm for the former and deterministic algorithms for the latter problem.

Strengths

The paper proposes interesting algorithms for sorting when additional unreliable information is available. The algorithms are simple and intuitive and the analysis (though several ideas are standard) is effective.

Weaknesses

- A potential weakness is the relevance to the learning (NeuRIPS) community. In my opinion, the paper is more suited to a theoretical computer science venue as opposed to an ML venue such as NeuRIPS. Would like to understand the author's view on this

Questions

- Will it help to detect at some point in the algorithm that the dirty comparisons are going to hurt more than help i.e., if the algorithm can learn/find out that it will spend more comparisons (dirty + clean) than just the O(nlogn) clean comparisons, it might abandon the dirty comparisons and thus help in reducing the total number of comparisons. Is this possible in the suggested algorithms? Is there a learning component that is natural here? - Standard metrics for comparing two permutations include the Kendall-Tau distance (between permutations), pairwise disagreement error (permutation vs tournament), etc. It might be useful to state the results in terms of these metrics. - How do these algorithms change when one has a budget of clean comparisons to work with? - An important pairwise comparison setup in the machine learning context is when the comparisons are not deterministic but probabilistic - for instance when a clean comparison is made between items i and j, the result might be based on the toss of a coin with probability P_{ij} = s_i/ (s_i + s_j) where s_k is the score of item k (The Bradley-Terry-Luce model). Here, one needs to learn the score vector to be able to rank (or sort) the items. How easy/hard is it to extend the results in this paper where one needs to be able to learn these scores from dirty or clean comparisons - if yes, this will make the work much more relevant to the ML community.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Listed above in the weakness section - the results are less suited to a ML venue in my opinion.

Reviewer S6Cj2023-08-13

Response

Thanks for providing answers to my queries. I have read both the review rebuttal and the general rebuttal. I think the authors did a fairly good job of explaining additional details and results under slightly different models. I strongly encourage the authors to include them in later versions (from my side, I think it is definitely worth highlighting that the constant factor in your dirty-clean sorting is $2$ in the worst case, and the overhead can be made $n \log\log n$). I also skimmed my colleagues' reviews and comments. I do believe work among the line of 'algorithms with prediction' is suitable for NeurIPs. I'm also fairly familiar with the literature therein, and it appears the sorting with prediction problem is not studied before. I'm not *entirely* dropping the technicality point: to be clear, I love simple and cute results, but I don't think this paper contains an 'aha' type of idea. I still think the paper is very positive despite this -- I'm only using the technicality point to distinguish award-quality work vs. others. A final question: By reading your response on ``other error measures'', it's unclear whether the bound is optimal from my end. If I understand it correctly, defining $D=\sum_{i} \eta_{i}$, your upper bound is $n \log(\frac{D}{n})$; in contrast, your lower bound is $\Omega(\sum_{i=1}^{n} \log \eta_{i})$. Log is a concave function so you have $n \log(\frac{D}{n}) \geq \sum_{i=1}^{n} \log \eta_{i}$, no? So it seems we can't simply the bound is tight, unless I missed something.

Authorsrebuttal2023-08-14

We appreciate the additional comments. To clarify why we claim tightness of the bound with respect to $D$: The idea is that our lower bound proof bounds each $\eta_i$ by the same quantity, so Jensen’s inequality is essentially tight for the constructed family of instances. More precisely, the same lower bound proof can be adapted to the alternative error definition with only the following minor modifications: In the definition of the $U$-candidate set, replace $\sum_i\log(\eta_i)$ by $n\log(D/n)$. Then replace lines 530-531 by the observation that $\eta_i\le 2^{U’}$, hence $D\le n2^{U’}$, hence $n\log(D/n)\le nU’=U$.

Reviewer S6Cj2023-08-14

Increasing socres by 1

Thanks for the clarification. I do not have any further questions. Btw, I increased both my evaluation and the confidence score by 1 to reflect a more firm opinion after the rebuttal discussions.

Authorsrebuttal2023-08-20

Thank you, we much appreciate the strong support.

Reviewer HoTq2023-08-15

Thanks to the authors for these detailed overviews of these priors works. First off, I understand the two settings of the of the positional predictors vs the dirty comparisons. I think there was just a typo on my part when I was summarizing the theorem statements (my apologies!). Secondly, I believe your discussion on prior work has assuaged my concerns on the novelty of this work. I had been aware that there was work on sorting (while not falling into the algorithm with predictions framework perfectly) that seemed similar from a high-level, and I was just confused why it wasn't discussed further. Indeed, the authors from the Kristo et al paper call their algorithm a "ML-enhanced sorting algorithm", and it's from at least some of the same authors that wrote the very influential paper in the algorithms with predictions space The Case for Learned Index Structures. However, “the problem has already been studied” was too flippant of a statement from myself and I will retract that in an edit of my review. All I was looking for was a more complete understanding of how your work compares to those works. I now understand how those works compare to yours. In particular, this part of your response on the work by Kristo et al. helped me understand why their ideas could not be used to get your results: "Indeed, this makes sense for numerical inputs drawn from a sufficiently nice distribution, since then one can extrapolate from a small part of the input to the rest. If inputs are non-numeric (and no monotonous mapping to numbers is known), then one has to rely on a comparison function, and the approach of Kristo et al. would not be well-defined. In contrast, our algorithms can sort arbitrary types of items since we only assume access to a comparison function and predictions (the latter being necessary to beat )". I additionally appreciated the discussion on the Lu et al. paper, as again I now understand why their techniques could not extend to your setting! I highly recommend adding some of this discussion as space allows. I will be updating my review to reflect a more positive view on this paper's acceptance.

Authorsrebuttal2023-08-20

Thank you, we much appreciate the response and the support for our paper.

Reviewer JwAo2023-08-20

Thanks for the response. I acknowledge that I have read the rebuttal.

Authorsrebuttal2023-08-20

We thank the reviewer for getting back to us. The response also leaves us somewhat puzzled though. We understood that the primary concern seemed to have been the relevance to the NeurIPS community, which we thought we addressed by pointing to (a small fraction of) the multitude of works of the same research area that appeared at NeurIPS in recent years. Though the reviewer's response (and maintenance of their score) makes us wonder whether they still believe that NeurIPS is an unsuitable venue, or if there is some other point that we may have missed. We note that some of the questions are answered in our general response rather than the reviewer-specific rebuttal. Granted, we of course cannot claim that our work aligns with the research interests of all parts of the community. This appears to be the case with every paper, from all areas. NeurIPS's interdisciplinary ambition led to many works on "algorithms with predictions" being accepted in recent years, including also as spotlight and oral presentations. We highlight that reviewer S6Cj's answer also addressed the raised concern, stating, “I do believe work among the line of 'algorithms with prediction' is suitable for NeurIPs.”

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC