Fairness-guided Few-shot Prompting for Large Language Models

Summary

The paper addresses the problem of constructing an appropriate prompt for improving the performance of in-context learning in large language models (LLMs). It introduces a metric based on predictive bias to evaluate prompt quality and empirically demonstrates that prompts with higher bias lead to unsatisfactory predictive quality. Leveraging this observation, the paper proposes a novel search strategy, combining T-fair-Prompting and G-fair-Prompting, to identify near-optimal prompts for enhancing in-context learning performance. The proposed method is extensively evaluated on various downstream tasks using state-of-the-art LLMs, such as GPT-3 and LLaMA. The results show that the method effectively improves in-context learning performance in an interpretable manner.

Strengths

The paper highlights the importance of prompt quality and introduces a novel metric based on predictive bias to evaluate prompt performance. The proposed method effectively searches for near-optimal prompts by addressing bias issues, leading to improved in-context learning performance. The paper conducts comprehensive experiments on mainstream LLMs and various downstream tasks to validate the effectiveness of the proposed method. The results consistently demonstrate performance enhancements, surpassing state-of-the-art methods in most cases. The paper presents two search strategies, T-fair-Prompting and G-fair-Prompting, which efficiently optimize prompts by considering bias and fairness. The strategies offer a balance between search quality and computational complexity, addressing the challenges of prompt construction. The paper is well-written and effectively conveys the motivation, contributions, methodology, and experimental setup. The algorithms and search strategies are clearly described, allowing for easy understanding and reproducibility.

Weaknesses

Scalability: The paper discusses the computational complexity of the search strategies and their limitations. While the proposed strategies show promising results, the complexity may become an issue when dealing with larger datasets or models. Further exploration of scalability and efficiency would be beneficial. Generalizability: The paper mainly focuses on LLMs and downstream text classification tasks. Extending the evaluation to other types of tasks and models would enhance the generalizability of the proposed method.

Questions

How generalizable is the proposed method beyond text classification tasks? Have you explored its applicability in other types of downstream tasks, such as language generation or sequence labeling? The search strategies, T-fair-Prompting and G-fair-Prompting, effectively optimize prompts by considering bias and fairness. However, how sensitive are these strategies to the choice of hyperparameters, such as the number of demonstrations (k) or the threshold for fairness improvement? Have you conducted sensitivity analyses to investigate their impact on performance?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

None.

Summary

The paper examines the sensitivity of large language models to prompts and analyzes the issue from a predictive bias standpoint. They utilize a fairness metric called a content-free strategy to evaluate the predictive bias of a fixed prompt. Methodologically, the paper proposes two strategies, T-fair-Prompting and G-fair-Prompting, to search for the fairest prompt and validates their effectiveness through extensive experiments on large-language models.

Strengths

1. The paper exploits the correlation between predictive bias and overall performance and proposes two strategies based on the finding to approximately estimate the fairest demonstration combination. 2. It is intriguing to observe that fairness-guided selection, rather than diversity or similarity-guided approach, often exhibits better performance.

Weaknesses

1. The predictive bias-based fairness metric, which is the most key and primary idea, is adopted directly from [18]. The way of using the fairness metric in each T-fair-Prompting and G-fair-Prompting is also straight-forward. I think that the methodological novelty of this paper is limited. 2. I can't find an ablation study regarding the proposed fairness metric. I am curious about the results and analysis when using other semantic-free prompts, apart from N/A. 3. When looking at Table 2, there are quite a few results where diversity-guided selection demonstrates higher performance. The overall performance is not particularly impressive.

Questions

Please provide an in-depth response regarding the weaknesses mentioned above.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The role of few-shot prompting in in-context learning is significant. Therefore, I believe that providing a new insight is beneficial for the in-context learning community. However, I would like to request a sufficient response that adequately addresses the novelty of the proposed work.

Summary

The paper proposes two new procedures to select examples to be used for few-shot learning (in-context learning) for LLMs. The key idea for example selection is to select those examples that lead to a distribution over classes that is as close to uniform as possible . The two proposed variants (a) select the top k examples which each individually have the least ‘bias’ and (b) greedily add examples that reduce ‘bias’ the most until any additional example would increase it. The authors evaluate the proposed methods on a variety of open source LLMs on multiple tasks, where the proposed methods perform well.

Strengths

In-context learning, and in particular the selection of effective demonstrations is an important area of study and thus of interest to the research community. The paper is well written and the proposed methods are clearly presented. The evaluation is well chosen to demonstrate the effectiveness of the proposed methods. The proposed methods are a clever idea that is simple yet effective.

Weaknesses

The proposed method is limited to multi-class classification tasks, which places a significant limit on the practical use-cases of the proposed method, i.e., it does not apply to summarization, translation, open ended generation etc. Some of the claims in the paper are too strong and not supported. For example in the abstract in lines 9-11 the authors claim that “Then we empirically show that prompts with higher bias always lead to unsatisfactory predictive quality.” Which is clearly not true and not supported by their findings in Figure 2, which shows a non-monotonic trend. While the writing is generally good, there are a few mistakes across the paper, e.g., line 34 to construct I think the use of the term fairness is slightly misleading in this paper. This the term ‘fairness’ is associated with many different meanings, but mostly with social aspects, the authors could directly use entropy as that would be a more technical description of their method.

Questions

I would love to see the authors address the usage of their method to other tasks that are not easily cast into a multi-class classification task. I would also like the authors to address the use of the term ‘fairness’ and why that is necessary for the paper at hand.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors did not address limitations and other impact. Especially, given the term fairness in the title, I would expect the authors to at least discuss the use of the proposed method to address fairness issues and also the potential misuse to introduce unwanted biases.

Summary

The paper shows that the accuracy of large language models on various text classification tasks for a given prompt tends to be correlated with the entropy of the output prediction when using that prompt for a context free input: in general, the more uniform the output probability distribution for a given prompt (made up of demonstrations from the dataset), the higher the accuracy of that prompting strategy for the dataset. The authors use this insight to select the demonstrations and show that greedily selecting demonstrations one-by-one can yield competitive performance to other strategies.

Strengths

1. The authors point out that empirical observation that higher entropy in the context-free case is correlated with higher accuracy was first introduced in Zhao et al. 2021. However, showing this correlation holds in the context of prompting is, to my knowledge, novel. 2. The proposed prompting search strategies are simple to implement and competitive. 3. The experiments seem pretty extensive, both in datasets and models tested.

Weaknesses

1. Given the time complexity of the G-fair-prompting algorithm (i.e. O(N^2) where N is the size of the training dataset), the proposed approach does not seem very practical. 2. The discussion of experimental results is vague at times. For instance, the explanation provided for why the similarity-guided method is best for AGNews (lines 293-296) seems as if it would apply to other tasks as well; a meaningful explanation would be able to differentiate between these settings. Also, it is not clear what "the model's sensitivity to the task is not immediately evident" means (line 300). In addition, practical discussion of the proposed approach relative to baselines would be helpful (e.g. when one is preferred over the other).

Questions

1. What does “Total Cost” in Table 1 refer to? What are the time complexities of the G-fair-prompting approach on the models and datasets tested? 2. The diversity baseline seems similarly competitive to the proposed approach. When would the authors suggest using one over the other? 3. Could the authors compare using a strategy such as the proposed one with a large language model vs. fine-tuning with a smaller model? Especially since the former requires iterating over the entire dataset anyway, albeit differently than in fine-tuning, when is the former advantageous over the latter? 4. Could the authors explain the text in lines 293-296 and 300 (as per comment #2 in weaknesses)?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors do not directly address limitations, which would be worth mentioning briefly. I did not factor this absence into my review.

Summary

Finding an (nearly) optimal prompt is crucial for successful in-context learning (ICL) of large language models (LLMs). It is known that performance of downstream tasks is sensitive to prompts themselves, few-shot examples, and their orders, often requiring large degree of non-trivial tuning. This paper presents a content-agnostic fairness metric that allows users to assess the quality of a given prompt purely in the perspective of predictive bias. Under this metric, the authors propose two methods for optimizing the prompts. Given the few-shot examples for ICL, T-fair Prompting computes the predictive bias of individual demonstrations by feeding each single shot. Then it chooses the user-specified number of examples with the lowest predictive biases. In contrast, G-fair Prompting promotes the greedy local search where it appends one additional shot that shows the best overall fairness. Users can choose one of the two proposed methods considering the tradeoff between algorithmic complexity and performance gain.

Strengths

(1) The paper provides simple yet effective approaches for prompt tuning. (2) Both T-fair and G-fair Prompting algorithms are natural and practical in the spirit of local individual search and greedy search. (3) The proposed methods achieve highly competitive accuracies comparing to the current state-of-the-art methods.

Weaknesses

(1) Computational cost of using G-fair Prompting becomes non-trivial as N goes larger even if it uses greedy approach for expanding local bias to global bias. (2) One can think of even better algorithm that can show benchmark performance that provides upper bound against G-Prompting. (3) Neither guidance for how to choose semantic-free tokens nor sensitivity analysis for various semantic-free candidates are explored.

Questions

(1) Can you provide computational cost for running T-fair and G-fair prompting over increasing N? (2) Under the proposed problem formulation, there will be a natural extension to test both different choices of few-shot examples and all possible ordering of them. Though such experiments would be expansive to navigate combinatorially many configurations, measuring the performance on a limited setting helps understanding the relative performance of T-fair and G-fair Prompting. (3) How to choose the semantic-free token without any guidance? It is less clear that such token would exist universally across different LLMs across various downstream tasks. In any case, using just one manually-reserved tokens for fairness evaluation is not easily justifiable without sensitivity analysis. (4) Predictive bias would be more useful for solving classification tests by generation (as if the paper tested the sentiment analysis). Would be the proposed method helpful for long-form QA?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

No specific points are described or probed.

Thanks for the response

I appreciate the response. I am happy to maintain my score. A few quick clarifying question: what does this point refer to? ** Note that although the former strategy requires iterating over the entire dataset, the latter strategy only requires performing inference. It does not incur any space cost for storing gradients nor time cost for updating gradients, making this process highly efficient.

Thanks for the response

I would like to thank the authors for the extensive and detailed rebuttal - it has been very helpful. I continue to think that this work is of interest.