Large language models (LLMs) exhibit in-context learning abilities which enable the same model to perform several tasks without any task-specific training. In contrast, traditional adaptation approaches, such as fine-tuning, modify the underlying models for each specific task. In-context learning, however, consistently underperforms task-specific tuning approaches even when presented with the same examples. While most existing approaches (e.g., prompt engineering) focus on the LLM's learned representations to patch this performance gap, our analysis actually reveal that LLM representations contain sufficient information to make good predictions. As such, we focus on the LLM's reasoning abilities and demonstrate that this performance gap exists due to their inability to perform simple probabilistic reasoning tasks. This raises an intriguing question: Are LLMs actually capable of learning how to reason in a task-agnostic manner? We answer this in the affirmative and propose TART which generically improves an LLM's reasoning abilities using a synthetically trained Transformer-based reasoning module. TART trains this reasoning module in a task-agnostic manner using only synthetic logistic regression tasks and composes it with an arbitrary real-world pre-trained model without any additional training. With a single inference module, TART improves performance across different model families (GPT-Neo, Pythia, BLOOM), model sizes (100M - 6B), tasks (14 NLP binary classification tasks), and even across different modalities (audio and vision). Additionally, on the RAFT Benchmark, TART improves GPT-Neo (125M)'s performance such that it outperforms BLOOM (176B), and is within 4% of GPT-3 (175B). Our code and models are available at https://github.com/HazyResearch/TART .
Paper
Similar papers
Peer review
Summary
This paper proposes and evaluates a method (TART) for improving the in-context learning of large, pretrained base models applied to downstream binary classification tasks. TART stacks a second transformer on top of the pretrained base transformer. The TART transformer is trained to rely on in-context examples to solve a large number of synthetically generated binary classification problems. TART’s training is task-agnostic in the sense that its synthetic training data is unrelated to the domain of the downstream tasks on which the combined base+TART model is evaluated. The experiments demonstrate that TART improves the combined model’s in-context learning across a range of binary classification tasks, and a diverse set of base models pretrained on language, vision, and audio data.
Strengths
Given the profound importance of large pretrained transformer models today, and their remarkable ability to learn from in-context examples, general methods for further boosting the performance of in-context learning are of great interest. The paper’s proposed method, TART, is a well-motivated and novel approach. And the paper’s experiments clearly demonstrate the degree to which TART boosts the in-context learning of large, pretrained base models applied to downstream binary classification tasks. Exciting results! Finally, as noted in section 5 of the paper, the TART approach may lead to similar methods that improve LLM performance on a wider range of tasks beyond binary classification.
Weaknesses
The paper claims to make three main contributions, which I summarize here: 1. WHAT CAUSES THE ICL GAP? Study why in-context learning does not perform as well as task-specific fine-tuning despite having access to the same information, via a representation-reasoning decomposition. 2. TART CLOSES THE ICL GAP Propose a new task-agnostic method, TART, which bridges the performance gap to task-specific methods and is trained using only synthetic data. 3. TART IS GENERAL Demonstrate that TART works across different NLP tasks for a range of model families. The same inference module generalizes to vision and speech domains as well. I follow this outline of claims in describing what I see as the paper’s weaknesses. Claim 1. WHAT CAUSES THE ICL GAP? The paper mounts an ambitious study of the gap in performance between in-context learning and task-specific fine-tuning, given a common set of examples. The experimental results are suggestive, providing ample motivation for the TART approach. However, the paper ventures far beyond the question of TART’s motivation when it attempts to draw broad conclusions about a clear representation-reasoning decomposition. The field has arrived at no settled definition of reasoning, so any distinction between representation and reasoning remains nebulous. Transformers themselves illustrate the complex, intertwined relationship between representation and reasoning. As a result the paper makes extraordinary claims (requiring extraordinary evidence) when it equates reasoning with probabilistic inference, and when it makes statements like the following: • “LLMs lack reasoning abilities” • “LLMs lack the ability to perform simple reasoning over their learned representations” • “this performance gap exists due to their inability to perform simple probabilistic reasoning tasks.” I believe that too much of the paper is devoted to pursuing such a challenging case, which would require far more evidence than the experiments provide. And I see it as unnecessary, since the experimental results are more than sufficient to provide adequate motivation for the TART approach. Furthermore, the paper’s heavy emphasis on its reasoning-centric terminology (like “reasoning module” or “probabilistic inference module”) obscures the relatively simple and limited nature of TART. To be concrete, the paper doesn’t need to wade into the philosophy of reasoning vs. representation to make the case that TART (as one might expect) boosts in-context learning by training a transformer (on task-agnostic, synthetic data!) to perform well at in-context learning. In summary, while I agree that the paper does cast some useful light on the performance gap between in-context learning and task-specific fine-tuning, much mystery remains there, so I don’t see this as a strong contribution of the paper, although it is sufficient to motivate the approach. Claim 2. TART CLOSES THE ICL GAP The experiments demonstrate to me that TART does succeed in largely closing the in-context learning gap, at least when given enough examples. But what happens when only a few examples are available at test time? The paper presents no performance comparisons against baselines tested on fewer than 20 examples, as far as I can see. Since the TART transformer is a task-agnostic module sitting on top of the base model, one would expect its performance to be no better than random in the absence of task-specific examples at test time. So how many test-time examples does it take for TART to show good results? Below 20 examples is an important range to explore, since in-context learning is known to be able to derive great benefit from as few as one or two examples. If it turns out that TART requires at least, for example, 10 examples to deliver benefit, then the paper would be over-claiming when it says that TART, “when composed with any LLM via its embeddings, generically improves upon its reasoning abilities.” The all-important technical details of TART’s training are poorly explained in the current version of the paper. Section 3.2.1 leaves too much to the reader’s imagination. Line 217 talks about updating the parameters $w$, which must therefore refer to the parameters of the TART transformer. But equation (3) also contains a symbol $w_t$ which cannot refer to the transformer’s parameters. Rather, the $w_t$ must refer to the randomly sampled weights for the linear layer used to define a binary classification problem that the transformer is trained on. At least this is my current understanding. The apparent conflict of notation needs to be sorted out, and eq. (3) should be explained much more carefully in the text. For instance, it’s worth mentioning that the angle brackets refer to a dot product, since this notation is rarely used in papers on transformers. The paper would be easier to follow if it used the term logistic regression correctly. Specifically, logistic regression is not a problem or a task. Rather, logistic regression is a method that can be used in solving classification problems. The term is used correctly on the following lines: Line 231 “For each problem, we train task-specific linear classifiers using logistic regression” Line 535 “To conduct linear probing over the embeddings, we perform logistic regression over the output embeddings of each model and the given labels in the training set using the built-in logistic regression solver from the scikit-learn python library, utilizing the lbgfs solver.” But the rest of the paper uses the term “logistic regression problem” as if logistic regression were a problem type rather than a solution method. Terminology for eq. (3) is tricky, because it involves (I believe) a linear layer of random weights which could be (but are not) obtained by the logistic regression method. And these random weights are used, along with random covariates $x$, to generate synthetic classification problems. Because the structure of this generator is related to the method of logistic regression, it is tempting to refer to the generated classification problems as logistic regression problems. But that’s conflating problems with methods. I suggest spending a few sentences to explain eq. (3) in simple terms, and replacing the phrase *logistic regression problem/task* with *binary classification problem/task* throughout the paper. It seems that positional encodings must be required in training the TART transformer, to establish the alternating positions of $x$ and $y$ vectors in the sequences. But I find no mention of positional encodings in the paper. Claim 3. TART IS GENERAL It is shown that TART works across different binary classification tasks and model families, including vision and speech domains. However, the results are all limited to binary classification, which is not a very general class. As explained in the discussion, “In future work, we seek to understand whether synthetic tasks exist for training other generic reasoning modules, capable of improving base LLM performance on tasks such as generation or summarization.” Such results would be highly significant. Until then, the current restriction to binary classification limits the significance of this contribution. MINOR SUGGESTIONS Line 130 “at par when compared with” should be changed to “on par with”. Line 191 “TART comprises of two components” should be changed to either “TART comprises two components” or “TART is comprised of two components”. Line 584 A transformer outputs a sequence of vectors, not embeddings. CONCLUSION Because of the problems noted in this section, I would vote to reject the paper in its current form. However, I have confidence that the authors can address all of these issues in the next version of the paper, so for now I’m optimistically voting for acceptance.
Questions
I assume that TART stands for Task-Agnostic Reasoning Transformer. Is this explained somewhere? Line 589 If the context length is 258, and each example is one pair, then there would be 129 in-context examples, not 256. Right? I’m not sure what’s meant by “the binary labels y are encoded as a one-hot vector to match the input dimension $d$ of the corresponding covariates $x$.” Does this mean that only 2 of the 16 dimensions are ever 1.0, while the other 14 dimensions are always 0.0?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
4 excellent
Contribution
3 good
Limitations
No problems noted.
Summary
The authors study why in context learning doesn't perform as well as task specific fine tuning by decomposing the process of in context learning into representation and reasoning. They discover that the gap is due mostly to deficiencies in reasoning and propose a new method called TART to bridge the gap. They demonstrate that TART is task and model agnostic.
Strengths
Originality: The authors note and address the problem that in-context learning underperforms model fine tuning. The paper's primary contributions are disentangling representation strength and reasoning ability and finding a way to measure them, then using the insight from their measurements to improve the model's reasoning ability and thus close the gap between in context learning and fine tuning. Both are very original. I thought the concept of training a reasoning model to do probabilistic inference using synthetic data sets was clever. The same for averaging embeddings of multi-token data points in order to piece together the base model and the reasoning module. I also like thinking of LLM reasoning as probablistic inference in the first place. That seems insightful. Quality and Clarity: The quality here is high. They provide extensive experimental results to validate their idea, comparing TART with baselines across modalities and models. Every concept is backed up by a full sweep of experiments, they answer questions before I have a chance to ask them. For example extending this to vision and speech went above and beyond. The writing is clear well structured and the methodology is clearly explained. Their figures and tables are clear and accessible. Significance: I believe the significance is high. The authors address a limitation of LLMs with a novel solution that is broadly useful. It has potential to significantly improve transformer model's in context learning performance across the range of tasks. Overall, great work.
Weaknesses
There's a typo on line 339, it should be CIFAR-10 (classes plane and bird) and MNIST (classes 0 and 8)
Questions
In a real world application of this technique it would be inconvenient for the user to have to indicate which tokens belong to which data points/labels, which I think the embedding averaging method requires. For example if I understand right Sentence: The movie is good Label: Positive The user would have to manually indicate which tokens belong to the data (sentence) and label so it can be averaged. I can imagine using the base model to separate them out for you. Is that how this would work?
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
Yes
Summary
The paper studies why in-content learning achieves inferior performance compared with finetuning and adaptor, then proposes an LM-based inference module that learns to perform logistic regression based on the sample and previous (sample, label) sequence, where the sample and linear cutting-plane are sampled vectors. When testing, the base LLM encodes the in-content examples into single vectors, concatenates them with their corresponding labels to form the (sample, label) sequence and combines with the test sample to predict the test label. Experimental results show the proposed inference module can achieve comparable results without task-specific parameters.
Strengths
The paper proposes a new way to classify a test sample based on training examples. The proposed inference module learns to predict the sample label given (x,y) examples, which share the same separation plane. By sampling enough various planes and input x, an LLM is possible to learn how to find the linear separation plane given (x,y) examples, thus correctly predicting the test label without further training. The proposed method may apply to meta-learning tasks as both settings are similar.
Weaknesses
Although the proposed inference module is interesting, I can not see it has a significant advantage over linear probing. Firstly, TART requires base LLM to encode examples and then fed the average pooled vector into the proposed inference module. This process requires a similar amount of computation and memory as in-context learning, and this process must repeat k times in the proposed LOO embedding if given k examples. On the contrary, linear probing can encode each sample independently and then train one linear layer for prediction. Secondly, since the inference module is trained with linear logistic regression, the LLM representation must be linear separable for task-specific labels. Thus, it has the same upper bound as linear probing, while one linear layer should be much easier to train than a GPT-2 model. Line 10 claims "performance gap exists due to their inability to perform simple probabilistic reasoning tasks" and Line 51, "...prompt engineering or active example selection, focus entirely on the LLM’s learned representations." The authors do not clearly explain what the reasoning ability is and why prompts improve learned representations. I assume prompt engineering is proposed due to the unavailability of hidden representation inside GPT-3 or GPT-4.
Questions
What do the "parameters w" in Line 217 and "w_t" in Equation 3 means? Are these two refer to different things? How does the TART obtain multi-way classification labels as it is trained with two-way classification? Can this method generalize to text generation with a much larger vocabulary? Authors may compare TART with linear probing besides FT-layer given the same amount of examples.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
The authors addressed the limitations and broader impact.
Summary
The paper presents an recipe for adapting an LLM to perform classification tasks in a task agnostic manner. They first try to tease apart if existing "in-context" methods which construct prompts to describe the task and then ask for the inference result are not achieving great performance because of information extraction or reasoning ability. By constructing a linear probe experiment they get indications that it may be the reasoning ability which hampers performance. The authors then propose to improve the task performance by training an inference module to solve a generic task, here few shot logistic regression, and then show that they can apply this to the outputs of the LLM to obtain good performance on desired tasks. The down stream tasks considered range from sentiment classification, news article categorization to spam detection. They also verify that the performance gains on text tasks carries over to image tasks and speech.
Strengths
* There is still a lot of scope and practical use in extracting information from a text model in a reliable way. * The method is novel in so far I can tell. * Contains reasonable baselines.
Weaknesses
* Although I find it intriguing the paper leaves me more questions than it answers. I think it would be very beneficial to make the presentation much more concrete and map some of the down stream tasks clear and exemplified in the main text. They remain very abstract and lots of intuition that the readers should build up is lost that way. * Although I think the line of scientific inquiry is quite intriguing I don't find the paper mature enough to be published in the current form. I find the writing can be improved a lot and maybe the context can be expanded a bit.
Questions
* Th1 seems to say that the error is controlled by how similar P_NL and P_syn are but they are really different. In some sense they should always be different because it's a different language with a very different structure. In NL we have a lot more expressivity and less regularity ? Did you do any experiment that broadens the task space to see if this improves ? * I think 3.3 should be more appropriately named how to leverage LLM embeddings. The embeddings are always the last layer it's more how they are used that changes ? * Are all the tasks considered binary classifications ? I guess line 339 should be corrected MNIST and CIFAR are switched. We can't easily evaluate this work when this is happening.
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
2 fair
Presentation
2 fair
Contribution
2 fair
Limitations
None.
Response to rebuttal
I applaud the authors for their detailed rebuttals, and especially for the additional experiments which addressed a couple of my concerns (TART when given few examples, and multiclass tasks). Unfortunately, the authors did not even mention my major points regarding the lack of clarity in describing the central method: The terse coverage of section 3.2.1 and especially Eq. (3), the apparent dual usage of the symbol $w$, and the key question of positional encodings. My initial rating was based on an optimistic assumption that a revised version of the writeup would be attached. Without reading it, I have little confidence that the published paper would explain the method well enough to benefit the community. For this reason, I am lowering my score from 5 to 4.
Thanks a lot for taking out time to read our rebuttal. Unfortunately, the conference does not allow us to submit a revised version of the draft and we are not allowed to put in any text in the attached pdf. However, we do provide a concrete retelling of the story in the global response which will allow us to restructure and add more content to our revised draft. - We added the clarification to the dual usage of the symbol $w$ in response to Reviewer hArx (https://openreview.net/forum?id=ZXbgVm3PSt¬eId=IdKRUWCfRs). It is a minor typo and should instead be $\theta$. - We will pull up the details of the algorithm from Appendix C.1 to section 3 -- it is already there and simply needs to be accommodated with the existing content. - Yes, the default GPT-2 architecture that we use (see Figure 15 in the draft) has absolute positional encoding for each position. We will clarify this in more detail in the paper. We hope this addresses some of the concerns you mentioned above.
Response
I appreciate the quick clarification, which helps. But Appendix C.1 says nothing that would help readers digest the austere equation (3). There needs to be a clear and detailed explanation of how the task-agnostic training dataset (used to train the TART module) is constructed. These details might even belong in the appendix, hopefully including a few sample examples for illustration. These missing details are at the core of the technique and arguably the most important part of the paper. Did I somehow miss them?
clarifying eq (3)
We really appreciate you taking the time for this discussion and your quick response. I think we misinterpreted your point in the review; please allow us to clarify your doubt. Equation (3) is the fundamental logit model (linear logistic model) which describes how we sample the synthetic data which lies in a d-dimensional space. This statistical model has been the basis for modeling uncertainty with discrete-valued output variables, since as early as 1940s [1]. In more detail, the sampling process goes as follows: - the regressor $w_t$ is sample from a standard Normal distribution - the features $x_{i,t}$ are sampled from a standard Normal distribution - the corresponding $y_{i,t}$ are sampled based on the sigmoid of the inner product of $x_t$ with $w_t$ Figure 2 (left) gives a demonstration of a 2d dataset sampled from the process -- it is simply drawing noisily linearly separable data in d-dimensions. The important point to note here is that it has nothing to do with downstream natural language tasks, and hence agnostic to them. High-dimensional variants of these correspond to datapoints on a d-dimensional sphere with noisy labels which is almost-linearly separable. Given this generation process, Figure 15 describes how we place these sampled real-valued data points in the TART architecture. Additionally, if you see Appendix A.1, Equation (5): this is equivalent to Equation (3) in the natural language setup. Lines 508-511 actually provide concrete examples of how such sequences look like when projected onto the language domain. This subsection contains more intuition and details into the sampling procedure. We understand your concern that readers might not be familiar with this logit models, and we will add in additional discussion describing the process above as background. [1] Cramer, J. S. (2004). "The early origins of the logit model". Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences.
Response
It's not a question of whether readers are familiar with logit models, especially when the paper doesn't explicitly mention the *logit model* at all. (And as we have discussed, the paper's usage of the term *logistic regression* muddies the waters, raising suspicion regarding the paper's usage of technical terms in general.) It's a question of whether the paper explains how the task-agnostic data is generated with sufficient clarity for readers to feel confident that they understand and could apply the technique themselves. I believe the bullet points in your last comment provide the right level of detail for walking readers through eq. (3). The reference to Fig. 2 is also helpful as an illustration. No need to get pedantic though, as when citing the early history of the logit model.
Thank you for your suggestion and time. We will add the bullets above as an explanation right after Eq. (3) and highlight the reference to Figure 2 there. The logit model is simply another name for the logistic model which is often used interchangeably -- we will be sure to be consistent in our usage in the updated manuscript. We hope that this addresses all your concerns. We will make sure to make these important modifications to our draft; these suggestions will help improve the presentation of our work. If you have any other questions, please do let us know and we will try our best to address it. We really appreciate you taking the time to discuss.
Response
Of course you're right that *logit model* and *logistic model* are the same thing, but the paper uses neither term. The paper uses the term *logistic regression*, which is not a model at all, but a method for estimating the parameters of a logistic model. If the paper confuses *logistic regression* with *logistic model*, many readers will be confused as well. Despite my lingering concern that the paper will not adequately handle these technical explanations, I'm impressed by the extra experiments provided in rebuttal, so I'll raise my score back to its original value.
Changes to draft
Thank you so much for engaging in this constructive discussion. We agree that the paper draft, in its current form, has some issues with clarity which you have raised through your review and discussion. While we are constrained by the conference rules to not upload an updated pdf, we would like to highlight the changes we have made to our revised draft, to make sure we adequately address your concerns. > Clarifying logistic model vs logistic regression Thanks for bringing up this distinction. We completely agree that this overuse of the term logistic regression problem can lead to confusion for the reader. Based on your feedback, we are changing the following lines. - Line 61: synthetically generated tasks from the logistic (or logit) model - Figure 2: synthetically generated tasks from the logistic model - Line 215: input sequence corresponding to a different problem sampled from the logistic model - Line 218: parameters $\theta$ comprises a different d-dimensional problem sampled from the logistic model given by: - Line 230: 64 different problems sampled from the logistic model - Line 238: noise level, $\alpha$, of the logistic model in Eq. (3) - Line 273: generalization error on the tasks sampled from the logistic model > Understanding Eq. (3) Lines 219-220 have been updated to: where $\sigma$ represents the sigmoid function and the multiplier $\alpha$ determines the noise level of the problem. The above sampling model comprises three components, the regressor $w_t$, features $x_{i,t}$ and labels $y_{i,t}$ sampled as - the regressor $w_t$ is sample from a standard Normal distribution - the features $x_{i,t}$ are sampled from a standard Normal distribution - the corresponding $y_{i,t}$ are sampled based on the sigmoid of the inner product of $x_t$ with $w_t$. See Figure 2 (left) for a demonstration of a 2d dataset sampled from this process -- it is simply drawing noisy linearly separable data in d-dimensions. > Confusion regarding symbol $w$ Line 217 is updated to "Each training sequence $s_t$ used to update the parameters $\theta$" > Architecture details We have updated line 207 to: The GPT-2 architecture comprises a causal (unidirectional) transformer with 12 decoder layers and 8 attention heads per layer for a total of 22 M parameters. The model has absolute positional encodings for each position. Please see Appendix C.1 for more details. Additionally we will also add the model card describing the type of attention (local vs global) to our appendix. > Additional experiments on low-data and multi-class settings As shown in the attached pdf, we have added two additional sub-sections in the evaluation section (Sec. 4). Section 4.2 has been updated with an additional paragraph on performance in low-data regime. An additional section has been added with extension to Multi-class tasks (Sec 4.4). > Reasoning vs. representation To make sure our claims are within the scope, we have updated: - Line 108: understand their relative performance for binary classification tasks. - Line 111: downstream binary classification tasks. - Line 144: This section investigates why this performance gap exists for binary classification tasks. - Line 151: insufficient reasoning abilities, i.e., the ability to learn linear decision boundary for a binary classification task - The hypothesis H1-H3 have been updated to focus on binary classification tasks. - Additionally, we will add an extra section in the appendix focussing on the same analysis with multi-class tasks. > Minor comments - The acronym for TART has been added to Line 59: We propose TART (Task Agnostic Reasoning Transformer) - Line 191 has been corrected to "TART comprises two components" - Line 589 has been updated to 129 in-context examples --- Thank you so much for helping us improve the readability and clarity of our manuscript. We sincerely hope these changes and the retelling of our story from the global response address your concerns and that our revised manuscript is up to the high standards you expect.
Response
Yes, these are great improvements to the presentation! I've raised my score one notch.
Thanks
Thank you for your thoughtful comments and feedback. The review and discussion has gone a long way in improving our paper's presentation and clarity. We really appreciate your time and effort!
I thank the authors for the response. My concerns about generalizing TART and Equation (3) have been addressed. But I agree with reviewer CKNq that the paper makes extraordinary claims about representation and reasoning. It requires much more evidence to support that the LLM can not perform simple reasoning. Without the claim, the proposed TART has its merits in improving ICL. It uses a GPT-2 to learn to approximate any linear transformation given examples (x_i,t, y_i,t). It is interesting and has similar concepts to meta-learning. However, it has several drawbacks: 1) You must sample enough w_t to span all possible distributions of the LLM output feature. 2) Why would one approximate the linear weights through extensive meta-learning instead of learning the linear weights, that is, linear probing straightly? With enough sampling and training costs, I think TART can outperform linear probing. But since Figure 5 and 6 shows the adapter surpass TART most of the time, I still have concerns about whether the merits are significant in practical use. Due to the above two points, I like to keep my rating unchanged but OK with both decisions.
Thank you for your response and detailed feedback on our work. We are glad we were able to address your concerns on multi-class evaluations and Eq. (3). Please allow us to address the additional questions you asked. We agree with both you as well as Reviewer CKNq that our claims in Section 2.3 are restricted in scope to binary classification tasks and that it is not reflected in our writing. As we highlighted in our response to Reviewer CKNq that we will restrict the scope of those statements and make sure they are not taken out of context. In addition to this, we will make the analysis more general by extending it for multi-class classification problems as well. (From our response https://openreview.net/forum?id=ZXbgVm3PSt¬eId=Pc5b4H5FKn) > Reasoning vs. representation To make sure our claims are within the scope, we have updated: - Line 108: understand their relative performance for binary classification tasks. - Line 111: downstream binary classification tasks. - Line 144: This section investigates why this performance gap exists for binary classification tasks. - Line 151: insufficient reasoning abilities, i.e., the ability to learn linear decision boundary for a binary classification task - The hypothesis H1-H3 have been updated to focus on binary classification tasks. - Additionally, we will add an extra section in the appendix focussing on the same analysis with multi-class tasks. > On linear probing vs in-context learning Thank you for pointing out this question and we believe that the paper and our initial response failed to sufficiently outline the precise regime where TART is actually preferable to linear probing. We believe there are two regimes: - Single-task regime: In this regime, practitioners would like to improve the performance of a base model on a single task such as sentiment analysis. As your review highlights, linear probing is preferable to TART in this setting since it is computationally cheaper to optimize a linear model for a given task. - Multi-task regime: In this regime, practitioners would like to improve the performance of a base model over multiple tasks (e.g., all tasks on the RAFT benchmark) where in-context prompt is used to adapt the model for each task. This paradigm is reflected in numerous LLM benchmarks like OpenLLM leaderboard [1], HELM [2], and RAFT [3]. For these benchmarks, researchers have focussed on task-agnostic techniques like modifying pre-training data and fine-tuning the model on instruction/chat data. These are computationally expensive as they both require training the whole model. Our work argues that TART is intended for this regime. For instance, we show on the RAFT benchmark, TART improves GPT-NEO (125M)’s performance such that it outperforms BLOOM (176B), and is within 4% of GPT-3 (175B). The comparison to linear probing is to demonstrate that TART can match task-specific interventions while being task-agnostic. TARTs advantage over linear probing scales with the number of tasks that practitioners care about -- TARTs advantage over linear probing for one task is minimal, but the advantage for 100s of tasks (as captured by benchmarks and real-world user interactions with chat-models such as chatGPT) is substantial. This is because for each new task, linear probing requires practitioners perform task specific optimization (around 1 million parameters for sequence length 1024 and embedding dimension 1024). We are glad that you pointed out this question on "Why one cant one use linear probing" and we will update the manuscript with the above discussion distinguishing the two setups. We hope that this addresses your concerns and are happy to answer any more questions that you might have. We are really grateful for your response and insights. *References* [1] https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard [2] Liang, Percy, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang et al. "Holistic evaluation of language models." arXiv preprint arXiv:2211.09110 (2022). [3] Alex, Neel, Eli Lifland, Lewis Tunstall, Abhishek Thakur, Pegah Maham, C. Jess Riedel, Emmie Hine et al. "RAFT: A real-world few-shot text classification benchmark." arXiv preprint arXiv:2109.14076 (2021).
Thanks for the detailed response on comparing with linear probing. It has addressed my concerns. I will raise my rating to WA.
Decision
Accept (poster)