Meta-in-context learning in large language models

Summary

The paper studies an ability of LLMs, called meta-in-context learning, which showcases that LLMs can recursively improve their in-context learning with demonstrations. The authors illustrate this capability with a regression task and a two-armed bandit task. The analysis demonstrates that LLMs are not only able to learn the task (underlying function) from the examples in this task but also can leverage the examples from other tasks (underlying functions).

Strengths

1. The paper highlights the meta-in-context learning capability of LLMs. The capability allows LLMs to be recursively improved via in-context learning with more tasks in context. 2, The paper is clearly written and easy to follow.

Weaknesses

1. The analysis in the paper demonstrates that LLMs can leverage the examples from other tasks, however, the definition of a task appears to be rather limited in this paper. In the linear regression experiment, for example, a task represents the parameters within a linear function. 2. The authors introduce the concept of meta-in-context learning as a novel ability of LLMs, but it remains unclear how this differs from the traditional in-context learning ability. In this paper, each regression function is regarded as a task with a few examples following the function, and LLMs are found to be able to learn from other tasks. However, this ability seems to have been demonstrated in various other applications. For instance, LLMs can learn to generate dialog responses by observing example responses from other dialogs. If we apply the terminology used in this paper, each dialog can also be considered a task, with each individual utterance serving as an example. While I think the analysis in the regression and two-armed bandit tasks is commendable, it would be valuable to see such analysis on more realistic tasks, such as dialog generation (a dialog is a task), and passage/image question answering (a passage or image is a task).

Questions

1. On line 118, it is mentioned that the GPT-3 is not told that underlying functions are linear, distinct from BLR. However, this raises the question of whether GPT-3 possesses the capability to handle non-linear functions. Additionally, if GPT-3 sees a few tasks with linear functions, is GPT-3 able to generalize to a non-linear function in a new task? The meta-in-context learning ability is more valuable if the tasks can be more different.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

n/a

Summary

The authors demonstrate that the in-context learning abilities of large language models can be recursively improved via in-context learning itself. The paper misses the method section. I don't know the details and cannot tell the difference from previous work. I didn't get the novel part of the method. It seems an empirical study. The authors had better do some experiments on benchmark datasets, such as the datasets from GPT-3 paper.

Strengths

1. The problem explored is critical. 2. The experimental analyses are interesing.

Weaknesses

1. Missing model section. 2. The method is more of an empirical study. 3. The experiments are not solid. The authors focus on the GPT-3 model exploration but miss comparison to any benchmark dataset from GPT-3 paper. The shot number, context length, and language understanding in benchmark datasets are all critical issues to study. And GPT-3 is released quite a long time ago. Exploring recent LLMs, such as ChatGPT or GPT-4, would be better. 4. The finding is similar to GPT-3 on domain adaptation, such as the work from "Prompting GPT-3 To Be Reliable".

Questions

Can you formulate meta-in-context learning?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

I didn't get the novel contribution of the work.

Summary

The authors undertake a study of "meta in-context learning" as a capability of Large Language Models, specifically focused on GPT-3 (with some initial experiments on GPT-4). The authors define meta in-context learning following a task-trial structure, in which the agent observes multiple tasks each consisting of multiple trials. In each trial, a given input-output pair is observed; on the final trial, the model needs to predict the output for an unpaired input. The (unobserved) function generating outputs from inputs differs between tasks. The authors show that GPT-3 is able to learn not only at the trial level (traditional in-context learning) but also across tasks; that is, it can get better both at modelling a specific function and at the general task of modelling functions (in a particular context). The authors further show that this holds across artificial simple regression, reinforcement learning and multiple regression.on real-world data.

Strengths

The paper is well-written, and the results are thoroughly investigated and clearly conveyed. The authors seek to identify the specific causes of the behaviours they document and the extent to which they can be attributed to simpler processes, e.g. learning the output distribution for predicting on the first task of a trial. The paper includes a discussion of context window size-related limitations to the applicability of the results. On originality, I don't know of any other works that address the question of meta-learning without weight updates. As the authors point out, the results could inspire other researchers and applied LLM users to persue multi-task setups rather than finetuning.

Weaknesses

The paper is well-reasoned within the specific niche of tasks demonstrated, though the overall implications of the work are arguable. Because the examples are constrained to simple numerical problems, it's difficult to tell whether the capability can be usefully leveraged on other types of language tasks. In the discussion, where it's argued that meta in-context could replace finetuning, it would help to give a few examples of real-world tasks that could be reframed as meta in-context. Within the artificial tasks, it would also be nice to see non-numerical demonstrations of meta in-context learning, both to show that it is possible on other types of tasks and because LLMs tend to struggle with numerical transformations. For instance, within-task could involve translating sentences in a simple made-up language into English (with words that are repeated across trials), while each task involves a different made-up language. This could provide a more convincing argument that meta in-context learning is a general phenomenon beyond math tasks. More detail on the exploration probit regression model and investigated regression coefficients in the Supplementary would be useful.

Questions

Is it possible to create a clear definition of what constitutes an in-context trial and what constitututes an across-context trial, i.e. where does in-context learning become meta in-context learning? Does an intial task trial always start with some degree of irreduceable uncertainty? See the above questions about real-world examples or non-numerical examples of meta in-context learning.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

2 fair

Limitations

No major negative societal implications are expected. The authors address a number of limitations, including low sample size and simplicity of the tasks, in the Discussion.

Summary

The paper explores a phenomenon referred to as “meta in-context learning” in large language models, where the in-context learning abilities of these language models can be recursively enhanced through in-context learning itself. To demonstrate this, the researchers examine two idealized domains: a one-dimensional regression task and a two-armed bandit task. They show that meta-in-context learning (1) improves in-context learning performance (2) dynamically shapes the large language model's priors over expected tasks and (3) also modifies its in-context learning strategies.

Strengths

* The high level question raised in this paper is interesting – can in-context learning be improved via the recursive operation of concatenating several different tasks in-context? * The experimental setup is clean and the obtained experimental results convey a clear message. They clearly show that for the examined problems meta in-context learning helps, and moreover they reveal that the improvement (also) comes from previous tasks in context shaping the LLM prior for the examined task.

Weaknesses

* I found the setup and particularly the examined tasks to be too limited. Although the high level question is interesting I feel like much more could be done in order to shed light on it and connect it to actual scenarios in which in-context learning is used in practice. The authors chose to focus on linear regression (synthetic and 60 real world tasks filtered down to 42) / synthetic bandit problems. The abstract and intro describe that in-context learning is a very important feature of LLMs which facilitated their success; while true it is not related to these types of tasks but rather to natural language ones. I think that the interesting high level question raised in this paper could easily have been studied for natural language tasks, and in particular relating it to papers showing multitask benefits which study which tasks are related and reinforce each other such as Sanh et al ICLR 2022, Aribandi et al, ICLR 2022 and others. I acknowledge that there is a growing literature of works theoretically analyzing and experimenting with in-context learning on synthetic / mathematical problems, the authors mention some of it. However, in this case I think that the merits of the synthetic / simple / mathematic setup were not fully utilized – for example, the authors could have studied in this simple setting how the relatedness of different tasks in the same context affects their observed signal (eg by defining a formal metric of distance / similarity between linear regression tasks). A small subset of the results in section 3.3 is dedicated to a statement about task similarity and I think it should be expanded for deeper insight. In the lack of such deeper investigations in the synthetic realm, and on the other hand no experimentation on language tasks, this paper misses out on real insight relevant for language related scenarios. For the scenarios studied in the paper, LLMs are not the go-to tool. I will note as an outlier to the above that the study of in-context learning strategies in the bandit case is deep, novel, and leverages the clean mathematical structure of the synthetic experiment. However, even in this case, I see no conclusions applicable to real world scenarios in which LLM in-context learning is the go-to tool. * The paper is not segmented into “method”, “experimental setup”, “results” sections, but rather includes one section describing all of the above linearly according to the different tasks that were ran. This is not a problem a-priori, but I found the presentation hard to follow at times, and the authors may want to consider structuring the paper a bit more. * Minor remarks: ** There are some some typos (eg, end of parentheses in line 77, named citations without a year) ** The name “meta in-context learning” is used for a different method presented by Min et al. 2021 in this influential paper: “MetaICL: Learning to Learn In Context”. You may want to rename this paper. ** I think that the authors overstate when they claim meta in-context learning to be related to psychological experiments.

Questions

What are the main practical consequences or implications of your findings for specific real world scenarios in which in-context learning in LLMs is currently used?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Yes

Summary

This paper demonstrates that large language models (LLMs) are capable of meta-in-context learning: updating their in-context-learning abilities when prompted with examples of several tasks The authors empirically show this capability on several learning paradigms, including supervised learning (1D linear regression), reinforcement learning (2-armed bandits), and several real-world linear regression problems. These experiments show that LLMs, and GPT-3/GPT-4 more particularly, can effectively adapt their in-context learning algorithm while also matching simple baselines like Bayesian linear regression (BLR), upper-confidence bound (UCB), and random forests. They also show that meta-in-context can outperform standard in-context learning, which is especially relevant given the popularity of this technique with LLMs.

Strengths

1. I found the paper an enjoyable read. The research is relatively well motivated, the analysis are carefully detailed, and the method is clearly explained. In fact, I believe I could easily replicate the results from the paper because all experimental testbeds are precisely described and the exact prompts are provided (which also help illustrate the method; see blue panels). While more baselines could be included for completeness, the experiments clearly show the meta-in-context learning effect so I don't think they are necessary. 2. Despite its simplicity, the presented idea is elegant and I could see it impacting research beyond large language models. For exampe it is not difficult to imagine extensions beyond text inputs (say, vision, audio, or robotics), or theory extensions (in the flavor of "Transformers learn in-context by gradient descent"), or even applying a similar strategy beyond in-context learning (eg, for prompt or prefix tuning). As such, I think it'll garner interest from the NeurIPS community.

Weaknesses

1. Experimental design: given that there is no theory, I find the experimental section a bit light. First, the authors only include results with OpenAI models (GPT-3 and GPT-4) which are not open-source. So it's not clear if meta-in-context learning works as well (or at all?) with publicly available LLMs. Second, can *any* LLM meta-learn in-context? How does the meta-learning ability improve with model/data size? Third, meta-in-context is only tested on small scale and toy datasets -- even the "real-world" ones are only require simple linear regression models. It'd be much more compelling if the authors could show that meta-in-context learning also improves upon in-context-learning on standard NLP tasks. NLI-type tasks would be a promising start but I'd find question answering (eg, SQuAD) or even GSM8K much more interesting. 2. Novelty: The proposed method is similar to "MetaICL: Learning to Learn In Context" by Min et al. and published at NAACL in 2022 -- yet, the authors don't even mention this work. As far as I understand, MetaICL pretrains the model such that the LLM weights are trained to adapt quickly to in-context prompts (hence they have learned to learn), while this work demonstrates that meta-learning emerges even with the standard LLM training objective. It'd be insightful to compare the two approach to quantify how much meta-training helps to adapt quickly. Min et al.'s work is especially relevant because it demonstrates benefits on real NLP tasks. 3. Scholarship: Many references in the bibliography are broken. This might seem a detail but not when a third of the references don't even state a venue.

Questions

- Do open-source models also exhibit meta-in-context learning? Why or why not? - How does meta-in-context learning depend on the size of the model / data it is trained on? - Can meta-in-context learning outperform in-context learning on standard NLP tasks like NLI, SQuAD, or GSM8K? - How is this work different from the Min et al.'s MetaICL paper? - Please fix your bibliography.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The limitations of the work are appropriately discussed.

Author-Reviewer Discussion phase (Aug 10-16)

Today begins the Author-Reviewer Discussion phase, which lasts 1 week (**Aug 10-16**). I ask the reviewers to please **carefully read all other reviews and the author responses and (if appropriate) respond to author responses promptly.** If you've read the author response, please take the time to leave a comment, even if you have nothing to add. I also encourage both authors and reviewers to monitor OpenReview for further comments in order to enable as much back-and-forth as possible during this short period. The authors response mentions a number of new experimental results which may address some reviewers' concerns, and I also note concerns about the practical implications which may deserve further discussion during this phase.

Re: Rebuttal

Thank you to the authors for the response, updates and undertaking the new analyses, particularly the MMLU analysis, which make me lean more confidently towards acceptance. I will review and consider raising my score.

Thanks for the extensive updates

Thank you for the many updates. Here are a few more thoughts and some smaller concerns: * The open-source and smaller-scale experiments look surprising — **thanks!** It's especially interesting to me that Meta-ICL works for MPT-30B but doesn't for the other open-source models, even when they are stronger (eg, LLaMa 2). Interestingly, it's also the model that gets the lowest MSE. Do you have a hypothesis for why that's the case? Could it be that the linear function learning benchmark isn't well suited for these models that are heavily text-optimized? * On the MMLU tasks: it looks like ICL is overtaking MetaICL already at 2 few-shot exemplars. This looks like a negative result for MetaICL on text tasks, which is still interesting. Were you able to tune the number of in-context tasks for MetaICL or was the context length a limiting factor? What accuracy does ICL reach if we allow it to use a similar context length as MetaICL? In other words, can we use MetaICL to make up for a lack of labelled exemplars in one task by leveraging other related tasks? * Regarding Min et al., 2022: thanks for mentioning it. Given that it's an older work, I think this submission could be further strengthened by showing if Min et al.'s pretraining objective is necessary to get the best meta-icl performance. For example, can the open-source models learn to meta-icl if pretrained / finetuned with this objective?