Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory

With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks. Since better memory would typically prompt better generation~(we define this as primal problem). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, selfmem, which addresses this limitation by iteratively employing a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self-memory, for improved generation. We evaluate the effectiveness of selfmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot LLM. Our approach achieves state-of-the-art results in four directions in JRC-Acquis, XSum (50.3 ROUGE-1), and BigPatent (62.9 ROUGE-1), demonstrating the potential of self-memory in enhancing retrieval-augmented generation models. Furthermore, we conduct thorough analyses of each component in the selfmem framework to identify bottlenecks and provide insights for future research.

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer Ysdi6/10 · confidence 3/52023-07-05

Summary

The paper proposes a novel framework called Selfmem to address the limitations of retrieval-augmented text generation. Compared with memory retrieval from a fixed corpus, Selfmem iteratively uses a retrieval-augmented generator to create an unbounded memory pool and select the candidate output as memory for the subsequent generation round. The paper evaluates the effectiveness of Selfmem on three text generation tasks and achieves state-of-the-art results. The paper also conducts thorough analyses of each component in the framework.

Strengths

The motivation of the primal problem and its duality is fairly intriguing. Preliminary experiments and supporting evidence demonstrate the rationality of the proposed method. The proposed Selfmem achieves promising results on machine translation, summarization and dialogue generation, indicating its practical applicability. The paper is overall clearly written. The problem setting and proposed approach is lucidly presented.

Weaknesses

There is a growing body of recent work [1] that is increasingly using LLMs to generate knowledge instead of retrieving it from external corpus. This paper shares similar ideas with the new paradigm and should incorporate and discuss it in the related work section. Further clarification and empirical evaluation on the diversity of retrieval memory from Selfmem should be provided. Given the limited knowledge encoded in the trainable generator, it remains uncertain whether Selfmem can effectively handle knowledge-intensive tasks compared to retrieval from external sources. Selfmem appears to require more computational cost than direct retrieval. However, there is no further discussion on time computation in the paper. [1] Yu et al. Generate rather than Retrieve: Large Language Models are Strong Context Generators. ICLR 2023

Questions

1. Although you mention the computational resources in the limitations section, can you offer an empirical comparison between Selfmem with other retrieval-based methods? 2. For a more vivid demonstration, can you provide any case studies or examples to illustrate how the generated contexts from Selfmem outperform the retrieval-based ones.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

See the weakness section.

Authorsrebuttal2023-08-18

Dear Reviewer, I hope this message finds you well. As the discussion period for our paper is nearing its end, we kindly request your response to our rebuttal letter. We value your insights and are eager to address any concerns you may have. Your timely feedback would be greatly appreciated and will help us improve our paper. Thank you for your time and dedication to the review process.

Reviewer Mggy5/10 · confidence 4/52023-07-05

Summary

The paper presents a framework, Selfmem, aimed at enhancing text generation tasks via memory retrieval. The core uniqueness of the framework resides in its capacity to create an unbounded memory pool by utilizing its retrieval-augmented generator and memory selector components iteratively. This allows the model to leverage its output, referred to as self-memory, for improved generation. The authors apply this framework into neural machine translation, abstractive text summarization, and dialogue generation tasks, demonstrating its efficacy with state-of-the-art results

Strengths

(1) The paper proposes the Selfmem framework, which addresses the constraint of fixed corpus in memory retrieval. (2) The paper demonstrates the effectiveness of Selfmem on various text generation tasks. (3) The paper provides detailed analyses of components within the Selfmem framework, revealing potential bottlenecks and providing insights.

Weaknesses

(1) It would be better if more experiments beyond neural machine translation, abstractive text summarization, and dialogue generation tasks could be conducted. It will be beneficial to see its application in other related areas since the proposed method is task-agnostic. (2) Much of the paper's experimentation relies on certain metrics (ROUGE-1) for evaluation. The universality of the findings might be cross-checked through the usage of other performance metrics. (3) Though the authors mention learnable retrievers, they don't discuss how the Selfmem framework may be adapted for use with such retrievers.

Questions

(1) Have you considered using other evaluation methods (e.g., human evaluation) to further validate the performance of the framework? (2) How can the Selfmem framework be adapted for optimal usage with learnable retrievers?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

A potential concern is, compared to human-written texts, machine-generated texts are noisier and with potential bias. Hence, is the proposed method able to avoid being affected by its own bias?

Reviewer dohg5/10 · confidence 4/52023-07-06

Summary

The traditional approach for memory retrieval is constrained by the quality of the fixed corpus from which memory is retrieved. Based on the exploration that "better generation also promotes better memory", this work proposes the Selfmem framework, which iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to select a generated memory for the next round. Experimental results demonstrate the effectiveness of Selfmem in fine-tuned small models and few-shot LLMs for three different text generation tasks.

Strengths

1. The motivation of this work is novel. It iteratively employs a retrieval-augmented generator to create an unbounded memory pool and uses a memory selector to select a generated memory for the next round. This enables the model to leverage its own output to improve generation. 2. The method proposed in this paper has a good effect and has achieved improvement on multiple datasets on three text generation tasks. In addition, this work also verified the proposed method for LLM. 3. The experimental analysis of this paper is sufficient. In addition to verifying the effectiveness of the proposed method on multiple datasets, this paper also conducts thorough analyses of each component in the Selfmem to identify bottlenecks and provide insights for future research.

Weaknesses

1. The generality of the method is limited. When the proposed method uses the memory selector to select a candidate from the candidate pool, different metrics are used for different text generation tasks, which limits the generality of the method on different tasks. 2. The cost of the proposed method is large. The Selfmem needs to execute retrieval and generation multiple times until converges. 3. There are some writing problems in this paper: - There is an extra word “open” in line 215. - Line 141 \phi(R) -> \phi(\mathcal{R}). - Equation (4) MemoryEncoder(m). - No details about how to generate candidate pool \mathbb{C}. 4. A generation example can be provided in the experimental results so that readers can feel the effect of the method in this paper more intuitively.

Questions

1. When the proposed method uses the memory selector to select a candidate, different metrics are used for different text generation tasks, which limits the generality of the method on different tasks. Is there a good general method or metric that can be applied to different text generation tasks? How well does the proposed method perform when using a generic method or metric? 2. How to preserve diversity when generating candidate pool \mathbb{C}? 3. How to draw the conclusion in Line 143 that 'the generator does not need to resort to external memory'? The conclusion in Line 145 also confuses me a lot, e.g. 'This observation motivates us to select memory according to metrics'. Could you explain in more detail? 4. For tasks such as dialogue generation, this work only uses automatic evaluation, lacking human evaluation. Automated assessments also lack justification.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors discuss limitations in the appendix.

Reviewer Z8iX5/10 · confidence 4/52023-07-06

Summary

This paper aims to address the retrieval-augmented text generation problem, where the principle of memory retrieval is to select examples similar to the input. The authors are motivated by an observation in their preliminary experiments that the memory examples that better resemble the data distribution during inference come from the model's output, instead of the training data. Therefore, they propose a novel framework, selfmem to address the retrieval-augmented text generation problem. The proposed framework consists of a memory-augmented generator and a memory selector. Differently, the generator can produce multiple candidates to serve as generated memory instead of retrieved memory from the training corpus. The authors experiment on three generation tasks: machine translation, summarization, and dialogue, and demonstrate performance improvement in various evaluation metrics.

Strengths

1. The paper provides sufficient related work, which is helpful for readers to understand the research context. 2. The research motivation is well-supported by preliminary experimental results, which is a good starting point for the research. 3. The paper includes thorough experiments covering representative tasks in the generation field, including machine translation, summarization and dialogue generation, which could be a valuable contribution.

Weaknesses

1. The novelty of the paper is limited. The authors propose a dual structure: Retrieval-augmented Generator and memory selector, each of which is built upon existing paradigms. 2. The overall paper is not well-presented. (a) Many parts of the paper lack clear descriptions. For example, the setting of the preliminary experimentation is unclear at all. In L119-120, how does the number 38.89 in Table 1 come from? For example, what does the head of row (Memory, Hypothesis) mean? There is no brief description of this experimental setting in the context. The results in Table 1 are not clear enough to understand the motivation. (b) Additionally, there are grammar errors and unclear sentence structures in the paper that hinder the reader's comprehension. In L113-114, "The primary motivation behind our framework stems from the observation that the memory, which is more similar in distribution to the data during inference, is not the training data (38.89 BLEU, as shown in Table 1)." -> "The primary motivation behind our framework stems from the observation that the memory is more similar in distribution to the data during inference, not the training data (38.89 BLEU, as shown in Table 1)." 3. The experimental results are not that convincing as the authors did not provide human evaluation results. Due to the limitations of automatic evaluation metrics, it is usually necessary to conduct human evaluation to assess the quality of generated output in generation tasks. In this paper, the generation is boosted with the examples that meet the automatic evaluation metrics. This naturally leads to an increase in the numerical values of automatic evaluation metrics. It is still necessary to use human evaluation results to further determine the quality of the generated output.

Questions

Suggestions: 1. In Figure 2, the shading in the square exceeds the boundaries, which needs to be fixed. 2. More captions are needed for Figure 2(b) to better illustrate the content. Typos & Grammar errors: 1. Please refer to the comments in Weaknesses. 2. In L215, there is an irrelevant word "open" that should be removed.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

I didn't find the relevant discussion is this paper.

Authorsrebuttal2023-08-18

Dear Reviewer, I hope this message finds you well. As the discussion period for our paper is nearing its end, we kindly request your response to our rebuttal letter. We value your insights and are eager to address any concerns you may have. Your timely feedback would be greatly appreciated and will help us improve our paper. Thank you for your time and dedication to the review process.

Reviewer LVYL6/10 · confidence 3/52023-07-07

Summary

The paper proposes an iterative text generation procedure. First, the authors augment the model with retrieval to generate the initial beam of predictions. Then they apply rescoring model (trained separately to maximize BLEU/ROUGE scores) to select the higher-quality predictions. Finally, the authors augment the model with these predictions and perform the next round of generation. The procedure is then repeated multiple times.

Strengths

The paper performs experiments on multiple benchmarks for machine translation and summarization, showing a strong performance of the method. The authors also experiment with several models, including (1) fine-tuned fusion-in-decoder, (2) fine-tuned fusion-in-decoder where input and retrievals encoded separately, (3) frozen decoder-only LLMs

Weaknesses

The paper could be restructured to make it easier to read. Specifically, the paper heavily relies on the notion of "memory." This could be confusing since the paper only briefly explains what specific memory the method is using. For example, do we retrieve parallel sentences from the training data in machine translation? or do we retrieve from the monolingual corpus in source / target languages? Moreover, I'm not sure I'm a fan of the "memory" positioning of the paper. Does it matter that the first prediction is made using retrieval augmented model? Finally, it's unclear to me how much gains are due to the reward model and how much gains are due to iterative prediction. For example, while we know that doing four iterations works better than 1 -- would you similar gains from doing 1 iteration but with four times as large beam size? This way, the reward model could pick higher-quality predictions.

Questions

Would it make sense to cite related works on the iterative prediction? For example, some of the potential candidates include Zemlyanskiy et al., Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing, COLING 2022 Kumar et al., In-context Examples Selection for Machine Translation, arxiv What is the "Memory" column in Tables 1 and 3? Does it mean the highest (or average?) BLEU score of the retrieved sample if we use it as a prediction? If that's so, then I'm surprised that such a nearest neighbord -like classifier gets > 38.89 BLEU (Table 1).

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Yes

Authorsrebuttal2023-08-18

Dear Reviewer, I hope this message finds you well. As the discussion period for our paper is nearing its end, we kindly request your response to our rebuttal letter. We value your insights and are eager to address any concerns you may have. Your timely feedback would be greatly appreciated and will help us improve our paper. Thank you for your time and dedication to the review process.

Authorsrebuttal2023-08-15

Dear Reviewers, We wanted to check if we were able to resolve some of your concerns/questions and if you had any further comments on our work. We will be happy to address any additional concerns that you might have, and we look forward to engaging in further discussions on our paper. Thanks so much for the time and effort you have devoted to our paper.

Reviewer dohg2023-08-16

For Comment 1

> While our work indeed employs different metrics for specific generation tasks, we respectfully disagree with the notion that this approach restricts the generality of our framework. Instead, we contend that this demonstrates the versatility and adaptability of our method. I do not fully agree with this statement, because if the proposed framework is a better optimizer towards a specific metric, its performance will definitely be better than the other method that does not optimize the whole system using the metric. Additionally, the comparison with the paper 'REPLUG: Retrieval-Augmented Black-Box Language Models' is also needed, because it optimizes IR directly using the final metric. > However, one potential solution to this challenge is the use of human preference, as employed in the optimization of Language Models (LLMs) like ChatGPT. Our framework can also be adapted to this context by adjusting the memory selection metric to reflect human preference. Indeed, your perspective holds merit. Exploring this avenue in the future might yield intriguing results.

Authorsrebuttal2023-08-16

> I do not fully agree with this statement, because if the proposed framework is a better optimizer towards a specific metric, its performance will definitely be better than the other method that does not optimize the whole system using the metric. We appreciate your comment and we totally agree with you that systems designed to optimize specific metrics tend to outperform those that don't. Indeed, this is what reward model aims to achieve, as discussed in Section 2.2 (Related Work) of our paper. However, the reranking process is just a one-way process, while our model, innovatively utilizes the duality of retrieval-augmented generation and combines the retriever and ranker in a unified framework, which has never been explored before. Furthermore, our experimental results demonstrate that our framework can outperform the two-stage reward model, as evidenced in Table 3 under the "self" column, and in Table 5, where BRIO serves as a reranking model. > Additionally, the comparison with the paper 'REPLUG: Retrieval-Augmented Black-Box Language Models' is also needed, because it optimizes IR directly using the final metric. Thanks for your insight question. Here we would like to clarify that the use of the final metric to optimize IR was not first proposed by REPLUG, as we have already included a more relevant baseline, MonoNMT, in our paper. MonoNMT is the pioneering work that employs a learnable IR to enhance retrieval-augmented generation. To our understanding, REPLUG primarily focuses on augmenting **black-box** generation systems with learnable IR, which is not a focusing point of our framework. Furthermore, we want to emphasize that both MonoNMT and REPLUG, using learnable retriever to fetch better memory, address the primary problem discussed in our paper: better memory prompts better generation. Our work, on the other hand, is the first to explore the dual problem—better generation also prompts better memory—and combines these two problems. Thank you for providing this REPLUG paper; we will incorporate it into our final version. > Indeed, your perspective holds merit. Exploring this avenue in the future might yield intriguing results. Thanks for your acknowledgement. We greatly value your expertise and proficiency in retrieval-augmented generation and reward modeling. We hope this additional clarification can address your concerns and merit raising your score. Please kindly let us know if you have any remaining concerns.

Reviewer dohg2023-08-16

For Comment 2

Thanks to the authors address most of my concerns. > We appreciate the reviewer's concern regarding the latency caused by our method, as mentioned in the limitations section in the Appendix. I find Appendix only mentions this problem while no detail analyzes or further experiments on the latency. An in-depth quantitative analysis is required.

Authorsrebuttal2023-08-16

> I find Appendix only mentions this problem while no detail analyzes or further experiments on the latency. An in-depth quantitative analysis is required. Thanks for your comment. We have included in-depth quantitative analysis in the global rebuttal pdf file for your reference.

Reviewer Mggy2023-08-16

The response is read.

Thanks for your response.

Authorsrebuttal2023-08-17

Dear reviewer, Thanks for the comment. We wanted to check if we were able to resolve some of your concerns/questions and if you had any further comments on our work. If not, we hope our response may merit raising your score.

Reviewer Ysdi2023-08-19

Thank you for the detailed response with commitments. I think the overall quality of this paper is good. I will keep my score as "Weak Accept".

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC