Demystifying Embedding Spaces using Large Language Models

Summary

The authors propose a framework (ELM) for interpreting the vectors in domain-embedding spaces using text-only LLMs. The main idea is to project vector representations and textual tokens to a conjoint vector space. This is achieved by training adapter layers that serve as an interface to the language model. This adapter layers, together with the pre-trained model, create a new, extended model that maps both tokens and embeddings (from a arbitrary latent metric space) into a common space. The authors put this framework to the test by attempting to interpret two different semantic embeddings that they generate from the MovieLens dataset: One that models the users by their ratings of movies and one that models the movies themselves based on their descriptions. The input data for the ELM framework then consists, for example, of text sequences in which the title of a movie has been replaced by the correspondingly generated semantic embedding. In order to verify the results of the framework, the authors rely on the expertise of test persons on the one hand, and on the other hand, they also design two metrics to test for consistency.

Strengths

I think the strengths of the article lie in the idea of extending a model to accept sequential tokens and embeddings as input data. I agree with the authors that this approach has the potential to provide a tool to study corresponding vector spaces spanned by semantic representations. In addition, this offers interesting possibilities for applications such like recommendation systems. Based on the elaborate and convincing experiments, the authors can very well prove the merits of their approach. Another positive feature is the detailed appendix.

Weaknesses

Unfortunately, in my opinion, the actual core of the article is somewhat overshadowed by the detailed experiments. I would also have liked to see a figure (in addition to Figure 2) that shows the structure of the framework at a glance. In my opinion, the surface planes shown in the 3D graphs of Figure 1 do not contribute much to the clarity either, since they only exemplify a distance of points in the vector space. The authors take effords to show the benefits of their framework with two examples (movie ratings - behavioral embeddings and movie descriptions - semantic embeddings), but perhaps one of the vector spaces studied could have come from a completely different domain, perhaps derived from a different dataset. But this probably falls into future work. On the whole, the paper is very well structured and easy to read, but the section on related work seems a bit out of place before the section conclusions. Perhaps it could have been placed further forward in the text.

Questions

• Has the framework been tested on other embeddings that may contain less semantic information than the domain embeddings used here (similar to the example in Appendix C)? • How do the authors assess the potential usefulness of their approach for the interpretation of other dense vector representations, for example graph or image embeddings?

Rating

8: accept, good paper

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Summary

This paper presents the Embedding Language Model (ELM), a novel language model framework for interpreting domain-embedding spaces. By inserting trainable adapters into existing large language models (LLMs), ELM can accept domain embedding vectors as parts of its textual input sequence to allow the interpretation of continuous domain embeddings using natural language. Abundant experiments demonstrate ELM's proficiency in understanding, navigating, and manipulating complex embedding representations.

Strengths

- Interpreting abstract embeddings into human-understandable natural language descriptions is intuitively appealing. - The proposed approach is simple and effective by reasonably leveraging the power of large language models (LLMs). - The authors have comprehensively assessed the quality of ELM's outputs using a variety of evaluation techniques, including qualitative human evaluations and specific consistency metrics.

Weaknesses

Honestly, one of my greatest concerns is the practicality of the proposed framework, given that training an ELM requires manually constructing a batch of tasks, which need to be diverse enough to extract rich semantic information from the target domain $\mathcal{W}$ to support the interpretation of the embeddings. Admittedly, the experimental part of the paper validates ELM's proficiency in interpreting two forms of embeddings on a movie-related dataset. However, I am not sure if ELM performs equally well in other more specialized domains, considering that the original training corpus of LLMs is likely to contain rich semantic information relevant to movies. In addition, constructing diverse task prompts may be tedious for a realistic user, so are the prompts presented in Appendix D basically applicable to other domains? After all, the most straightforward need for a user is to know what an embedding represents in general. - Given the prevalence of adapter tuning [1], there is nothing new to me in the method. 1. Parameter-efficient transfer learning for NLP, Neil Houlsby et al.

Questions

- What is the detailed form of the loss function $\mathcal{L}\_{\theta}$ used in the experiment? Did the authors use reinforcement learning from AI feedback to optimize $\mathcal{M}_{ELM}$? - Since the output of instances in the training data is generated by LLMs, can ELM reasonably interpret embeddings in specialized domains outside the scope of LLMs?

Rating

6: marginally above the acceptance threshold

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Ethics concerns

No ethical issues found.

Summary

This paper presents ELM, a framework that uses LLM to interpret embeddings. By training an adapter, ELM maps the domain embedding space to the LLM token embedding space, allowing users to use natural language to query and understand the domain embedding space. An evaluation of 24 original tasks with a movie rating dataset shows that ELM provides high semantic and behavioral consistency. Finally, the paper shows promising results of using ELM to query hypothetical embeddings and generalize concept activation vectors.

Strengths

- [S1] The proposed method ELM is novel and intuitive. The two-stage training (adapter first, full model next) is interesting and makes sense. - [S2] I appreciate including human evaluation in section 3. The analysis and discussion on hypothetical embedding vectors and concept activation vectors (section 4) are very interesting. - [S3] The paper overall is well-written and easy to follow.

Weaknesses

## Major weaknesses - [W1] The evaluation can be improved. (1) The training data are all synthesized from an LLM. (2) The 24 tasks are all original. (3) ELM is only evaluated on the MovieLens. (4) There is no baseline method. It is unclear how well ELM will perform on real data and tasks compared to other methods. ## Minor weaknesses - [M1] It would be great to explain $E_A$ in Figure 2 (the definition is on page 4). - [M2] The 24 tasks are essential to understand this paper. I recommend adding a table to provide a high-level description of these 24 tasks in the main paper. - [M3] The ELM method is quite intuitive, but the writing in section 2 is overly-complex. It would be helpful to have a small glossary to explain all notations. - [M4] It is hard to make sense of Table 1. Are these numbers good?

Questions

- [Q1] Have you tried ELM on other datasets and tasks other than movie reviews? - [Q2] Figure 1 is a bit hard to understand. For example, how do people generate the embedding for the animated version of Forrest Gump? Interpolating the embedding of Forrest Gump with an animated movie? Is the black text output from an LLM? Who writes the blue text (e.g., users, researchers)?

Rating

8: accept, good paper

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Summary

This paper presents a paradigm for demystifying the compressed embedding representations of deep models (such as MF trained on the MovieLens dataset in the paper) with Large Language Models (ELM). ELM first generates training data by prompting PaML 2-L with the movie title and task-specific information. The training procedure is composed of two stages: (1) training an adapter to project the domain embedding; and (2) full-training the full model including the language model and the adapter. Then the authors construct different tasks on the MovieLens dataset to test the performance of demystifying domain embeddings.

Strengths

1. It is sound to demystify domain embedding with Large Language Models. 2. The authors introduce their approach step-by-step in Sec 2 & 3, which is clear. 3. The performance is evaluated by human raters.

Weaknesses

1. The training data is generated by PaML 2-L, and ELM uses PaLM 2-XS to interpret the domain embeddings. Their similar architectures and training procedures may make the contribution limited. 2. The authors argue that ELM is a general framework for different tasks, but the experiments only involve one model (MF) on one dataset (MovieLens). Therefore, the generalization of ELM to other models and datasets is not guaranteed. 3. Some related works are overlooked. Existing works [1][2] have shown that tuning the adapter can let LLM demystify and reason over the embeddings of images. The authors should provide more explanations and experiments for their differences. reference: * [1] Zhu, Deyao, et al. "Minigpt-4: Enhancing vision-language understanding with advanced large language models." arXiv:2304.10592 (2023). * [2] Koh, Jing Yu, et al. "Generating images with multimodal language models." arXiv: 2305.17216.

Questions

1. Can ELM perform well with training data generated by other LLMs such as ChatGPT? 2. Can the training data be real other than generated? Some datasets contain descriptions and revives (such as Amazon datasets). Why not use these kinds of datasets? 3. Can ELM perform well with other models and other datasets.

Rating

5: marginally below the acceptance threshold

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Ethics concerns

NA

Additional Experiments on the Amazon Dataset

Dear reviewers, Thank you all again for your reviews and your helpful comments. We've added new experiments using the public Amazon dataset (see Appendix A in the updated version of the paper). We focus on a subset within the "Clothing, Shoes, and Jewelry" category, which involves 5.7 million reviews for 1.5 million products. Similar to the Movielens experiments, we introduced both semantic and behavioral embeddings. Behavioral embeddings were trained on users' rating data, while semantic embeddings were generated using item titles, descriptions, categories, and other meta information. We describe this in detail in Appendix A (all changes marked changes in red). Our evaluation of ELM includes six tasks: item description, positive review, negative review, neutral review, item endorsements, and user profile. We are still waiting for human rating results for the user profile tasks, but already have comprehensive results of all other tasks in the paper. Thank you for helping us improve our paper. We hope our new experiments address all reviewers' concerns.