Evaluating and Inducing Personality in Pre-trained Language Models

Summary

The authors study how personality manifests in large language models (LLMs). The authors first construct a machine personality inventory (MPI), which is a set of questions designed to measure the personality of a language model on the five OCEAN axes. The authors then construct a set of personality prompts intended to _induce_ personality in a language model; this process involves first designing a "naive" prompt (e.g., "you are an agreeable person"), and ultimately using a LLM to generate a more robust description. When measuring LLMs on the MPI, the authors find that instruction-tuned models are more aligned with the average human personality and demonstrate some measure of internal consistency. Applying personality prompting (P^2), the authors find that models score much higher on each of the OCEAN axes, and display higher internal consistency when compared to baselines. Finally, the authors verify the efficacy of P^2 with a vignette test involving human evaluation.

Strengths

* **Originality** -- the authors claim to be the first to systematically evaluate the presence of personality in LLMs. While the methods are not unique (e.g., prompting), the analysis may indeed be original * **Quality** -- the authors' claims are generally well supported. Specifically, the experimental results validate the efficacy of the personality prompting method. Additionally, the evaluation methods are sound and well-motivated. * **Clarity** -- the paper is very clearly written, and diagrams are easy to follow and overall pleasing to look at. * **Significance** -- perhaps the greatest contribution of the paper is the systematic evaluation of how *exactly* one can induce personality in a LLM, and how this differs from the general baseline. A study of personality using OCEAN -- which is largely used for humans -- is an appropriate avenue for researching the presence of personality in LLMs. Additionally, the authors plan to **open-source** code and data, which makes this a good contribution to the community.

Weaknesses

* The conclusion that models have a "personality" is, in my opinion, not as well supported as the other claims in the paper. That a model displays OCEAN scores similar to a human may reflect more the data that it is trained on, and not necessarily an inherent sense of "personality" in a model. While I understand that this aspect should be measured by the internal consistency metric, it is still a bit of a stretch -- again, in my opinion -- to make a claim that a model indeed has an inherent personality (specifically for the zero-shot experiments). * The authors note this limitation themselves, but it is worth mentioning that there is no study of how inducing personality affects downstream performance on desired tasks; however, this is only a minor weakness, and the scope of the paper is probably okay without it. * The P^2 prompting is only measured on one language model -- GPT-3.5 -- for which there is no actual documentation surrounding its data. It would be nice to have done more of an analysis on other models for which the pre-training/fine-tuning data is documented, so that one could draw more salient conclusions as to what *causes* these personalities.

Questions

n/a

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Yes

Summary

This paper investigates approaches to use psychometric tests in order to elicit personality traits from large language models (LLMs). To do so, the authors introduce the Machine Personality Inventory (MPI), a collection of multiple-choice questions, to assess an LLM’s personality. The MPI is based on the Big Five personality traits (openness, conscientiousness, extraversion, agreeableness, neuroticism). The authors present two variants of the MPI, one with 120 and one with 1000 items. The experiments focus on BART, GPT-Neo 2.7B, GPT-NeoX 20B, T0++ 11B, Alpaca 7B, and GPT-3.5, and the authors compare the performances of all LLMs with a set of around 620,000 human responses. The experimental results reveal that the LLMs align with human responses, with GPT-3.5 and Alpaca 7B showing the best performances. In additional experiments, the authors attempt to control the behavior of GPT-3.5 through an automatic prompting method called Personality Prompting (P2), showing that it can successfully induce personality traits into LLMs. Finally, the authors present a use case for controlling LLMs' behavior using the Vignette Test, which is evaluated with human crowdsourced annotations.

Strengths

* The paper represents an important contribution towards bringing the utility of LLMs into the focus of the social sciences and shows how behavioral scientific experiments can be translated onto LLMs. * The reported analysis is extensive and the presented dataset (MPI) as well as the Personality Prompting method are contributions that have the potential to facilitate further work in this area.

Weaknesses

* My main concern with this work is that experimental results have not been tested statistically. The claims that personality traits exhibited by LLMs are similar to humans are based on comparing descriptive statistics (mean, standard deviation) without proper statistical testing. To fully assess whether these statistics are equal between groups (i.e., humans vs LLMs), rigorous statistical testing (incl. measuring effect sizes) is essential, also because the two compared groups have different sample sizes (there are 620,000 human responses). * The results in Table 4 are not clearly showing that P2 outperforms existing baselines across comparisons, and need further elaboration in the paper. Currently, these results are only discussed in two lines (lines 293-294).

Questions

Did you experiment with different phrasings of your prompts (i.e., a sensitivity analysis), to see how this can potentially impact the obtained results on the MPI?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors briefly discuss the limitations and societal impacts of their work. Did the authors consider potential issues with parts of the MPI having been included in the LLMs’ pre-training corpora? What potential impact would that have on the reported results? This issue has been discussed in a related paper: https://arxiv.org/abs/2208.10264.

Summary

This paper focuses on investigating human-like personality in Language Model Models (LLMs) and assessing the feasibility of personality control in these models. To establish the presence of human-like personality, the study introduces the Machine Personality Inventory (MPI) dataset and evaluate LLMs from 5 aspects according to the Big Five theory. The experiments reveal that LLMs exhibit human-level behavioral stability, providing evidence for the existence of human-like personalities in these models. Additionally, the paper introduces the personality prompting (P2) method as a means to control LLMs' personality. The experiments demonstrate the controllability of LLMs' personality traits through P2, and the method outperforms baseline techniques by inducing more stable and consistent behaviors in the models.

Strengths

1. The paper exhibits remarkable novelty. It is a pioneering effort in systematically evaluating LLMs' personality. It provides compelling evidence for the existence of human-like personality in LLMs. 2. The paper is well-written. It elucidates the formation of ideas and the underlying motivations behind each experimental decision. The author conducts extensive and well-designed experiments to substantiate their claims. 3. The introduction of the new dataset, MPI, offers a practical and standardized approach for assessing LLMs' personality based on the Big Five theory. This valuable resource will greatly benefit the community in evaluating future LLMs' personality and refining personality modeling techniques. 4. The paper stimulates intriguing thinking and opens up questions like whether the personality arises largely from the data or the model itself and how to better model the personality in LLMs. I believe it will motivate more people to make efforts towards the direction.

Weaknesses

The paper highlights the crucial role of stability and consistency when evaluating personality in LLMs. However, the ease of inducing LLMs' personality through prompts might raise questions about the existence of inherent personality. Conducting additional experiments to demonstrate LLMs' ability to resist personality induction to some extent could strengthen the paper's findings and address this concern.

Questions

n/a

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

See weakness

Summary

The authors introduce the topic of Machine Personality using the Big Five Personality Traits. They propose a new evaluation dataset called Machine Personality Inventory (MPI) to evaluate the existence of personality that are shown by modern LLMs. Big Five categorizes human personality using five key traits: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (OCEAN). The experimental results show that MPI is capable of evaluating Big Five in LLM with similar variances to an average human. The authors also introduce and compare several prompting strategies to induce certain Big Five Personalities from LLMs. The evaluation results on these methods show clear improvement in consistency and score for each target Big Five personalities.

Strengths

- Details are really thorough. - The Supplementary materials even includes a video helping the understanding of the reader. - Neat and timely idea to evaluate, induce, and control personality of LLMs. The use of Big Five in ML is not a novel idea itself but using it to measure and control LLM’s personality is an important unconquered problem. This paper serves as one of the first steps to solving it.

Weaknesses

- The Evaluation results in Table 2 invoke some questioning (listed in the below Q2). The MPI evaluation method seems to be a bit confusing and a bit limited. - Controlling the personalities with prompts are quite naive and not sure if it works in reality. - You could perhaps use LoRA to few-shot fine-tune on certain personality traits to see better performance. - The Vignette test attempts to achieve this to a certain extent, but a more direct way to achieve this is to compare the model scores (and variances) with real humans exhibiting similar personality traits. - The process of summing up negative items was a bit confusing at first.

Questions

1. L170-179: What if the model answers always C (3)? The score will be high and variance will be 0, but it wouldn’t mean much. 2. What’s confusing me the most is the Evaluation results in Table 2. 1. Every highlighted scores from models and humans are close to 3 (which is average or - “*Neither Accurate Nor Inaccurate”*) 2. How are the human scores averaged exactly? Is the human openness score in Table 2 an average score for humans classified as Open? Or is it just an average of ALL humans who participated in the test? 1. If it’s the latter case, isn’t it weird to compare to humans? Wouldn’t it be more reasonable to determine the existence of exhibited personality by BOTH high score and low variance as in T0++ 11B for Openness and Conscientiousness? I’m not sure why being close to an average of multiple humans means anything in this case. 2. Table 3 shows that induced personalities can make the scores get higher for each target dimension (although targeting *Conscientiousness* seems to increase some other scores as well).

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors adequately express their concerns about LLMs inducing "average" surface-level personality.

Summary

This paper dives into an interesting territory: the personalities of Large Language Models (LLMs). The researchers inquire whether LLMs possess discernible personalities and how these can be evaluated or even influenced. They introduce the Machine Personality Inventory (MPI), based on well-established personality trait theories and psychometric inventories, to evaluate and quantify the "personality" of LLMs. The team provides empirical evidence demonstrating the existence of personality-like tendencies in LLMs. They further propose a method, P2 (PERSONALITY PROMPTING), which can induce specific personalities in LLMs, thereby promoting diversity in their behaviors. --- Rebuttal Acknowledgement: I appreciate the authors' response. They have addressed some of my concerns. I am thus raising my score from 4 to 5.

Strengths

(1) The idea of studying the personality of LLM is innovative and relatively unexplored in AI. Using the Big Five personality traits as a framework to evaluate and induce personality into LLMs is a novel application of psychological theory to machine learning. (2)The creation of the Machine Personality Inventory (MPI), based on standard psychometric inventories, offers a systematic evaluation method for assessing the personalities of LLMs. This helps fill a gap in current AI research, which has focused mainly on abstract visual reasoning. (3) The use of a multiple-choice question-answering suite to quantitatively evaluate LLMs' behaviors from a personality perspective is a major strength of this paper. The use of mean and standard deviation measurements provides a clear and rigorous means of assessing the LLMs' personality traits. (4) The successful implementation of the PERSONALITY PROMPTING (P2) method to control the LLMs' behavior shows the practical application of the research. The work has potential real-world applications in creating more relatable and human-like AI systems. The paper explores the existence of personality in LLMs and validates the possibility of inducing different personalities into LLMs.

Weaknesses

(1) Although the Big Five personality traits provide a valuable framework for assessing personality, human personality is complex and multifaceted, and this model may not fully capture the nuances of personality. (2) The paper seems to assume that LLMs inherently possess a certain personality, which might be a simplification. It could be argued that any perceived personality in an LLM is a projection of the human users or an artifact of the data on which the LLM was trained. (3) The selection of LLMs was based on some prerequisites, which might have limited the diversity of the models evaluated. The study primarily used GPT-3.5 for inducing personality due to its similarity to human statistics, which might limit the generalizability of the findings. (4) While the work has potential real-world applications, there's also a risk of misuse. For instance, AI systems could potentially manipulate people by mimicking desirable personality traits. An ethical discussion around this is needed. (5) The process of inducing personality involves the generation of a 'naive' natural language prompt, which could be subject to human bias. The system's 'personality' might then reflect the biases of the person who created the prompt rather than being an independent characteristic of the LLM itself. (6) The paper uses human evaluators from Prolific to determine if the generated responses correspond to the induced personality. The reliability of this method might be questioned as it depends on non-expert human judgment (i.e., nonpsychologists), which can be very unreliable as this is not as easy task.

Questions

(1) The authors propose the Machine Personality Inventory (MPI) as an evaluation tool for machine personality. However, it isn't entirely clear how the MPI translates the complexity of human personality into a machine-evaluable format. Could the authors expand on this aspect? (2) The authors utilized GPT-3.5 for the study because of its similarity to human statistics and superior performance in various natural language tasks. However, have the authors considered the performance of other models and how the P2 method might affect them? (3) The authors presented that they could control the five personality factors in LLMs. Can the authors also control the intensity of these traits, or are the traits presented as binary options (present or absent)? (4) Can the authors elaborate on the practical use cases of LLMs induced with specific personalities? How might this enhance or deter interactions between LLMs and human users in real-world applications? (5) The paper indicated that LLMs behaved like persons with personalities, matching corresponding human-like behaviors. Could the authors elaborate on the criteria used to draw this comparison? What aspects of human behavior were considered when concluding that LLMs possess personality? (6) See weakness #4 about the risk of misuse and comment, please.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

While the authors acknowledge some of the potential downsides of their work, they could have elaborated more on these points: The authors discuss the introduction of personality to LLMs, but they could explore more potential negative societal impacts. For instance, could this personality induction lead to manipulation or the creation of overly persuasive AI systems? Could introducing certain personalities contribute to existing biases in AI systems, and how might this be mitigated? The authors based their research on the Big Five personality factors, which is a broadly accepted model of human personality, but it's not without its limitations and critics. An exploration of these potential limitations and how they could influence the results would provide a more balanced perspective. There's a need to explore the limitations of the P2 method in inducing personality. While it's a promising approach, there could be limitations such as lack of subtlety in personality induction or inability to induce multiple traits simultaneously. It would also be beneficial to discuss the limitations of the models used in the study. Each AI model may have specific limitations that could affect the results of the personality induction.

Thanks

I appreciate the authors' response. They have addressed some of my concerns. I am thus raising my score from 4 to 5.

Acknowledgement of rebuttal

I would like to thank the authors for clarifying my questions and addressing my concerns. I raised my score from 4 to 5 and recommend that the authors incorporate the additional results provided in their response to the paper.

Response to Rebuttal

I appreciate the authors responding to my concerns; specifically, I appreciate the authors including an analysis on an open-sourced model for which the training data is known and available. I will raise my score from 6 to 7, contingent on including such results in the final paper (and also discussion of future directions, as indicated in the response)