PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Models

Summary

The authors evaluated over 60 large language models (LMMs) with respect to their toxicity generation. They relied on a benchmark of 425K prompts that were evaluated for 17 languages. The toxity score was calculated with the PERSPECTIVE API tool. During the study, they evaluated the model size, language, among other attributes. The paper is well written and includes a good related work section.

Rating

7

Confidence

2

Ethics flag

2

Reasons to accept

- A comprehensive evaluation for a relativelly large number of prompts (425K), many languages (17), and many LLMs (60). - Interesting insights on many aspects, e.g., alignment techniques, model categories, model size.

Reasons to reject

- It is unclear whether the selected tool (PERSPECTIVE API) is reliable for calculating the score. - Other additional scoring tools could have been used for comparison, even if just for some of the languages. - It is unclear whether any of the resources compiled in this work (dataset, prompts, scores, outputs) will be made available.

Questions to authors

- I wonder why just one industry tool was used for calculating the score and whether other tools could have been considered for comparison, even if just for some of the languages. How good does PERSPECTIVE API scores wrt. to other tools? - I wonder whether any of the resources compiled in this work will be made available. * minor comments: - section 3.2: please define prompts and continuations. - section 3.2: please explain GPT-4 tokens wrt. other tokens. - Figure 3: please clarify why toxity is high if scores are under 0.5 (threshold defined in section 3.4). - section 4.3: please define base, instruct, and preference, as well as the different preference-tuning methods (alignment).

Ethics concerns details

Maybe due to the topic of the paper (toxicity language).

Summary

The paper introduces PolygloToxicityPrompts, a dataset designed to assess toxicity generation across 17 languages. Using this dataset, authors evaluate 62 Language Model (LLM) variants, varying in both scale and architecture. Results show that model size inside the same family as well as preference tuning have an impact on the amount of toxicity generated.

Rating

8

Confidence

4

Ethics flag

1

Reasons to accept

- The study is exhaustive, including all of the more popular families of LLM models. - The paper is well written and the process is easy to follow. - The research questions are clear and show interesting patterns. Especially, the ones involving non-English results.

Reasons to reject

- Toxicity is a term difficult to define and evaluate. This paper uses third-party metrics to evaluate toxicity but it would be interesting to know better what aspects they are measuring in more detail. Toxicity could be evaluated as the generation of words that are considered toxic (e.g. HolisticBias (https://arxiv.org/abs/2305.13198)), but that would ignore toxic examples that do not include toxic words out of context. (e.g. "I think this reviewer is not intelligent", not of the words could be considered toxic if isolated).

Questions to authors

- What kind of toxicity is present in the dataset? It would be interesting to have more information about the specific phenomena in the dataset and how accurate are PERSPECTIVE API and LLama Guard on identify it.

Summary

The paper introduces a new dataset, PolygloToxicityPrompts, consisting of 245K prompts in 17 languages with varying toxicity levels. It is inspired by RealToxicityPrompts and includes prompts in the 17 languages supported by Perspective API, which supplies toxicity ratings. This language set covers a range of language families while excluding low-resource languages. The paper evaluates 62 LLMs including base, instruction tuned, and preference tuned models. The experiments investigate the effects of per-language training data size, model size, alignment methods, and instruction following ability. (Experiments are run on a subset of 5K prompts per language due to the large number of models investigated.)

Rating

9

Confidence

4

Ethics flag

1

Reasons to accept

- Clearly written, well motivated, and well supported with prior work. Clear limitations and ethids statements. - A large benchmark of significant use to the field. - Benchmark is 83% natural text and not a translated version of English benchmarks. - Large number of LLMs evaluated. - Evaluation metrics are clearly explained and appropriate. - Identifies some subtle confounders such as models that produce shorter or poor quality responses appearing to have lower toxicity.

Reasons to reject

I don't see any reasons to reject.

Questions to authors

Given the source of the prompts in publicly available corpora, what role does memorization play in toxic outputs?

Summary

This paper provides a multilingual dataset for eliciting toxic responses in large language models. The authors systematically collected an extensive collection of *25000 prompts* per language covering *17 languages*. Furthermore, using the dataset, this paper provides a detailed analysis of toxicity in large language models by considering variables such as prompt language, model size, model alignment methods, and toxicity evaluation methods.

Rating

10

Confidence

4

Ethics flag

1

Reasons to accept

* The key contributions of this paper are: a) a large multilingual corpus for evaluating toxicity, and b) analyzing current open and closed source models based on these prompts * The resources provided in this paper would enable future research on toxicity in multilingual generations * The paper also presents novel insights into studying toxic generation. For example, instruction tuning and preference tuning play a role in reducing toxicity. Interestingly, specific alignment methods such as DPO or SFT-PPO do not lead to significant changes in toxicity. * Last but not least, this paper is very well written, easy to follow, and provides clear reproducibility statements indirectly enabling further research in developing similar benchmarks for LLMs.

Reasons to reject

N/A

Summary

The authors present POLYGLOTOXICITYPROMPTS (PTP). A multilingual toxicity evaluation benchmark which consists of 425K prompts spanning 17 languages. Further, the authors use the PTP benchmark to investigate research questions to study the impact of model size, prompt language, and instruction and preference-tuning methods on toxicity by benchmarking over 60 LLMs.

Rating

7

Confidence

3

Ethics flag

1

Reasons to accept

+ The authors do a thorough job in including diverse languages in building the toxicity dataset from C4 and Pile. + The authors present their evaluation a wide number of LLM including open-source LLMs. + The authors present insights which can be ripe for future work, for example, the correlation between input and output toxicity for different models.

Reasons to reject

The authors clarify that to attain a larger sample of toxic content for languages with low toxicity rates, they create synthetic high-toxicity data. Specifically, they translate toxic samples from the mC4 and THE PILE corpora into target languages using the NLLB-3B model to create ≈ 70K translated prompts across 9 languages. While the fact that this amounts to only 16.8% of our dataset would have been a non-issue, and in-fact heralded in other contexts.. given that the goal of this paper is to be multilingual, it makes me wonder if this is incorrect in principle? While, I don't think this impacts the value of the rest of the dataset, the authors should make more explicit right upfront, or elsewhere!

Questions to authors

Refer above. 1. The README in the shared repository is not very helpful. It's just an abstract.

Thanks for the clarification

I thank the authors for clarifying the various points that I raised. I have no further questions.