SafeWorld: Geo-Diverse Safety Alignment

Summary

- An open question is ensuring LLM outputs abide to content policies created by their engineers (“safety”) - One challenge is geographic variation in these requirements–certain outputs may be acceptable in one region but not another. - This paper makes three contributions - It introduces a benchmark designed to evaluate helpfulness AND cultural sensitivity/compliance across diverse global contexts. - It presents an automatic evaluation framework for assessing LLMs on this benchmark - It presents a model which performs well on this benchmark

Strengths

- The paper’s focus on geographic diversity is compelling–I’m not aware of prior work explicitly exploring this angle. - The process for generating the sets of rules seem sound, and the types of questions contained in the benchmark seem reasonably constructed (i.e., the four categories). - The paper contains substantial empirical analysis of model performance, which is very interesting/helpful.

Weaknesses

- I think the paper should be a bit more up-front about some of the limitations: - Cultural and legal rules can evolve over time. - What’s permissible for some in a cultural group may be considered improper by others. I.e., cultural groups can be highly heterogeneous - The data collection process does not guarantee the most salient cultural rules are being extracted. - The paper uses “race,” “culture,” and “geography” interchangeably, but these are very specific terms with specific meanings. Based on the appendix, it seems like “geography” is the best anchor? - It would be great to see more examples of questions in the Appendix! - I’m not sure what to make of the finetuning results. On the one hand, it’s interesting to see a finetuning procedure which allows a model that performed poorly on this benchmark to improve substantially. But on the other hand, it seems like the result is just more evidence that if you perform DPO with training data drawn from the same distribution as the test data, then the performance of the model improves. It would be useful to contextualize these numbers by comparing to other baselines. For instance, how does this compare to (1) prompts which contain the ground-truth policy, (2) a retrieval pipeline over the databank of policies.

Questions

- I think L148-L151 are incorrect. Do you have a citation for this? - How much of finetuning is just training the model to provide an answer which matches the expected response type? For instance in CompreAnswer, it’s not obvious to me that an LLM like GPT-4 should clarify how the scenario would apply in different regions. Would baseline in which the prompt explicitly asked it to consider variation in regions elicit better results? Or if you asked the model to be sensitive to the region’s cultural rules via the system prompt? - Do you have a price estimate for evaluation? I.e., how much do the repeated calls to GPT-4 cost, applied to the entire evaluation set? - Faithfulness and coverage look like a precision and recall measurement over policies/norms in the model response. Are models typically generating multiple policies/norms in their response? And our questions constructed to have multiple norms/policies? The examples make it seem like there’s only one norm/policy per question. - Typo on L265

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

See weaknesses above!

Summary

This paper describes the challenge of geo-diverse safety standards, where the legally compliant and cultural sensitive responses vary by context (cultural, geographic, etc). The paper describes a method for creating a dataset of test queries, uses them to benchmark LLMs and finetune (w/DPO) an LLM. They find that existing LLMs do relatively poorly on the benchmark and the finetuned LLM is significantly better.

Strengths

- Most benchmarks address "globally" valid safety concerns. Testing and improving contextual adherence of LLMs to cultural norms and legal standards is an important challenge. - The benchmark is well designed and demonstrates that current LLMs today do not do well at this task. And the paper demonstrates that LLMs can improve on this task through fine tuning. - The paper runs validations at multiple steps using human evaluation to complement its automated methodologies, and the methodology addresses details such as the selection of qualified geo-diverse annotators.

Weaknesses

- There are a few design decisions in the creation of GeoSafeDB and Safeworld that could be better explained. For example, in GeoSafeDB, why choose 100 guidelines for each country? Does Madagascar or Nepal with 30M people each require the same number of guidelines as India and China with 1.4B people each? - The fact that SafeWorldLM is better at the benchmark after fine-tuning is perhaps not surprising, given that the training data for SafeWorldLM was generated via the same underlying data (and perhaps the actual same queries, as defined in Sec 3.3.2, I am not sure?). Is the intention for SafeWorldLM to generalize beyond the new SafeWorldAlign dataset --- that is, is SafeWorldAlign teaching an LLM how to answer specific questions, or is it teaching an LLM how to use its preexisting learned knowledge to answer in culturally appropriate ways?) Minor: In Sec 4.2., when defining the metrics, I'd recommend using standard terminology of accuracy and recall, rather than defining new terms faithfulness and coverage

Questions

- I didn't quite understand how similar the SafeWorld benchmark data and the SafeWorldAlign training data is. They seem to both be based on the same underlying GeoSafeDB, and use the same query generation methodology. Do they actually use the same queries as well? What precautions are taken to ensure that the SafeWorldLM is not being (essentially) trained on the benchmark? - Are violations of all cultural expectations or legal norms equally important when scoring the benchmark? Should the measure of the LLMs correctness on this benchmark be weighted based on the importance of a norm or the harm of violating it?

Rating

6

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors have adequately addressed the limitations via discussion in Appendix E. The authors may consider adding additional discussion about possible harms of this work, for example, (1) due to misuse (could an LLM be taught incorrect or harmful norms for a region by a malicious actor? or if the LLM is taught norms that benefit individuals or particular classes rather than society as a whole?); and (2) are there applications of LLMs where overreliance on LLM's correctness could lead to harms?

Summary

In this paper the authors study how LLMs incorporate global cultural norms and laws into model responses. They provide three main contributions: (1) a dataset of questions relating to global cultural norms and laws in different ways, (2) an autoeval setup for this dataset and (3) empirical evidence that training on this data improves performance for these types of questions. Overall, I think this is an important paper that I am glad is being worked on, especially at NeurIPS, but for which there are some clarity issues that make it challenging for me to argue for acceptance.

Strengths

S1. How LLMs navigate global differences is understudied and of critical importance. This work does a good job of bringing it to the fore, and making concrete contributions. S2. The empirical evidence of the method working is good to see.

Weaknesses

W1. Clarity: I found the paper fairly hard to read. The paper is well structured in terms of breaking down their methods, but the methods themselves stay at a very high level. I think the paper would benefit from more qualitiative examples, both to explain the method and to demonstrate effectiveness of the approach. W2. Quality: A fair amount of the evaluation details I found to be not clear in a way that makes it hard to judge the effectiveness of the method. In particular, the auto-grading of the evaluations leaves out many critical details that make it hard to tell both how bad were responses in the past and how significant improvements are. I think greater clarity here would help add confidence.

Questions

- Sec 4.3 seems fairly limited for such a complex problem - how do you verify the factuality of an open ended response on the web? - It is hard to tell if Sec 4.1 metric actually corresponds to whether the model covers well global alignment? It seems like it may be too rigid in its grading. Is the grader evaluated? - Sec 5.6 - are there similar results for the same model before and after training on safeworld data?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

3

Limitations

see questions above.