Thanks for the engagement! We're revising as follows
Thank you for raising interesting points and helping us refine the paper. We're revising as follows:
> the exact same theoretical model with different choices plugged in could do substantially better or worse in practice.
This is an important point, and relevant to many structured Bayesian cognitive models: there are typically multiple reasonable choices for the prior, likelihood, inference method, and hypothesis space. In our past experience with Bayesian models, good agreement with humans requires at least *some* “tinkering” of these components, and can require sampling budgets larger than what we think humans plausibly process.
Relative to other Bayesian Program Learners we’ve worked with, this new model required less "tinkering", and vastly smaller sampling budgets. For example, all LLMs reported are the first ones we tried (except for logical concepts: we tried codex before GPT4). We needed less “tinkering” as we largely remove a critical degree of freedom: The design of the structured symbolic hypothesis space itself. We also introduce new degrees of freedom, like the choice of LLM, and new continuous parameters for estimating the prior.
Some related models, like Rational Rules, have almost no continuous parameters, but require designing a custom discrete hypothesis space. Piantadosi et al. ‘16 shows such model performance depends on that design. We have extra learnable parameters, but remove combinatorial degrees of freedom as a result.
We'll add this to the discussion:
**Generalizability of the theoretical framework.** The basics of the model make few commitments, yet instantiating it requires selecting specific language models, engineering prompts and likelihoods, etc. More broadly, a high-resolution cognitive model, particularly a structured Bayesian one, requires domain-specific modeling choices. How much credit should we assign to the general theoretical framing, as opposed to particular engineering decisions? Although our paradigm introduces new degrees of freedom (which LLMs/prompts to use), it removes others (the grammatical structure of the symbolic hypothesis space). On balance, we are cautiously optimistic that the framework will generalize with significantly less domain-specific tinkering, at least for abstract symbolic domains. This optimism is because the framework replaces hand-designed structured hypothesis spaces with pretrained neural models, and because reasonable “default” neural networks worked well across our experiments.
> someone includes this model as a baseline… plug in a set of 'reasonable' off-the-shelf models… find that it performs very poorly relative to their approach. Would they then be licensed to reject the whole model?
Following the above discussion, we’d feel comfortable with other researchers using the theoretical framework as a baseline and rejecting it if it flunks their data. (And we couldn’t say the same for DreamCoder, BPL, SOAR, etc.) One nuance, though, is that the proposal distribution needs a strong LLM, which we now have hard data to support, thanks to your earlier input.
> the terminology... 'inference' is being used in a deeply misleading way here... [suggesting] the problem of 'performing Bayesian inference over natural language'... using, e.g., principles of pragmatics and social cognition
Thank you for explaining. We’ll change the title to not include the phrase “Bayesian Inference over Natural Language”, and clarify that our work has nothing to do with the Rational Speech Act model, recursive/social reasoning, etc. Thanks for suggesting possible titles.
We also use “utterance” in the paper. Do you think that could mislead readers into thinking this is about external communicative language? If so, we’ll globally remove “utterance”.
> I'm not familiar with any adjacent literature that uses 'inference' to describe the problem of marginalizing over a pre-existing posterior distribution
Respectfully, we’d like to explain why we think the phrase “Bayesian Inference” is consistent with literature on Bayesian Program Learning and Bayesian computational cognitive science.
We view $q$ as a data-driven proposal distribution. Given that, Lake et al. ‘15 uses “inference” to refer to generating data-driven proposals, which are then weighed by prior and likelihood, like our model. This approximate posterior—the result of inference—is then marginalized over to form predictions, like our model. DreamCoder [Ellis et al. ‘21] also uses this terminology, as does Latent Language [Andreas et al. ‘18] (“inference”, minus the term Bayesian).
Other NeurIPS papers in cognitive modeling have long used similar terminology. From Shi&Griffiths ‘09: “importance sampling provides a simple and efficient way to perform Bayesian inference, approximating the posterior distribution with samples from the prior weighted by the likelihood.”
Thinking of the raw LLM as a pre-existing amortized posterior is an interesting perspective, but not the only appropriate vocabulary.