The Epistemological Consequences of Large Language Models: Rethinking collective intelligence and institutional knowledge

In this paper, we interrogate the epistemological implications of human–LLM interaction with a specific focus on epistemological threats. We develop a theory of epistemic justification that synthesizes internalist and externalist conceptions of epistemic warrant termed collective epistemology. Collective epistemology considers the way epistemological warrant is distributed across human collectives. In pursuing this line of thinking, we take bounded rationality and dual-process theory as background assumptions in our analysis of collective epistemology as a mechanism of collective rationality. Following this approach, we distinguish between internalist justification as a robust standard of rationality and externalist justification as a reliable knowledge transmission mechanism. We argue that while these standards jointly constitute necessary and sufficient conditions for collective rationality, only internalist justification produces knowledge. We posit that reflective knowledge entails three necessary and sufficient conditions: a) rational agents reflectively understand the basis on which a proposition is evaluated as true b) in absence of a reflective evaluative basis for a proposition, rational agents consistently evaluate the reliability of truth sources, and c) rational agents have an epistemic duty to apply a) and b) as rational standards in their domains of competence. Since distributed rationality is socially scaffolded, we pursue the consequences of unchecked human–LLM interaction on social epistemic chains of dependence. We argue that LLMs approximate a type of externalist justification termed reliabilism but do not instantiate internalist standards of justification. Specifically, we argue that LLMs do not possess reflective justification for the information they produce but rather reliably transmit information whose reflective basis has been established in advance. Since LLMs cannot produce knowledge with reflective justifiedness but only reliabilist justifiedness, we argue that human outsourcing of reflective knowledge to reliable LLM information threatens to erode reflective standards of justification at scale. As a result, LLM information reliability disincentivizes comprehension and understanding in human agents. Human agents that forfeit comprehension and understanding for reliably correct results reduce the net justifiedness of their own beliefs and, consequently, reduce their ability to perform their epistemic duties professionally and civically. The scaled outsourcing of reflective knowledge to LLMs across collectives threatens the impoverishment of the production of reflective knowledge. To mitigate these potential threats, we propose developing epistemic norms across three tiers of social organization: a) normative epistemic model for individual human–LLM interaction, b) norm setting through institutional and organizational frameworks and c) the imposition of deontic constraints at organizational and/or legislative levels to instill LLM discursive norms that reduce epistemic vices.