Coherent Predictive Inference under Exchangeability with Imprecise Probabilities (Extended Abstract)

Abstract

Coherent reasoning under uncertainty can be represented in a very general manner by coherent sets of desirable gambles. This leads to a more general foundation for coherent (imprecise-)probabilistic inference that allows for indecision. In this framework, and for a given finite category set, coherent predictive inference under exchangeability can be represented using Bernstein coherent cones of multivariate polynomials on the simplex generated by this category set. We define an inference system as a map that associates a Bernstein coherent cone of polynomials with every finite category set. Inference principles can then be represented mathematically as restrictions on such maps, which allows us to develop a notion of conservative inference under such inference principles. We discuss, as particular examples, representation insensitivity and specificity, and show that there is an infinity of inference systems that satisfy these two principles. 1 The Setting: Predictive Inference We deal with predictive inference for categorical variables, and are therefore concerned with a (possibly infinite) sequence of variables Xn that assume values in some finite set of categories A. After having observed a number ?̌? of them, and having found that, say X1 = x1, X2 = x2, . . . , X?̌? = x?̌?, we consider some subject’s belief model for the next ?̂? ≥ 1 variables X?̌?+1, . . . X?̌?+?̂?. In the probabilistic tradition that we build on in the paper, this belief is modelled by a conditional predictive probability mass function p(·|x1, . . . , x?̌?) on the set A of their possible values. These probability mass functions can be used for prediction or estimation, for statistical inferences, and in decision making. In this sense, predictive inference lies at the heart of statistics, and more generally, of learning under uncertainty. For this reason, it is also of crucial importance for dealing with uncertainty in Artificial Intelligence. We refer to the synthesis by Geisser [1993] and the collection of essays by Zabell [2005] for introductions to predictive inference and the underlying issues that the paper is also concerned with. *This paper is an extended abstract of an article in the Journal of Artificial Intelligence Research [De Cooman et al., 2015]. The predictive probability mass functions for various values of ?̌?, ?̂? and (x1, . . . , x?̌?) are connected by requirements of time consistency and coherence. The former requires that when n1 ≤ n2, p1(·|x1, . . . , x?̌?) can be obtained from p2(·|x1, . . . , x?̌?) through marginalisation; the latter essentially demands that these conditional probability mass functions should be connected with time-consistent unconditional probability mass functions through Bayes’s Rule. A common assumption about the variables Xn is that they are exchangeable, meaning roughly that the subject believes that the order in which they are observed, or present themselves, has no influence on the decisions and inferences she will make regarding these variables. This assumption, and the analysis of its consequences, goes back to de Finetti [1937]. His famous Representation Theorem states, in essence, that the time-consistent and coherent conditional and unconditional predictive probability mass functions associated with a countably infinite exchangeable sequence of variables in A are characterised by, and characterise, a unique probability measure on the Borel sets of the simplex of all probability mass functions on A, called their representation.1 2 The Central Problem of Predictive Inference This leads us to the central problem of predictive inference: since there is an infinity of such probability measures on the simplex, which one does a subject choose in a particular context, and how can a given choice be motivated and justified? The subjectivists of de Finetti’s persuasion might answer that this question needs no answer: a subject’s personal predictive probabilities are entirely hers, and time consistency and coherence are the only requirements she should heed. Earlier scholars, like Laplace and Bayes, whom we would now also call subjectivists, invoked the Principle of Indifference To clarify the connection with our argumentation in the paper, the essence of de Finetti’s argument is that the representation is a coherent prevision on the set of all multinomial polynomials on this simplex [De Cooman et al., 2009b]. As a (finitely additive) coherent prevision, it can be extended uniquely only so far as to the set of all lower semicontinuous functions, but it does determine a unique (countably additive) probability measure on the Borel sets of that simplex, through the F. Riesz Representation Theorem [De Cooman and Miranda, 2008; Troffaes and De Cooman, 2014]. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17)