Academic literature on the topic 'Bayesian surprise'

This dissertation investigates how grammatical functions in transitive sentences (i.e., `subject' and `direct object') are distributed in written Swedish discourse with respect to morphosyntactic as well as semantic and referential (i.e., prominence-based) information. It also investigates how assignment of grammatical functions during on-line comprehension of transitive sentences in Swedish is influenced by interactions between morphosyntactic and prominence-based information. In the dissertation, grammatical functions are assumed to express role-semantic (e.g., Actor and Undergoer) and discourse-pragmatic (e.g., Topic and Focus) functions of NP arguments. Grammatical functions correlate with prominence-based information that is associated with these functions (e.g., animacy and definiteness). Because of these correlations, both prominence-based and morphosyntactic information are assumed to serve as argument interpretation cues during on-line comprehension. These cues are utilized in a probabilistic fashion. The weightings, interplay and availability of them are reflected in their distribution in language use, as shown in corpus data. The dissertation investigates these assumptions by using various methods in a triangulating fashion. The first contribution of the dissertation is an ERP (event-related brain potentials) experiment that investigates the ERP response to grammatical function reanalysis, i.e., a revision of a tentative grammatical function assignment, during on-line comprehension of transitive sentences. Grammatical function reanalysis engenders a response that correlates with the (re-)assignment of thematic roles to the NP arguments. This suggests that the comprehension of grammatical functions involves assigning role-semantic functions to the NPs. The second contribution is a corpus study that investigates the distribution of prominence-based, verb-semantic and morphosyntactic features in transitive sentences in written discourse. The study finds that overt morphosyntactic information about grammatical functions is used more frequently when the grammatical functions cannot be determined on the basis of word order or animacy. This suggests that writers are inclined to accommodate the understanding of their recipients by more often providing formal markers of grammatical functions in potentially ambiguous sentences. The study also finds that prominence features and their interactions with verb-semantic features are systematically distributed across grammatical functions and therefore can predict these functions with a high degree of confidence. The third contribution consists of three computational models of incremental grammatical function assignment. These models are based upon the distribution of argument interpretation cues in written discourse. They predict processing difficulties during grammatical function assignment in terms of on-line change in the expectation of different grammatical function assignments over the presentation of sentence constituents. The most prominent model predictions are qualitatively consistent with reading times in a self-paced reading experiment of Swedish transitive sentences. These findings indicate that grammatical function assignment draws upon statistical regularities in the distribution of morphosyntactic and prominence-based information in language use. Processing difficulties in the comprehension of Swedish transitive sentences can therefore be predicted on the basis of corpus distributions.

We consider a statistical theory as being invariant when the results of two statisticians' independent data analyses, based upon the same statistical theory and using effectively the same statistical ingredients, are the same. We discuss three aspects of invariant statistical theories. Both model checking and checking for prior-data conflict are assessments of single null hypothesis without any specific alternative hypothesis. Hence, we conduct these assessments using a measure of surprise based on a discrepancy statistic. For the discrete case, it is natural to use the probability of obtaining a data point that is less probable than the observed data. For the continuous case, the natural analog of this is not invariant under equivalent choices of discrepancies. A new method is developed to obtain an invariant assessment. This approach also allows several discrepancies to be combined into one discrepancy via a single P-value. Second, Bayesians developed many noninformative priors that are supposed to contain no information concerning the true parameter value. Any of these are data dependent or improper which can lead to a variety of difficulties. Gelman (2006) introduced the notion of the weak informativity as a comprimise between informative and noninformative priors without a precise definition. We give a precise definition of weak informativity using a measure of prior-data conflict that assesses whether or not a prior places its mass around the parameter values having relatively high likelihood. In particular, we say a prior Pi_2 is weakly informative relative to another prior Pi_1 whenever Pi_2 leads to fewer prior-data conflicts a priori than Pi_1. This leads to a precise quantitative measure of how much less informative a weakly informative prior is. In Bayesian data analysis, highest posterior density inference is a commonly used method. This approach is not invariant to the choice of dominating measure or reparametrizations. We explore properties of relative surprise inferences suggested by Evans (1997). Relative surprise inferences which compare the belief changes from a priori to a posteriori are invariant under reparametrizations. We mainly focus on the connection of relative surprise inferences to classical Bayesian decision theory as well as important optimalities.