DEPENDENCE OF MODEL SELECTION CRITERIA AND FISHER INFORMATION MATRIX ON SAMPLE SIZE

When conducting model averaging for multiple groundwater models, a number of model selection criteria have been widely used for calculating model probabilities (or averaging weights). The criteria include the Akaike’s Information Criterion (AIC), small-sample correction of AIC (AICc), Bayesian Information Criterion (BIC), and Kashyap’s Information Criterion (KIC). This paper presents a numerical investigation on asymptotic features of the criteria based on a synthetic example of alternative geostatistical models. Adopting the results of Markov Chain Monte Carlo simulations as reference we confirm that KIC reduces asymptotically to BIC and provides consistently more reliable indications of model quality than does BIC for samples of all sizes. Practical considerations often cause analysts to replace the observed Fisher information matrix entering into KIC with its expected value. Our results show that this causes the performance of KIC to deteriorate with diminishing sample size. These results are equally valid for one and multiple realizations of uncertain data entering into our analysis. Bayesian theory indicates that, in the case of statistically independent identically distributed data, posterior model probabilities become asymptotically insensitive to prior probabilities as sample size increases. We do not find this to be the case when working with samples taken from an autocorrelated random field.