Title | The P_rep statistic as a measure of confidence in model fitting |
Publication Type | Journal Article |
Year of Publication | 2008 |
Authors | F Ashby, G., & O'Brien J. B. |
Journal | Psychonomic Bulletin & Review |
Volume | 15 |
Issue | 1 |
Pagination | 16-27 |
Date Published | 2008 Feb |
ISSN | 1069-9384 |
Keywords | Analysis of Variance, Bayes Theorem, Confidence Intervals, Data Collection, Decision Support Techniques, Discrimination Learning, Humans, Linear Models, Mathematical Computing, Models, Statistical, Models, Theoretical, Pattern Recognition, Visual, Psychomotor Performance, Reproducibility of Results |
Abstract | In traditional statistical methodology (e.g., the ANOVA), confidence in the observed results is often assessed by computing thep value or the power of the test. In most cases, adding more participants to a study will improve these measures more than will increasing the amount of data collected from each participant. Thus, traditional statistical methods are biased in favor of experiments with large numbers of participants. This article proposes a method for computing confidence in the results of experiments in which data are collected from a few participants over many trials. In such experiments, it is common to fit a series of mathematical models to the resulting data and to conclude that the best-fitting model is superior. The probability of replicating this result (i.e., Prep) is derived for any two nested models. Simulations and empirical applications of this new statistic confirm its utility in studies in which data are collected from a few participants over many trials. |
Alternate Journal | Psychon Bull Rev |
PubMed ID | 18605475 |
Grant List | MH3760-2 / MH / NIMH NIH HHS / United States |