significance test

n. A significance test is a statistical analysis of whether or not a particular trend or effect exists in the population at large. The conventional test of significance is a determination of the probability that a trend in the data has been observed by chance, specifically whether that probability is 5% or less. Traditionally, significance testing has been a dichotomous yes or no decision. Many researchers have begun to question conventional significance testing, and many have adopted less rigid approaches to statistical significance. An emerging movement in research is to examine closely the magnitude of trends in the data, which are commonly referred to as effect sizes. While effect sizes and significance tests are not necessarily at odds with each other, many argue that effect sizes are more meaningful than traditional dichotomous significance tests. The American Psychological Association has adopted a stance in favor of reporting and discussing effect size estimates alongside significance tests, and many editors of scholarly journals have begun to require that research reports include effect size estimates in addition to traditional significance tests. – MWP

▶ See also P-VALUE