p-value

n. P-value, a very common concept in empirical research, refers to the statistical probability that an observed trend in the data (e.g., a relationship or difference) has occurred by chance alone. In a technical sense, the p-value is a statistical estimate of the likelihood, assuming a representative sample of data from the population at large, that a particular effect observed in the sample data resulted from chance sampling error. P-values can range from 0 (0%) to 1 (100%). By convention, researchers generally adhere to the guideline that an effect is statistically significant if the p-value is less than or equal to 5% (p = .05). Many researchers have come to question this convention for a variety of reasons. Since p-values fluctuate, depending on sample size, with a large enough sample, even a very minor variation or relationship in a data set will reach a p-value of less than .05. This has led to a blurring of the line between statistical significance and nonsignificance; sometimes research reports include discussion of “marginally significant” findings for effects with p-values ranging from .05 to .15.

P-values are associated with type I error, the likelihood of falsely concluding that a trend exists in the population, and type II error, the likelihood of missing a trend in the data that exists in the population. Researchers can choose to be more conservative about type I errors by choosing a more stringent criterion level for decisions about the significance of p-values, also known as the alpha level. While the general convention is to set the alpha level at 5% (p = .05), sometimes researchers will set an alpha level at a more stringent level such as 1% (p = .01) or even .1% (p < .001) if the cost of a type I error is especially high. Statistical power is the likelihood of avoiding a type II error, or missing a signal that is present in the data. Because p-values are more sensitive with larger samples, the simplest way to boost the power of a study is to increase the sample size. – MWP

▶ See also SIGNIFICANCE TEST and TYPE I and TYPE II ERROR