web.archive.org

analysis of variance: Definition and Much More from Answers.com

️Wed Jul 01 2015

Analysis of variance (ANOVA) is a statistical technique that can be used to evaluate whether there are differences between the average value, or mean, across several population groups. With this model, the response variable is continuous in nature, whereas the predictor variables are categorical. For example, in a clinical trial of hypertensive patients, ANOVA methods could be used to compare the effectiveness of three different drugs in lowering blood pressure. Alternatively, ANOVA could be used to determine whether infant birth weight is significantly different among mothers who smoked during pregnancy relative to those who did not. In the simplest case, where two population means are being compared, ANOVA is equivalent to the independent two-sample t-test.

One-way ANOVA evaluates the effect of a single factor on a single response variable. For example, a clinician may be interested in determining whether there are differences in the age distribution of patients enrolled in two different study groups. Using ANOVA to make this comparison requires that several assumptions be satisfied. Specifically, the patients must be selected randomly from each of the population groups, a value for the response variable is recorded for each sampled patient, the distribution of the response variable is normally distributed in each population, and the variance of the response variable is the same in each population. In the above example, age would represent the response variable, while the treatment group represents the independent variable, or factor, of interest.

As indicated through its designation, ANOVA compares means by using estimates of variance. Specifically, the sampled observations can be described in terms of the variation of the individual values around their group means, and of the variation of the group means around the overall mean. These measures are frequently referred to as sources of "within-groups" and "between-groups" variability, respectively. If the variability within the k different populations is small relative to the variability between the group means, this suggests that the population means are different. This is formally tested using a test of significance based on the F distribution, which tests the null hypothesis (H0) that the means of the k groups are equal:

H₀ = μ₁ = μ₂ = μ₃ = …. μ_k

An F-test is constructed by taking the ratio of the "between-groups" variation to the "within-groups" variation. If n represents the total number of sampled observations, this ratio has an F distribution with k-1 and n-k degrees in the numerator and denominator, respectively. Under the null hypothesis, the "within-groups" and "between-groups" variance both estimate the same underlying population variance and the F ratio is close to one. If the between-groups variance is much larger than the within-groups, the F ratio becomes large and the associated p-value becomes small. This leads to rejection of the null hypothesis, thereby concluding that the means of the groups are not all equal. When interpreting the results from the ANOVA procedures it is helpful to comment on the strength of the observed association, as significant differences may result simply from having a very large number of samples.

Multi-way analysis of variance (MANOVA) is an extension of the one-way model that allows for the inclusion of additional independent nominal variables. In some analyses, researchers may wish to adjust for group differences for a variable that is continuous in nature. For example, in the example cited above, when evaluating the effectiveness of hypertensive agents administered to three groups, we may wish to control for group differences in the age of the patients. The addition of a continuous variable to an existing ANOVA model is referred to as analysis of covariance (ANCOVA).

In public health, agriculture, engineering, and other disciplines, there are numerous study designs whereby ANOVA procedures can be used to describe collected data. Subtle differences in these study designs require different analytic strategies. For example, selecting an appropriate ANOVA model is dependent on whether repeated measurements were taken on the same patient, whether the same number of samples were taken in each population, and whether the independent variables are considered as fixed or random variables. A description of these caveats is beyond the scope of this encyclopedia, and the reader is referred to the bibliography for more comprehensive coverage of this material. However, several of the more commonly used ANOVA models include the randomized block, the split-plot, and factorial designs.

Bibliography

Cochran, W. G., and Cox, G. M. (1957). Experimental Design, 2nd edition. New York: Wiley.

Cox, D. R. (1966). Planning of Experiments. New York: Wiley.

Kleinbaum, D. G.; Kupper, L. L.; and Muller, K. E. (1987). Applied Regression Analysis and Other Multivariate Methods, 2nd edition. Boston: PWS-Kent Publishing Company.

Snedecor, G. W., and Cochran, W. G. (1989). Statistical Methods, 8th edition. Ames, IA: Iowa State University Press.

— PAUL J. VILLENEUVE

Wikipedia: analysis of variance

In statistics, analysis of variance (ANOVA) is a collection of statistical models, and their associated procedures, in which the observed variance is partitioned into components due to different explanatory variables. The initial techniques of the analysis of variance were developed by the statistician and geneticist R. A. Fisher in the 1920s and 1930s, and is sometimes known as Fisher's ANOVA or Fisher's analysis of variance, due to the use of Fisher's F-distribution as part of the test of statistical significance.

Overview

There are three conceptual classes of such models:

Fixed-effects model assumes that the data come from normal populations which may differ only in their means. (Model 1)
Random-effects models assume that the data describe a hierarchy of different populations whose differences are constrained by the hierarchy. (Model 2)
Mixed effects models describe situations where both fixed and random effects are present. (Model 3)

In practice, there are several types of ANOVA depending on the number of treatments and the way they are applied to the subjects in the experiment:

One-way ANOVA is used to test for differences among three or more independent groups.
One-way ANOVA for repeated measures is used when the subjects are subjected to repeated measures; this means that the same subjects are used for each treatment. Note that this method can be subject to carryover effects.
Factorial ANOVA is used when the experimenter wants to study the effects of two or more treatment variables. The most commonly used type of factorial ANOVA is the 2×2 (read: two by two) design, where there are two independent variables and each variable has two levels or distinct values. Factorial ANOVA can also be multi-level such as 3×3, etc. or higher order such as 2×2×2, etc. but analyses with higher numbers of factors are rarely done because the calculations are lengthy and the results are hard to interpret.
When one wishes to test two or more independent groups subjecting the subjects to repeated measures, one may perform a factorial mixed-design ANOVA, in which one factor is independent and the other is repeated measures. This is a type of mixed effect model.
Multivariate analysis of variance (MANOVA) is used when there is more than one dependent variable.

Models

Fixed-effects model

The fixed-effects model of analysis of variance applies to situations in which the experimenter has subjected the experimental material to several treatments, each of which affects only the mean of the underlying normal distribution of the "response variable".

Random-effects model

Random effects models are used when the treatments are not fixed. This occurs when the various treatments (also known as factor levels) are sampled from a larger population. Because the treatments themselves are random variables, some assumptions and the method of contrasting the treatments differ from Anova model 1.

Most random-effects or mixed-effects models are not concerned with making inferences concerning the particular sampled factors. For example, consider a large manufacturing plant in which many machines produce the same product. The statistician studying this plant would have very little interest in comparing the three particular machines to each other. Rather, inferences that can be made for all machines are of interest, such as their variability and the overall mean.

Assumptions

Independence of cases - this is a requirement of the design.
Normality - the distributions in each of the groups are normal (use the Kolmogorov-Smirnov and Shapiro-Wilk normality tests to test it). Some say that the F-test is extremely non-robust to deviations from normality (Lindman, 1974) while others claim otherwise (Ferguson & Takane 2005: 261-2). The Kruskal-Wallis test is a nonparametric alternative which does not rely on an assumption of normality.
Homogeneity of variances - the variance of data in groups should be the same (use Levene's test for homogeneity of variances).

These together form the common assumption that the error residuals are independently, identically, and normally distributed for fixed effects models, or:

Failed to parse (unknown function\thicksim): \varepsilon \thicksim N(0, \sigma^2).

Anova 2 and 3 have more complex assumptions about the expected value and variance of the residuals since the factors themselves may be drawn from a population.

Logic of ANOVA

Partitioning of the sum of squares

The fundamental technique is a partitioning of the total sum of squares into components related to the effects used in the model. For example, we show the model for a simplified ANOVA with one type of treatment at different levels.

$SS_{\hbox{Total}} = SS_{\hbox{Error}} + SS_{\hbox{Treatments}}\,\!$

The number of degrees of freedom (abbreviated df) can be partitioned in a similar way and specifies the chi-square distribution which describes the associated sums of squares.

$df_{\hbox{Total}} = df_{\hbox{Error}} + df_{\hbox{Treatments}}\,\!$

The F-test

The F-test is used for comparisons of the components of the total deviation. For example, in one-way, or single-factor Anova, statistical significance is tested for by comparing the F test statistic

$F^* = \frac{\mbox{MSTR}}{\mbox{MSE}}$

where:

$\mbox{MSTR} = \frac{\mbox{SSTR}}{I-1}$ , I = number of treatments

and

$\mbox{MSE} = \frac{\mbox{SSE}}{n_T-I}$ , n_T = total number of cases

to the F-distribution with I-1,n_T degrees of freedom. Using the F-distribution is a natural candidate because the test statistic is the quotient of two mean sums of squares which have a chi-square distribution.

ANOVA on ranks

As first suggested by Conover and Iman in 1981, in many cases when the data do not meet the assumptions of ANOVA, one can replace each original data value by its rank from 1 for the smallest to N for the largest, then run a standard ANOVA calculation on the rank-transformed data. "Where no equivalent nonparametric methods have yet been developed such as for the two-way design, rank transformation results in tests which are more robust to non-normality, and resistant to outliers and non-constant variance, than is ANOVA without the transformation." (Helsel & Hirsch, 2002, Page 177). However Seaman et al. (1994) noticed that the rank transformation of Conover and Iman (1981) is not appropriate for testing interactions among effects in a factorial design as it can cause an increase in Type I error (alpha error). Furthermore, if both main factors are significant there is little power to detect interactions

Conover, W. J., Iman, R. L. (1981). Rank transformations as a bridge between parametric and nonparametric statistics. American Statistician, 35, 124-129. [1] [2]

Helsel, D.R. and R. M. Hirsch, 2002. Statistical Methods in Water Resources: Techniques of Water Resourses Investigations, Book 4, chapter A3. U.S. Geological Survey. 522 pages.[3]

Seaman, J. W., Walls, S. C., Wide, S.E. and Jaeger, R.G.(1994) Caveat emptor: rank transform methods and interactions. Trends Ecol. Evol. 9, 261-263.

Examples

Group A is given vodka, Group B is given gin, and Group C is given a placebo. All groups are then tested with a memory task. A one-way ANOVA can be used to assess the effect of the various treatments (that is, the vodka, gin, and placebo).

Group A is given vodka and tested on a memory task. The same group is allowed a rest period of five days and then the experiment is repeated with gin. The procedure is repeated using a placebo. A one-way ANOVA with repeated measures can be used to assess the effect of the vodka versus the impact of the placebo.

In an experiment testing the effects of expectations, subjects are randomly assigned to four groups:

expect vodka-receive vodka
expect vodka-receive placebo
expect placebo-receive vodka
expect placebo-receive placebo (the last group is used as the control group)

Each group is then tested on a memory task. The advantage of this design is that multiple variables can be tested at the same time instead of running two different experiments. Also, the experiment can determine whether one variable affects the other variable (known as interaction effects). A factorial ANOVA (2×2) can be used to assess the effect of expecting vodka or the placebo and the actual reception of either.

References

Ferguson, George A., Takane, Yoshio. (2005). "Statistical Analysis in Psychology and Education", Sixth Edition. Montréal, Quebec: McGraw-Hill Ryerson Limited.
King, Bruce M., Minium, Edward W. (2003). Statistical Reasoning in Psychology and Education, Fourth Edition. Hoboken, New Jersey: John Wiley & Sons, Inc. ISBN 0-471-21187-7
Lindman, H. R. (1974). Analysis of variance in complex experimental designs. San Francisco: W. H. Freeman & Co.

Statistics
Descriptive statistics	Mean (Arithmetic, Geometric) - Median - Mode - Power - Variance - Standard deviation
Inferential statistics	Hypothesis testing - Significance - Null hypothesis/Alternate hypothesis - Error - Z-test - Student's t-test - Maximum likelihood - Standard score/Z score - P-value - Analysis of variance
Survival analysis	Survival function - Kaplan-Meier - Logrank test - Failure rate - Proportional hazards models
Probability distributions	Normal (bell curve) - Poisson - Bernoulli
Correlation	Confounding variable - Pearson product-moment correlation coefficient - Rank correlation (Spearman's rank correlation coefficient, Kendall tau rank correlation coefficient)
Regression analysis	Linear regression - Nonlinear regression - Logistic regression

This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)