In factor analysis, eigenvalues are not a single, fixed value, but rather a set of values, one for each potential factor. Each eigenvalue represents the total amount of variance that can be explained by a given factor. These values are crucial for understanding the underlying structure of your data and for deciding how many factors to retain for further analysis.
Understanding Eigenvalues in Factor Analysis
Eigenvalues emerge from the analysis of a correlation or covariance matrix of your observed variables. They are a measure of the variance captured by each extracted factor. Essentially, a larger eigenvalue indicates that a factor accounts for more variance in the original variables.
What Do Eigenvalues Signify?
- Variance Explained: An eigenvalue quantifies the proportion of total variance explained by its corresponding factor. For instance, in a correlation matrix where variables are standardized (mean=0, standard deviation=1), the total variance is equal to the number of variables. If you have 10 variables, the total variance is 10. An eigenvalue of 2 for a factor means that factor explains as much variance as two of the original variables.
- Factor Strength: Factors with higher eigenvalues are considered more significant as they explain a greater amount of the common variance among the variables.
- Positive Values are Key: While eigenvalues can theoretically be positive or negative, in practical applications for explaining variance, they are always positive. An eigenvalue greater than zero is a good sign, indicating that the corresponding factor explains some amount of variance.
Role of Eigenvalues in Determining the Number of Factors
One of the primary uses of eigenvalues in factor analysis is to help determine the optimal number of factors to retain for your model. Retaining too many factors can lead to an overfitted model, while too few can result in a loss of important information. Here are common criteria that utilize eigenvalues:
-
Kaiser's Criterion (Eigenvalue Greater Than One)
- This is one of the most widely used methods. It suggests retaining all factors with an eigenvalue greater than 1.0.
- The rationale is that a factor must explain at least as much variance as a single original variable to be considered significant. If an eigenvalue is less than 1, it means the factor explains less variance than a single variable, making it less useful.
- Example: If your factor analysis yields eigenvalues of 3.5, 2.1, 0.8, 0.5, and 0.2, you would typically retain the first two factors based on Kaiser's criterion.
-
Scree Plot
- A scree plot is a graphical representation of the eigenvalues in descending order. The eigenvalues are plotted on the y-axis against the factor number on the x-axis.
- The "elbow" or "inflection point" of the plot, where the slope of the line changes dramatically and flattens out, indicates the optimal number of factors to retain. Factors before the elbow are typically retained.
- This method is more subjective but can be very insightful when used in conjunction with other criteria.
-
Percentage of Variance Explained
- Analysts often look for factors that collectively explain a substantial cumulative percentage of the total variance (e.g., 60-70% or more).
- This approach focuses on the practical significance of the factors in terms of how much information they capture from the original dataset.
Interpreting Eigenvalues: A Summary
The following table provides a concise overview of eigenvalue interpretation in factor analysis:
| Eigenvalue Range | Interpretation | Implication |
|---|---|---|
| > 1.0 | Factor explains more variance than a single original variable. | Potentially significant factor; often retained based on Kaiser's criterion. |
| < 1.0 | Factor explains less variance than a single original variable. | Less significant factor; typically discarded unless other methods (e.g., scree plot) suggest otherwise. |
| Large Value | Factor captures a substantial amount of common variance; strong underlying dimension. | Indicates a powerful and important factor; contributes heavily to explaining relationships between variables. |
| Small Value | Factor captures little common variance; weak underlying dimension. | Suggests a less important factor; may be considered noise or too specific to be generally useful. |
| Positive | Always the case in practice when explaining variance. | Confirms the factor contributes to explaining variability in the data. |
Practical Insights and Considerations
- Context Matters: The "best" number of factors also depends on the theoretical framework and the research question. Statistical criteria should always be balanced with substantive knowledge.
- Multiple Methods: It's common practice to use a combination of Kaiser's criterion, scree plots, and interpretability of factor loadings to make the final decision.
- Software Output: Statistical software (like R, SPSS, SAS) will typically provide a table of eigenvalues, percentage of variance explained, and cumulative percentage, which are essential for this decision-making process.
By carefully evaluating eigenvalues, researchers can effectively reduce the dimensionality of their data and identify meaningful latent constructs that underpin observed variables.
