These are the pages relevant to the topic of factor analysis:  Regression.  Introduction to principal components analysis.  Orthogonal factor rotation.  Oblique rotation.  Full factor extraction.  This page, Principle Axes extraction.  Cluster analysis.

Up to now we have analysed a correlation matrix with 1.0 in the main diagonal, being the correlation of the variable with itself.  This is the key feature of Principle Components Analysis -- PCA starts with the correlation matrix main diagonal comprising all 1.0's.

Introduction to Principle Axes Factor Analysis

Principle Axes analysis starts with a correlation matrix where the main diagonal comprises the estimated communality, h², of each variable, being the proportion of variance that the variable has in common with all the other variables in the correlation matrix.  The main diagonal entry for a variable at the start of Principle Axes (PA) Analysis is given by setting the variable as the DV in a regression analysis with all the other variables as IVs, so h² = R².  Thereafter, extraction and rotation are as for PCA.

Just so you are aware, hundreds of books, tens of thousands of learned academic articles, and many millions of words have been written explaining PA and its differences from PCA.  The difference takes just six words -- The main diagonal in PA is R² (§1).  But it seems that millions of words are needed to explain, Why?

(§1)  In early Factor analysis, MLA was not used to provide the R² estimates of initial communality.  Instead, these estimates were set equal to the largest correlation shown by the variable with any other in the matrix and then squared. 

Why?

Undaunted, we'll add a few words to the many millions.

Both PCA and PA construct hypothetical variables, linear composites of the variables of the correlation matrix, to account for a maximum of the variance seen in that matrix.  By convention, PCA gives us Components while PA gives us Factors.  PCA uses the variables "as given", their inter-correlations "as found", and provides Components which are hypothetical accounts of their covariances "as seen".  PA provides a somewhat different hypothetical account of the inter-correlations and covariances after first adjusting the correlation matrix for the "participation" or "contribution" of the variables, and so provides Factors which are hypothetical accounts of how the variables participate in the resulting factor structure.  The degree of participation of a variable is the degree to which it relates to all the other variables in the matrix as given by its R².

PA is usually considered a desirable approach by recognising that variables differ in the degree to which they have anything to contribute.  This may be related to the degree to which they have been accurately or adequately measured, known as their reliability, and related to the degree to which they "belong" with the other variables.  Variables which participate or contribute less have lower loadings on the resulting factors and hence have lower impact when the factor is interpreted.

Whether PCA or PA is appropriate for your correlation matrix is usually given by the theoretical context of your study.  As a very broad brush, PCA finds application in the "harder" sciences such as engineering, while PA is applied in the "softer" arts such as psychology, economics, and information technology.

The difference in practice -- Communalities after extraction

We can begin by examining the communalities placed in the correlation matrix at the start of PA factor extraction.  Table 1 shows these in column 1.  Column 2 shows the communalities reported by PCA after extraction of 4 components, being the variance accounted for by the 4 components, while column 3 shows the communalities reported by PA after extraction of 4 factors, being the variance accounted for by the 4 factors.

Table 1.  Communalities before PA, of 4 Components, and of 4 Factors

Starting with column 1 R², we note that we have seen the communality of the variable "Interest in generating income through online activities" already, in the Regression page where it was .41.  We may note that the starting communality, R² for "Right to decide how to use personal income", is a very modest .20, the lowest in the table, while for "Ability to evaluate whether online information is trustworthy" the starting communality is .72, the highest in the table.

Looking across to column 2, after four factors of PCA extraction (see Full factor extraction) we find that the variable with the highest starting communality has retained its position, given as .75, representing a very modest increase of 3%.  Similarly, the variable with the lowest starting communality is still the lowest, but at .54 it shows a substantial 34% increase in its contribution towards accounting for the variance in the correlation matrix (§2).  This isn't the largest increase, however.  The variable "Experience with borrowing from banks or formal financial institutions" started with R² = .30 and increased to h² = .67 after four factors of PCA extraction, an increase of 37% in variance accounted for.

Column 3, h² after four factors of PA extraction, provides considerable interest.  The first point is that, in general, PA h² has increased from the starting R² after four factors of PA extraction.  In extracting components or factors, both PCA and PA are iterative.  They start with a correlation matrix, and extract a component or factor.  Amongst other matters, they extract the variance accounted for by that component or factor from the correlation matrix, and repeat the process, extracting the next component or factor.  As we know, this next component or factor thus accounts for variance not accounted for by the previous component or factor.  The main diagonal elements of the correlation matrix, as well as the off-diagonal inter-correlations, are adjusted to reflect the variance "left behind".  Hence, the communality of a variable can increase from the initial PA estimate of R² to a higher h² value after successive loadings on the extracted factors.

The second point is that, in particular for some variables, the PA h² has decreased from the starting R² .  We can see that the variable with the lowest starting communality, .20, has an almost negligible h² = .11 after four factors of PA extraction.  While PCA found a respectable contribution of "Right to decide how to use personal income" in accounting for the variance in the correlation matrix, h² = .54, PA found this was almost negligible, .11.

(§2) Correlating column 1 with the difference between column 1 and column 2 (the change in communality from initial PA R² to PCA 4 Components h²) shows r = –.85.  There is a very strong negative relationship between the starting R² and its increase to h² after four factors of PCA extraction -- larger R² values increase to h² only modestly, while smaller values increase substantially.

The difference in practice -- Variance accounted for

Next, we examine the change in variance accounted for as PCA components, or PA factors, are extracted.  The initial Eigenvalues of PCA components and PA factors are identical;  the differences appear in the squared loadings of the extracted components or factors.  Table 2 shows this for the four components and factors of PCA and PA extraction in our example data set, and the total as might be found after 12 and 13 components and factors.

Table 2.  Total variance accounted for

After 4 components or factors, we see that the cumulative % of variance accounted for by PCA in the correlation matrix with 1.0 main diagonal is 65.1%, while it is 55.7% for PA in the correlation matrix with R² main diagonal. 

More importantly, we also see that 13 PCA components will account for 98.6% (14 components would, technically, account for 100% of the variance of 14 variables), while 13 PA factors will account for no more than 68.1%.  This is an excellent summary of the difference between PCA and PA -- PCA will account for up to 100% of the correlation matrix if required, while PA will only ever account for a more limited percentage, somewhere in the region of the largest R².  Accounting for up to 100% of a pattern of inter-correlations might be reasonable and appropriate in the "harder" sciences dealing with engineered artefacts, but this is unlikely to be so in the "softer" arts dealing with human behaviour.

The difference in practice -- Loadings

Next, we examine the loadings on the four PCA components or PA factors.  As previously noted, considering the loadings "as extracted" is usually unrewarding, so we examine them following a Varimax orthogonal rotation, as shown in Table 3, where the loadings are edited (blank if less than .4) (§3) and sorted by size on the component or factor.

(§3) It is likely that your advisor, supervisor, or reviewer may prefer that all loadings are shown in a table and not selectively blanked out, with the substantial loadings shown in an eye-catching bold font or perhaps larger point size. They will not want to have any results hiding behind a curtain.

Table 3.  Orthogonal rotation loadings, edited and sorted

Both PCA and PA show, broadly, the same loadings on the same factors.  That the PCA loadings are somewhat larger than the PA loadings is not particularly important and is entirely expected.

There are two differences, however, between the two solutions.

First, the PA solution provides a "cleaner" Factor 3 compared with Component 3, due to the generally smaller loadings which drop two variables off Component 3 after editing, and makes for a more straightforward interpretation of Factor 1 because the two dropped variables, major contributors to Factor 1, no longer show substantial loadings elsewhere.

Second, and a major difference, is that "Right to decide how to use personal income" no longer has a substantial loading on any PA Factor, whereas it loaded substantially on PCA Component 4.  (Of course, the variable does have loading on PA Factor 4, it is in fact .26, but it does not reach the "substantial" threshold.  The point is that this is quite a change from its PCA loading of .73 on Component 4.)

Marker variables

PA factor analysis is often deployed in the softer arts to find distinct clusters of variables, and it is common for some selected demographic variables to be added to the analysis to see if they can "mark" a cluster or factor with "age", "education", "income", or similar. 

We add the variables of "age", "education", and "job" to our example correlation matrix.  Education is coded on a four-point scale, 0=none, 1=primary, 2=secondary, and 3=tertiary.  Job is coded on a two-point scale, 1=formally employed, 0=not formally employed. 

Note that there are now five factors with Eigenvalues above 1.  Because the intent is to add markers to the earlier factor solution of four factors, we extract and orthogonally rotate four factors.  The result is shown in Table 4.

Table 4.  PA extraction and orthogonal rotation with marker variables added, loadings edited and sorted

The comparison of Table 4 with the PA Factors of Table 3 is interesting.

Factors 1, 2, 3, and 4 are essentially unchanged.

The markers of "Job" and "Education" are placed on Factor 3, and so are associated with the two variables loading substantially on that factor, "Availability of trusted people or groups who can help find customers" and "Support from peers for online income-generating activities".  The marker "Age" does not appear on any factor at the desired criterion level of 0.4.

Concluding comments

Factor analysis has quite a history and quite a number of controversies.  There are Factor Analytic warriors who police the Factor Analytic publications space;  some are evangelists, some are pragmatists, and some are, well, other things.  You may find one is your assigned reviewer or referee.  I can only offer Shakespeare's line in Hamlet, "There is nothing either good or bad, but thinking makes it so".  The replacement of 1.0 with another value between 0 and 1 by something like R² or the largest r² is usually policed as though it needs exhaustive justification, although curiously the use of the default 1.0 usually attracts little attention.  Over to you to provide appropriate references for your decisions.

If the research question is primarily to identify distinct clusters of variables, and to then ask how those clusters are related, an alternative technique is Cluster Analysis.  This page analyses the current example data, and gives some insights which factor analysis does not provide.  There are many fewer Cluster Analysis warriors in case that is a helpful hint....

©2025 Lester Gilbert