Saturday, November 21, 2026

Classical Test Theory Analysis of the 16 Factor Personality Measure

In my last post, I talked through how to clean the dataset of careless responders, responders with too much missing data, and people older than time itself. Today, I’ll go through a classical test theory approach to psychometric analysis. If you’d like to go through the cleaning process with the original dataset, I recommended going through the last post. If you want, you can even change the cutoffs I used for various flags and see how it changes the dataset and the results going forward.

If, on the other hand, you want to start with the exact dataset I’m using without going through the full cleaning process on your own, you can download my clean copy here. Just remember that inspecting and cleaning data is always part of the process when analyzing a dataset.

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.2.1     ✔ readr     2.2.0
## ✔ forcats   1.0.1     ✔ stringr   1.6.0
## ✔ ggplot2   4.0.3     ✔ tibble    3.3.1
## ✔ lubridate 1.9.5     ✔ tidyr     1.3.2
## ✔ purrr     1.2.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
PF16 <- read_csv("Cleaned PF16 Data.csv")
## Rows: 46427 Columns: 171
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr   (2): country, age_group
## dbl (169): A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, B1, B2, B3, B4, B5, B6, ...
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

Classical test theory approaches are very much “what you see is what you get.” What I mean by that is, the range of abilities (or more specifically to a measure of traits, the range of how much of each trait people possess) is defined entirely by the sample we selected, and how well we can measure those traits is defined entirely by the items we used (though we recognize there are other, potentially better items). We can make assumptions that our sample is well representative of the full range of presence of these traits, but subjecting a different sample to this measure will likely produce very different results. This drawback is one of the reasons psychometricians gravitate to item response theory models, like Rasch, but the benefit of using CTT is that there are very few statistical assumptions we need to meet to use these analyses. And that sometimes makes CTT a better approach; it really depends on context.

I’ll talk more about the assumptions of IRT in my next post. But the key assumption in CTT is that a person’s observed score is the sum of their true score (their underlying ability or how much of the trait they actually posses - what we’re trying to measure) and error (caused by a variety of factors that get in the way of obtaining that true score, like distractions in the testing environment, a poorly written item, lack of attention by the respondent, and so on). We want that error to be 0, but we know in actual practice, it probably isn’t. So we work to try to minimize that error: we use multiple items to measure the same trait, hoping that will help cancel out some the error, and we conduct analyses that help us measure that error and separate it from the true score. We also use those analyses to identify the items that increase measurement error, so that we can edit or remove them.

Now, in some cases IRT has influenced CTT. For instance, someone who takes a purely CTT approach to psychometrics would treat rating scales as continuous variables. In fact, CTT literature often states that rating scales with at least 5 points can be treated as continuous regardless. IRT would consider rating scales to be ordinal (that is, until analysis converts them). Parametric statistics, like mean, standard deviation, and Pearson’s correlation coefficient (stats that assume normal distributions of the variable) are built specifically for continuous variables, whereas ordinal variables would require non-parametric approaches, like median, interquartile range, and Spearman’s rank-order correlation coefficient, which don’t make any assumptions about distributions. In my case, I’ll sometimes treat raw ratings scales as continuous, if the parametric test I’m using is robust with data that doesn’t quite meet assumptions, and sometimes as ordinal.

First, I want to reverse code the negatively worded items. Then we can start generating descriptive statistics by item and scale. The number of negatively worded items varies by scale, so we’ll need to specify the range for each scale, then use that list to reverse code the correct variables. As I did in the last post, I’m using mutate across, but this time we want it to use all items listed in ‘reverse’. Subtracting each rating from 6 (number of options + 1) will reverse the scale, so now 1 = 5, 2 = 4, and so on. 3, our midpoint ‘neutral’ option will remain unchanged.

reverse <- list(paste0("A", 8:10), paste0("B", 9:13), paste0("C", 6:10),
                paste0("D", 7:10), paste0("E", 7:10), paste0("F", 6:10),
                paste0("G", 6:10), paste0("H", 7:10), paste0("I", 7:10),
                paste0("J", 8:10), paste0("K", 6:10), paste0("L", 8:10),
                paste0("M", 6:10), paste0("N", 8:10), paste0("O", 6:10),
                paste0("P", 8:10))

reverse <- unlist(reverse)

PF16R <- PF16 %>% 
  mutate(across(all_of(reverse), ~ 6 - .x))

I’d also like to have an index dataframe that contains the full name of each scale and how many items each scale contains. When I start generating statistics at the scale-level, I can add to this dataframe.

Scales <- tibble(Code = c(toupper(letters[1:16])),
                     Name = c("Warmth",
                              "Intellect",
                              "Emotional Stability",
                              "Assertiveness",
                              "Gregariousness",
                              "Dutifulness",
                              "Friendliness",
                              "Sensitivity",
                              "Distrust",
                              "Imagination",
                              "Reserve",
                              "Anxiety",
                              "Complexity",
                              "Introversion",
                              "Orderliness",
                              "Emotionality"),
                     Items = c(10, 13, rep(10, 14)))

I’ll start by generating means and standard deviations for each item. (This is an instance of treating the rating scales as continuous.) Since we want to aggregate, we’ll be using summarise instead of mutate, but we can use summarise in very similar ways as mutate: such as summarizing across a range of variables.

ByItem <- PF16R %>%
  summarise(across(A1:P10, list(
    Mean =  ~mean(., na.rm = TRUE),
    SD = ~sd(.x, na.rm = TRUE))
  )) %>%
  pivot_longer(everything(), names_to = c("Column", ".value"), names_sep = "_")

This first step of the code above creates a dataset with 1 row and 2 columns per item (163 * 2 = 326 columns), one with the item mean and the other with standard deviation. The default is to name each column as ColumnName_Function, so we’d have A1_Mean, A1_SD, A2_Mean, etc. Pivot longer moves the column names to become row names, and “.value”, combined with names_sep = “_” does two things: it removes _Mean and _SD from the columns (leaving 163 rows, 1 for each item), and instead creates 2 columns next to the list of items names, one for Mean and one for SD. We can then add additional summary data to this file.

One of the big metrics used in CTT analyses is p-value, typically the proportion of people responding to an item correctly or, if the measure is a checklist, the proportion of people endorsing that item. But this measure doesn’t have incorrect and correct answers, nor is it a checklist. We can still generate a p-value-like measure, by dividing the mean rating by the number of rating scale categories (5). Instead of measuring percentage correct, it measures degree of aggreement with the statment (on average). By itself, it isn’t extremely useful, but we can examine it alongside other analyses to flag items that may not fit as well on the measure.

ByItem <- ByItem %>%
  mutate(pVal = Mean/5)

Next, I’ll add counts selecting each rating scale option to the ByItem data. Since the ByItem data is formatted as a row for each item, I’ll create my counts of categories using pivot functions to put it in the same format. In fact, for the next few analyses, I’ll want a set that pivots my item response data into long form, so I’ll first create that set.

This makes joining the two sets together very easy. I’ve also counted the number of people who skipped each item, though I didn’t include them when I calculated proportions selecting each option.

Items <- PF16R %>%
  pivot_longer(
    cols = A1:P10,
    names_to = "Item",
    values_to = "Rating"
  ) %>%
  mutate(Code = substr(Item, 1, 1)) %>%
  left_join(Scales, by = "Code")

 CatCounts <- Items %>%
  count(Item, Rating) %>%
  complete(Item, Rating = 1:5, fill = list(n = 0)) %>%
  group_by(Item) %>%
  pivot_wider(names_from = Rating, values_from = n)

ByItem <- ByItem %>%
  left_join(CatCounts, by = c("Column" = "Item"))

ByItem <- ByItem %>%
  group_by(Column) %>%
  mutate(TotalValid = sum(across(`1`:`5`)),
         StronglyDis = `1`/TotalValid,
         Disagree = `2`/TotalValid,
         Neither = `3`/TotalValid,
         Agree = `4`/TotalValid,
         StronglyAgr = `5`/TotalValid) %>%
  ungroup()

I’d also like to look at how well each item correlates to total scale score on the remaining items. For knowledge tests and checklists, this would be a point biserial, the correlation between a binary variable (0/1, or incorrect/correct) and a continuous variable (score on remaining items). But for a rating scale measure, we’ll be using polyserial correlations, the correlation between an ordinal variable (1-5 on the rating scale) and a continuous variable (score on the remaining items). Since some respondents are missing an item from a scale, we’ll compute total scores as averages of their ratings. Once we have our scores, we can use the CTT package to compute polyserial correlations by item.

library(CTT)

Polyserials <- Items %>%
  group_by(ID, Code) %>%
  mutate(Total = sum(Rating),
         Count = sum(!is.na(Rating)),
         Score = (Total - Rating)/(Count - 1)) %>%
  group_by(Item) %>%
  summarise(disc = polyserial(Score, Rating, ml = FALSE))

ByItem <- ByItem %>%
  left_join(Polyserials, by = c("Column" = "Item"))

I called our polyserial variable “disc” for “discrimination”, a psychometric term for how well the item differentiates between people who are high in the trait being measured and those low in the trait.

Lastly, I want to compute our scale reliabilities, as measured by Cronbach’s alpha. This measures what is called internal consistency by determining how much variance is attributable to true score variance - that is, how much people naturally vary in the traits being measured - by separating the variance shared by items (covariance) from total variance, which would include both true scores and measurement error. Alpha ranges from 0 to 1, and equals the proportion of variance attributable to true scores; theoretically, an alpha of 1 means the scale has no measurement error. In practice, it typically means redundancy in items, multiple items that address the exact same concept within a scale. For example, if I had a bunch of items measuring how much you enjoy reading books as well as negatively worded items on how much you don’t enjoy reading books. The scale would probably have a high alpha, but it’s not giving you a lot of useful information, especially if book reading is only one aspect of the concept I’m measuring.

There are multiple R packages that compute alpha, but we'd need to reformat the data. However, the formula for alpha is pretty simple:


I can easily compute the required variances, add them to my Scales frame (which already includes number of items), and compute alpha for each scale. For individual item variance, I already have standard deviation, so I just need to square that value to get variance. Generating total scores is a bit more involved: I’ve retained respondents with up to 2 missing values, but I want a total score on the same scale as respondents without missing values, so I’ll create a corrected total score - the average score of the ratings they provided (sum divided by number of items completed), multiplied by the number of items on the full scale. Since I wanted to score my sample anyway, I created a new dataframe of individual scores by scale, then aggregated for the variance I need to generate my alphas.

ByItem <- ByItem %>%
  mutate(Variance = SD^2)

Scores <- Items %>%
  ungroup() %>%
  group_by(ID, Code) %>%
  summarise(Total = sum(Rating, na.rm = TRUE),
            NItems = sum(!is.na(Rating))) %>%
  ungroup() %>%
  left_join(Scales, by = "Code") %>%
  mutate(Score = Total/NItems,
         CorTotal = Score*Items) 
## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by ID and Code.
## ℹ Output is grouped by ID.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(ID, Code))` for per-operation grouping
##   (`?dplyr::dplyr_by`) instead.
ByScale <- Scores %>%
  group_by(Code) %>%
  summarise(Variance = var(CorTotal, na.rm = TRUE))

ByScale2 <- ByItem %>%
  ungroup() %>%
  mutate(Code = substr(Column, 1, 1)) %>%
  group_by(Code) %>%
  summarise(EItemVar = sum(Variance))

Scales <- Scales %>%
  left_join(ByScale, by = "Code") %>%
  left_join(ByScale2, by = "Code")

Scales <- Scales %>%
  mutate(Alpha = (Items/(Items-1)) * (1 - (EItemVar/Variance)))

Now let’s go through these item-level and scale-level indices and start flagging items that could be problematic.

I also have some information on the scales’ alphas from the validation study, where 501 respondents completed both the original 16PF and the IPIP version I’m using here. I can easily pull the table from the webpage, though it needs a bit of cleaning to be useful.

library(rvest)
## 
## Attaching package: 'rvest'
## The following object is masked from 'package:readr':
## 
##     guess_encoding
url <- "https://ipip.ori.org/new16PFTable.htm"
page <- read_html(url)
all_tables <- html_table(page)

comptable <- all_tables[[2]]

comptable <- comptable %>%
  rename(IPIPNames = X1,
         PF16Names = X2,
         IPIPItems = X3,
         PF16Items = X4,
         IPIPCor = X5,
         PF16Cor = X6,
         IPIPAlpha = X7,
         PF16Alpha = X8,
         Correlation = X9)

comptable <- comptable[c(5:20), ]

ComparisonAlpha <- comptable %>%
  mutate(IPIPAlpha = as.numeric(IPIPAlpha)) %>%
  select(IPIPNames, IPIPAlpha)

Scales <- Scales %>%
  left_join(ComparisonAlpha, by = c("Name" = "IPIPNames"))

library(ggthemes)

Scales %>%
  pivot_longer(
    cols = c(Alpha, IPIPAlpha), 
    names_to = "source", 
    values_to = "alpha"
  ) %>%
  ggplot(aes(x = Code, y = alpha, color = source)) +
  geom_point() +
  labs(y = "Alpha", color = "source") +
  theme_economist_white()

For 11 of the scales, this sample produced higher alphas than the original validation sample. But what is a good alpha? The context of the measure is important here. If it’s being used to make high stakes decisions, like whether someone obtains a license or certificate or gets diagnosed with a condition, reliability should be high, preferably in the 0.9 range. But for a lower stakes measure, like this one, we can be a bit more generous in what alphas are acceptable. I’ll consider 0.90 and greater to be excellent, 0.80 to 0.89 good, 0.70 to 0.79 acceptable, and less than 0.70 questionable.

Scales <- Scales %>%
  mutate(AlphaCat = cut(Alpha, breaks = c(0, 0.7, 0.8, 0.9, Inf),
                        labels = c("Questionable",
                                   "Acceptable",
                                   "Good",
                                   "Excellent"),
                        right = FALSE))

Scales %>%
  ggplot(aes(x = Code, y = Alpha, colour = AlphaCat)) +
  geom_point() +
  theme_economist_white()

Overall, our alphas are fine, with the one exception being scale H, sensitivity, which has an alpha of 0.67. Perhaps our item analysis will help us determine the issue with this scale.

Let’s start with the p-value-like measure. It’s measuring average degree of agreement with the item. Low agreement could just mean the trait is rare. But if most of the items are easy to endorse in a scale, and there are some exceptions, those items may not be measuring the trait well. Let’s look at the items with a “p-value” of less than 0.5.

ByItem %>%
  filter(pVal < 0.5) %>%
  select(Column, Mean, pVal, StronglyDis:StronglyAgr)
## # A tibble: 7 × 8
##   Column  Mean  pVal StronglyDis Disagree Neither  Agree StronglyAgr
##   <chr>  <dbl> <dbl>       <dbl>    <dbl>   <dbl>  <dbl>       <dbl>
## 1 F8      2.43 0.487       0.252    0.387   0.135 0.126       0.0992
## 2 H4      2.27 0.455       0.286    0.384   0.151 0.126       0.0528
## 3 I6      2.11 0.422       0.344    0.362   0.167 0.0940      0.0332
## 4 K10     2.44 0.489       0.178    0.453   0.168 0.150       0.0515
## 5 N8      2.35 0.471       0.170    0.492   0.195 0.103       0.0401
## 6 P9      2.32 0.465       0.202    0.479   0.151 0.129       0.0389
## 7 P10     2.50 0.499       0.166    0.404   0.236 0.157       0.0374

7 items showed low agreement, with average rating in the “disagree” range, and disagree was the most frequent rating. So let’s add our first flag to the item data.

ByItem <- ByItem %>%
  mutate(Flag1 = ifelse(pVal < 0.50, "Low P", ""))

Next is discrimination, how well the item differentiates between people high and low in the trait, as measured by polyserials. We want all of our polyserials to be positive - and we would expect that since we reverse-coded the negatively worded items - so let’s see if any are 0 or negative. Since we probably don’t have any that are exactly 0, let’s also include discrimination that would essentialy round down to 0.

ByItem %>%
  filter(disc <= 0.004) %>%
  select(Column, disc)
## # A tibble: 0 × 2
## # ℹ 2 variables: Column <chr>, disc <dbl>

None of the items are negative or 0, which is good. But that doesn’t mean they provide meaningful information. So let’s look at items with positive, but low discrimination. We’ll use a cutoff of 0.20, or around 10% of the variance in scores.

ByItem %>%
  filter(disc < 0.20) %>%
  select(Column, disc)
## # A tibble: 1 × 2
##   Column   disc
##   <chr>   <dbl>
## 1 H4     0.0523

Only one item has low discrimination: item 4 on the Sensitivity scale, “Don’t like action movies”. This scale mostly deals with enjoyment of the arts, such as movies, books, poetry, and dance. Apparently sensitive people aren’t supposed to like action movies? But since this item has minimal correlation with sensitivity, apparently you can still be a sensitive artist who loves Die Hard.

ByItem <- ByItem %>%
  mutate(Flag2 = ifelse(disc < .20, "Low Discrimination", ""))

Let’s also look at rating scale function. Respondents have 5 options to choose from for each statement: Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree. There’s disagreement in the survey literature of whether agreement scales should provide a neutral option or not, with strong arguments either way. But with psychometrics, we typically make these types of decisions empirically - are people using the neutral category and are they using it as intended? If so, we keep the category. If not, we should consider revising the rating scale.

Remember, we have 163 items, which gives us a lot of graphs to look at. But we can at least make it less overwhelming by creating a grid of each scale with all 10 (or 13, for scale B, intellect) items. I’d also like the items to appear in order, and since the Item in the data is alphanumeric, the 10+ item in the scale will appear after 1 instead of 9. So I’ll make a new numeric variable of item number. Then I’ll write a function to create the charts (one for each scale using the filter function, with a grid using facet_wrap of Item Number), and lastly, apply them to each scale. The map function allows me use a list of scales and apply them one-by-one to the plotting function. Note that the list is called scale_names (plural) and the function uses scale_name (singular).

Items <- Items %>%
  mutate(Item_Num = substr(Item, 2, 3),
         Item_Num = as.numeric(Item_Num))

histogram_scale <- function(scale_name) {
  Items %>%
    filter(Name == scale_name) %>%
    ggplot(aes(factor(Rating, levels = 1:5))) +
    geom_bar() +
    scale_x_discrete(drop = FALSE) +
    facet_wrap(vars(Item_Num)) +
    labs(title = scale_name, x = "Rating", y = "Count") +
    theme(plot.title = element_text(hjust = 0.5))
}

scale_names <- unique(Items$Name)
scale_histograms <- map(scale_names, histogram_scale) %>% set_names(scale_names)

Now I have a large list (scale_plots) containing the data for each plot, and I can view each plot by referencing it. For instance, to view scale B, Intellect:

scale_histograms$Intellect

Across all items and scales, all response options are being used, though in some cases strongly disagree and strongly agree aren’t heavily used. But the bigger question is whether the rating scale is being used as intended. That is, are people lower in the trait more likely to select Strongly Disagree, middle of the trait more likely to select Neither Agree nor Disagree, and so on? Our polyserial correlations tell us some of this, and I’ll use some of the same code I did to compute those correlations, but we can dig deeper into scale function by visualizing that relationship. Specifically, I’d like to chart average total score (excluding the current item) for each level of the rating scale. If the categories are functioning properly, we should see an increase in average score as we go up the rating scale. To create my groups of scale graphs, I’ll do something similar as I did above.

ScaleFunction <- Items %>%
  group_by(ID, Code) %>%
  mutate(Total = sum(Rating),
         Count = sum(!is.na(Rating)),
         Score = (Total - Rating)/(Count - 1)) %>%
  ungroup() %>%
  group_by(Item, Rating) %>%
  summarise(Code = first(Code),
            Name = first(Name),
            Item_Num = first(Item_Num),
            Score = mean(Score, na.rm = TRUE)) %>%
  filter(!is.na(Rating)) %>%
  ungroup()
## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by Item and Rating.
## ℹ Output is grouped by Item.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(Item, Rating))` for per-operation grouping
##   (`?dplyr::dplyr_by`) instead.
plot_scalefunction <- function(scale_name) {
  ScaleFunction %>%
    filter(Name == scale_name) %>%
    ggplot(aes(Rating, Score)) +
    geom_point() +
    geom_smooth(method = "lm") +
    facet_wrap(vars(Item_Num)) +
    labs(title = scale_name, x = "Rating", y = "Average Total Score") +
    theme(plot.title = element_text(hjust = 0.5))
}

scale_scalefunctions <- map(scale_names, plot_scalefunction) %>% set_names(scale_names)

scale_scalefunctions$Intellect
## `geom_smooth()` using formula = 'y ~ x'

The steepness of the slope is, of course, the polyserial correlation for that item, so we could probably learn a lot from examining our discrimination indices. But these figures show where a point on a rating scale falls out of line - perhaps equal to or even lower than the point below it. Going through the figures scale by scale, the following variables show signs of scale dysfunction: A10, B1, B4, B8, D6, D8, E7, E9, F3, H4, H9, H10, I5, J9, J10, K8, L8, M3, M6, M7, M8, M9, M10, N7.

Flag3 <- list("A10", "B1", "B4", "B8", "D6", "D8", "E7", "E9", "F3", "H4", "H9", "H10", "I5", "J9", "J10", "K8", "L8", "M3", "M6", "M7", "M8", "M9", "M10", "N7")

ByItem <- ByItem %>%
  mutate(Flag3 = ifelse(Column %in% Flag3, "Scale Dysfunction", ""))

Now it’s time to conduct our CFA. This isn’t necessarily required, especially if you’re conducting item and scale analysis on an established measure. It’s more necessary if you’re conducting the validation study of a measure. But let’s go ahead and conduct one to be thorough. I’ve blogged about using lavaan for CFA before, so check out that post for a bit more exposition. I just need to specify my model and feed it into the cfa function.

library(lavaan)
## This is lavaan 0.7-2
## lavaan is FREE software! Please report any bugs.
Factor16 <- '
  "Warmth" =~ A1 + A2 + A3 + A4 + A5 + A6 + A7 + A8 + A9 + A10
  "Intellect" =~ B1 + B2 + B3 + B4 + B5 + B6 + B7 + B8 + B9 + B10 + B11 + B12 + B13
  "EmotSta" =~ C1 + C2 + C3 + C4 + C5 + C6 + C7 + C8 + C9 + C10
  "Assert" =~ D1 + D2 + D3 + D4 + D5 + D6 + D7 + D8 + D9 + D10
  "Gregar" =~ E1 + E2 + E3 + E4 + E5 + E6 + E7 + E8 + E9 + E10
  "Duti" =~ F1 + F2 + F3 + F4 + F5 + F6 + F7 + F8 + F9 + F10
  "Friend" =~ G1 + G2 + G3 + G4 + G5 + G6 + G7 + G8 + G9 + G10
  "Sens" =~ H1 + H2 + H3 + H4 + H5 + H6 + H7 + H8 + H9 + H10
  "Dist" =~ I1 + I2 + I3 + I4 + I5 + I6 + I7 + I8 + I9 + I10
  "Imag" =~ J1 + J2 + J3 + J4 + J5 + J6 + J7 + J8 + J9 + J10
  "Reser" =~ K1 + K2 + K3 + K4 + K5 + K6 + K7 + K8 + K9 + K10
  "Anx" =~ L1 + L2 + L3 + L4 + L5 + L6 + L7 + L8 + L9 + L10
  "Complex" =~ M1 + M2 + M3 + M4 + M5 + M6 + M7 + M8 + M9 + M10
  "Intr" =~ N1 + N2 + N3 + N4 + N5 + N6 + N7 + N8 + N9 + N10
  "Order" =~ O1 + O2 + O3 + O4 + O5 + O6 + O7 + O8 + O9 + O10
  "Emot" =~ P1 + P2 + P3 + P4 + P5 + P6 + P7 + P8 + P9 + P10
'

CFA16 <- cfa(model = Factor16, data = PF16R)

summary(CFA16, fit.measures = TRUE)
## lavaan 0.7-2 ended normally after 229 iterations
## 
##   Estimator                                         ML
##   Optimization method                           NLMINB
##   Number of model parameters                       446
## 
##                                                   Used       Total
##   Number of observations                         35127       46427
## 
## Model Test User Model:
##                                                         
##   Test statistic                              825227.500
##   Degrees of freedom                               12920
##   P-value (Chi-square)                             0.000
## 
## Model Test Baseline Model:
## 
##   Test statistic                           2538331.585
##   Degrees of freedom                             13203
##   P-value                                        0.000
## 
## User Model versus Baseline Model:
## 
##   Comparative Fit Index (CFI)                    0.678
##   Tucker-Lewis Index (TLI)                       0.671
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)           -7709410.175
##   Loglikelihood unrestricted model (H1)   -7296796.425
##                                                       
##   Akaike (AIC)                            15419712.350
##   Bayesian (BIC)                          15423488.509
##   Sample-size adjusted Bayesian (SABIC)   15422071.122
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.042
##   90 Percent confidence interval - lower         0.042
##   90 Percent confidence interval - upper         0.042
##   P-value H_0: RMSEA <= 0.050                    1.000
##   P-value H_0: RMSEA >= 0.080                    0.000
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.074
## 
## Goodness of Fit Index:
## 
##   Goodness of Fit Index (GFI)                    0.664
##   90 Percent confidence interval - lower         0.663
##   90 Percent confidence interval - upper         0.664
## 
## Parameter Estimates:
## 
##   Standard errors                             Standard
##   Information                                 Expected
##   Information saturated (h1) model          Structured
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)
##   Warmth =~                                           
##     A1                1.000                           
##     A2                0.907    0.008  116.796    0.000
##     A3                0.799    0.008   97.549    0.000
##     A4                0.819    0.008  104.850    0.000
##     A5                0.854    0.007  114.200    0.000
##     A6                0.788    0.007  108.222    0.000
##     A7                0.730    0.007  100.615    0.000
##     A8                0.620    0.009   70.406    0.000
##     A9                0.864    0.008  106.308    0.000
##     A10               0.501    0.008   60.932    0.000
##   Intellect =~                                        
##     B1                1.000                           
##     B2                1.107    0.014   76.995    0.000
##     B3                0.810    0.011   72.482    0.000
##     B4                0.887    0.010   85.535    0.000
##     B5                0.993    0.012   80.909    0.000
##     B6                0.760    0.013   59.640    0.000
##     B7                0.723    0.013   54.657    0.000
##     B8                0.692    0.012   57.775    0.000
##     B9                1.002    0.016   62.204    0.000
##     B10               1.190    0.016   75.870    0.000
##     B11               0.826    0.016   52.107    0.000
##     B12               1.136    0.014   80.092    0.000
##     B13               1.112    0.016   69.476    0.000
##   EmotSta =~                                          
##     C1                1.000                           
##     C2                1.294    0.015   84.880    0.000
##     C3                0.892    0.012   74.627    0.000
##     C4                1.007    0.014   74.003    0.000
##     C5                1.107    0.015   74.148    0.000
##     C6                1.401    0.017   82.245    0.000
##     C7                1.585    0.018   89.142    0.000
##     C8                1.568    0.018   87.897    0.000
##     C9                1.407    0.016   85.686    0.000
##     C10               1.351    0.016   86.058    0.000
##   Assert =~                                           
##     D1                1.000                           
##     D2                0.788    0.007  107.111    0.000
##     D3                0.595    0.007   82.615    0.000
##     D4                0.587    0.007   80.777    0.000
##     D5                0.832    0.006  130.873    0.000
##     D6                0.506    0.006   83.606    0.000
##     D7                0.953    0.007  134.242    0.000
##     D8                0.494    0.006   79.039    0.000
##     D9                0.787    0.007  113.375    0.000
##     D10               0.779    0.008   97.128    0.000
##   Gregar =~                                           
##     E1                1.000                           
##     E2                1.195    0.008  147.997    0.000
##     E3                0.502    0.007   76.939    0.000
##     E4                1.075    0.008  135.968    0.000
##     E5                0.389    0.005   72.376    0.000
##     E6                0.737    0.008   92.805    0.000
##     E7                0.243    0.007   34.327    0.000
##     E8                1.009    0.008  126.263    0.000
##     E9                0.182    0.007   27.905    0.000
##     E10               0.498    0.008   64.492    0.000
##   Duti =~                                             
##     F1                1.000                           
##     F2                1.281    0.014   94.733    0.000
##     F3                0.790    0.016   48.169    0.000
##     F4                1.526    0.016   97.453    0.000
##     F5                0.958    0.015   64.289    0.000
##     F6                1.485    0.015   96.367    0.000
##     F7                1.357    0.015   90.393    0.000
##     F8                0.716    0.014   52.375    0.000
##     F9                1.489    0.015   96.937    0.000
##     F10               0.864    0.012   70.283    0.000
##   Friend =~                                           
##     G1                1.000                           
##     G2                1.104    0.008  146.184    0.000
##     G3                0.850    0.008  111.975    0.000
##     G4                1.122    0.007  158.342    0.000
##     G5                1.049    0.007  157.691    0.000
##     G6                1.156    0.007  155.565    0.000
##     G7                1.003    0.007  135.526    0.000
##     G8                0.709    0.007   99.902    0.000
##     G9                1.007    0.007  137.614    0.000
##     G10               0.939    0.007  134.239    0.000
##   Sens =~                                             
##     H1                1.000                           
##     H2                0.436    0.005   89.926    0.000
##     H3                1.068    0.005  203.172    0.000
##     H4                0.072    0.006   11.678    0.000
##     H5                0.155    0.007   21.820    0.000
##     H6                0.212    0.006   36.384    0.000
##     H7                0.172    0.007   25.600    0.000
##     H8                0.397    0.006   63.360    0.000
##     H9                0.392    0.005   77.523    0.000
##     H10               0.158    0.006   26.655    0.000
##   Dist =~                                             
##     I1                1.000                           
##     I2                1.033    0.013   81.293    0.000
##     I3                0.869    0.012   74.463    0.000
##     I4                1.499    0.015   97.554    0.000
##     I5                0.780    0.012   67.351    0.000
##     I6                1.020    0.013   80.370    0.000
##     I7                1.152    0.013   90.327    0.000
##     I8                1.405    0.015   96.341    0.000
##     I9                1.124    0.012   91.269    0.000
##     I10               0.922    0.012   78.039    0.000
##   Imag =~                                             
##     J1                1.000                           
##     J2                1.328    0.014   91.827    0.000
##     J3                1.252    0.015   85.968    0.000
##     J4                1.252    0.014   90.596    0.000
##     J5                0.673    0.011   60.718    0.000
##     J6                0.659    0.011   58.507    0.000
##     J7                0.647    0.011   57.134    0.000
##     J8                0.590    0.011   53.147    0.000
##     J9                1.144    0.014   80.221    0.000
##     J10               1.058    0.014   77.027    0.000
##   Reser =~                                            
##     K1                1.000                           
##     K2                0.833    0.007  113.944    0.000
##     K3                0.767    0.007  103.698    0.000
##     K4                0.776    0.007  111.451    0.000
##     K5                0.798    0.007  121.476    0.000
##     K6                1.128    0.007  164.339    0.000
##     K7                1.087    0.007  164.699    0.000
##     K8                0.516    0.007   72.865    0.000
##     K9                0.940    0.007  134.376    0.000
##     K10               0.751    0.006  117.066    0.000
##   Anx =~                                              
##     L1                1.000                           
##     L2                0.958    0.009  106.417    0.000
##     L3                1.085    0.010  111.895    0.000
##     L4                0.923    0.008  108.930    0.000
##     L5                0.852    0.009   96.666    0.000
##     L6                0.700    0.009   74.979    0.000
##     L7                0.908    0.009  102.565    0.000
##     L8                0.835    0.009   88.412    0.000
##     L9                0.855    0.009   97.660    0.000
##     L10               0.885    0.009  101.337    0.000
##   Complex =~                                          
##     M1                1.000                           
##     M2                0.899    0.015   60.116    0.000
##     M3                0.801    0.012   65.360    0.000
##     M4                1.068    0.017   64.564    0.000
##     M5                0.625    0.015   40.985    0.000
##     M6                1.515    0.021   71.353    0.000
##     M7                1.190    0.019   62.960    0.000
##     M8                1.453    0.021   70.317    0.000
##     M9                1.235    0.018   68.609    0.000
##     M10               1.212    0.019   64.739    0.000
##   Intr =~                                             
##     N1                1.000                           
##     N2                0.775    0.007  108.927    0.000
##     N3                0.727    0.006  119.027    0.000
##     N4                0.936    0.007  131.165    0.000
##     N5                0.593    0.008   77.295    0.000
##     N6                0.761    0.007  107.645    0.000
##     N7                0.560    0.005  102.222    0.000
##     N8                0.727    0.007  107.999    0.000
##     N9                0.719    0.008   95.143    0.000
##     N10               0.623    0.008   78.019    0.000
##   Order =~                                            
##     O1                1.000                           
##     O2                2.543    0.050   50.422    0.000
##     O3                1.703    0.035   48.881    0.000
##     O4                1.518    0.034   45.129    0.000
##     O5                1.368    0.030   45.559    0.000
##     O6                1.773    0.039   46.024    0.000
##     O7                1.921    0.041   47.273    0.000
##     O8                3.013    0.059   51.243    0.000
##     O9                2.347    0.048   49.026    0.000
##     O10               1.887    0.039   47.884    0.000
##   Emot =~                                             
##     P1                1.000                           
##     P2                0.940    0.006  163.104    0.000
##     P3                0.530    0.006   88.216    0.000
##     P4                0.512    0.006   89.093    0.000
##     P5                0.437    0.005   84.117    0.000
##     P6                0.484    0.006   81.521    0.000
##     P7                0.396    0.006   63.895    0.000
##     P8                0.846    0.006  148.026    0.000
##     P9                0.435    0.006   76.558    0.000
##     P10               0.394    0.006   68.305    0.000
## 
## Covariances:
##                    Estimate  Std.Err  z-value  P(>|z|)
##   Warmth ~~                                           
##     Intellect         0.045    0.002   19.331    0.000
##     EmotSta           0.119    0.003   40.813    0.000
##     Assert            0.139    0.004   36.779    0.000
##     Gregar            0.311    0.005   65.131    0.000
##     Duti              0.117    0.003   40.744    0.000
##     Friend            0.376    0.005   77.040    0.000
##     Sens              0.034    0.005    7.444    0.000
##     Dist             -0.225    0.004  -59.546    0.000
##     Imag             -0.034    0.003  -12.601    0.000
##     Reser            -0.369    0.005  -72.007    0.000
##     Anx              -0.035    0.003  -10.163    0.000
##     Complex           0.084    0.002   34.019    0.000
##     Intr             -0.297    0.005  -63.609    0.000
##     Order             0.031    0.002   20.261    0.000
##     Emot             -0.267    0.005  -53.438    0.000
##   Intellect ~~                                        
##     EmotSta           0.090    0.002   42.220    0.000
##     Assert            0.197    0.003   60.419    0.000
##     Gregar           -0.023    0.003   -8.550    0.000
##     Duti             -0.026    0.002  -14.976    0.000
##     Friend            0.063    0.003   24.553    0.000
##     Sens              0.232    0.004   58.667    0.000
##     Dist              0.006    0.002    3.482    0.000
##     Imag              0.056    0.002   28.282    0.000
##     Reser            -0.029    0.003  -10.569    0.000
##     Anx              -0.114    0.003  -41.711    0.000
##     Complex           0.163    0.003   55.379    0.000
##     Intr              0.074    0.003   27.102    0.000
##     Order             0.031    0.001   26.585    0.000
##     Emot             -0.086    0.003  -26.908    0.000
##   EmotSta ~~                                          
##     Assert            0.211    0.004   56.972    0.000
##     Gregar            0.132    0.003   39.308    0.000
##     Duti              0.072    0.002   33.265    0.000
##     Friend            0.247    0.004   61.511    0.000
##     Sens             -0.018    0.004   -5.172    0.000
##     Dist             -0.156    0.003  -51.590    0.000
##     Imag             -0.104    0.003  -41.240    0.000
##     Reser            -0.155    0.004  -43.925    0.000
##     Anx              -0.376    0.005  -68.570    0.000
##     Complex           0.036    0.002   20.334    0.000
##     Intr             -0.146    0.003  -43.165    0.000
##     Order             0.048    0.001   32.170    0.000
##     Emot             -0.359    0.005  -66.888    0.000
##   Assert ~~                                           
##     Gregar            0.235    0.005   51.078    0.000
##     Duti             -0.020    0.003   -7.516    0.000
##     Friend            0.370    0.005   75.878    0.000
##     Sens              0.047    0.005    9.370    0.000
##     Dist             -0.024    0.003   -8.180    0.000
##     Imag             -0.059    0.003  -19.623    0.000
##     Reser            -0.219    0.005  -47.302    0.000
##     Anx              -0.277    0.005  -60.925    0.000
##     Complex           0.106    0.003   38.450    0.000
##     Intr             -0.132    0.004  -31.046    0.000
##     Order             0.078    0.002   36.130    0.000
##     Emot             -0.040    0.005   -8.186    0.000
##   Gregar ~~                                           
##     Duti             -0.020    0.003   -6.772    0.000
##     Friend            0.538    0.006   88.407    0.000
##     Sens             -0.184    0.006  -33.177    0.000
##     Dist             -0.140    0.004  -39.876    0.000
##     Imag             -0.016    0.003   -4.913    0.000
##     Reser            -0.345    0.005  -63.469    0.000
##     Anx              -0.141    0.004  -32.927    0.000
##     Complex           0.016    0.003    6.375    0.000
##     Intr             -0.485    0.006  -79.410    0.000
##     Order            -0.015    0.002   -9.087    0.000
##     Emot             -0.093    0.005  -17.515    0.000
##   Duti ~~                                             
##     Friend            0.056    0.003   19.773    0.000
##     Sens             -0.023    0.004   -6.588    0.000
##     Dist             -0.126    0.003  -47.210    0.000
##     Imag             -0.158    0.003  -53.202    0.000
##     Reser            -0.076    0.003  -24.417    0.000
##     Anx              -0.007    0.003   -2.646    0.008
##     Complex          -0.068    0.002  -34.325    0.000
##     Intr             -0.082    0.003  -27.119    0.000
##     Order             0.075    0.002   39.158    0.000
##     Emot             -0.146    0.004  -38.826    0.000
##   Friend ~~                                           
##     Sens             -0.073    0.005  -14.513    0.000
##     Dist             -0.211    0.004  -56.194    0.000
##     Imag             -0.114    0.003  -35.699    0.000
##     Reser            -0.482    0.006  -83.318    0.000
##     Anx              -0.271    0.005  -59.695    0.000
##     Complex           0.057    0.002   22.944    0.000
##     Intr             -0.452    0.006  -80.620    0.000
##     Order             0.042    0.002   23.978    0.000
##     Emot             -0.247    0.005  -47.273    0.000
##   Sens ~~                                             
##     Dist              0.005    0.004    1.229    0.219
##     Imag              0.173    0.004   41.725    0.000
##     Reser             0.033    0.006    6.045    0.000
##     Anx               0.006    0.005    1.219    0.223
##     Complex           0.229    0.004   53.692    0.000
##     Intr              0.252    0.006   44.922    0.000
##     Order             0.003    0.002    1.718    0.086
##     Emot             -0.072    0.006  -11.450    0.000
##   Dist ~~                                             
##     Imag              0.094    0.003   37.328    0.000
##     Reser             0.268    0.004   60.801    0.000
##     Anx               0.143    0.003   43.392    0.000
##     Complex          -0.010    0.002   -5.710    0.000
##     Intr              0.229    0.004   56.477    0.000
##     Order            -0.018    0.001  -14.858    0.000
##     Emot              0.318    0.005   63.357    0.000
##   Imag ~~                                             
##     Reser             0.087    0.003   25.777    0.000
##     Anx               0.118    0.003   37.386    0.000
##     Complex           0.135    0.003   48.962    0.000
##     Intr              0.170    0.004   46.729    0.000
##     Order            -0.076    0.002  -38.556    0.000
##     Emot              0.118    0.004   30.514    0.000
##   Reser ~~                                            
##     Anx               0.114    0.004   26.597    0.000
##     Complex          -0.056    0.003  -20.535    0.000
##     Intr              0.401    0.006   70.747    0.000
##     Order            -0.026    0.002  -14.528    0.000
##     Emot              0.171    0.006   30.948    0.000
##   Anx ~~                                              
##     Complex          -0.056    0.002  -23.138    0.000
##     Intr              0.138    0.004   32.817    0.000
##     Order            -0.038    0.002  -22.601    0.000
##     Emot              0.459    0.006   73.932    0.000
##   Complex ~~                                          
##     Intr              0.039    0.003   15.348    0.000
##     Order            -0.009    0.001   -9.319    0.000
##     Emot             -0.103    0.003  -31.667    0.000
##   Intr ~~                                             
##     Order            -0.006    0.002   -3.931    0.000
##     Emot              0.218    0.005   40.239    0.000
##   Order ~~                                            
##     Emot             -0.044    0.002  -20.728    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)
##    .A1                0.590    0.005  116.592    0.000
##    .A2                0.463    0.004  115.829    0.000
##    .A3                0.693    0.006  123.897    0.000
##    .A4                0.573    0.005  121.544    0.000
##    .A5                0.452    0.004  117.342    0.000
##    .A6                0.473    0.004  120.215    0.000
##    .A7                0.524    0.004  122.985    0.000
##    .A8                1.025    0.008  129.023    0.000
##    .A9                0.608    0.005  120.991    0.000
##    .A10               0.948    0.007  130.051    0.000
##    .B1                0.558    0.005  122.992    0.000
##    .B2                0.731    0.006  123.616    0.000
##    .B3                0.491    0.004  125.438    0.000
##    .B4                0.292    0.002  118.132    0.000
##    .B5                0.477    0.004  121.535    0.000
##    .B6                0.800    0.006  128.707    0.000
##    .B7                0.920    0.007  129.524    0.000
##    .B8                0.725    0.006  129.034    0.000
##    .B9                1.230    0.010  128.209    0.000
##    .B10               0.895    0.007  124.119    0.000
##    .B11               1.360    0.010  129.878    0.000
##    .B12               0.654    0.005  122.019    0.000
##    .B13               1.072    0.008  126.409    0.000
##    .C1                1.000    0.008  128.625    0.000
##    .C2                0.654    0.005  122.467    0.000
##    .C3                0.639    0.005  127.664    0.000
##    .C4                0.845    0.007  127.844    0.000
##    .C5                1.013    0.008  127.803    0.000
##    .C6                0.947    0.008  124.395    0.000
##    .C7                0.652    0.006  117.267    0.000
##    .C8                0.728    0.006  119.182    0.000
##    .C9                0.722    0.006  121.731    0.000
##    .C10               0.644    0.005  121.360    0.000
##    .D1                0.410    0.004  105.119    0.000
##    .D2                0.755    0.006  123.351    0.000
##    .D3                0.837    0.007  127.820    0.000
##    .D4                0.859    0.007  128.066    0.000
##    .D5                0.454    0.004  115.444    0.000
##    .D6                0.586    0.005  127.682    0.000
##    .D7                0.544    0.005  113.808    0.000
##    .D8                0.641    0.005  128.290    0.000
##    .D9                0.640    0.005  121.729    0.000
##    .D10               0.958    0.008  125.477    0.000
##    .E1                0.585    0.005  112.127    0.000
##    .E2                0.573    0.006  102.590    0.000
##    .E3                0.817    0.006  128.902    0.000
##    .E4                0.694    0.006  112.680    0.000
##    .E5                0.567    0.004  129.391    0.000
##    .E6                1.103    0.009  126.725    0.000
##    .E7                1.110    0.008  131.905    0.000
##    .E8                0.815    0.007  117.714    0.000
##    .E9                0.952    0.007  132.121    0.000
##    .E10               1.211    0.009  130.122    0.000
##    .F1                0.840    0.007  126.688    0.000
##    .F2                0.417    0.004  113.030    0.000
##    .F3                2.153    0.016  131.109    0.000
##    .F4                0.446    0.004  106.579    0.000
##    .F5                1.433    0.011  129.389    0.000
##    .F6                0.476    0.004  109.547    0.000
##    .F7                0.671    0.006  118.994    0.000
##    .F8                1.429    0.011  130.771    0.000
##    .F9                0.450    0.004  108.066    0.000
##    .F10               0.869    0.007  128.323    0.000
##    .G1                0.426    0.004  117.899    0.000
##    .G2                0.728    0.006  122.143    0.000
##    .G3                0.962    0.008  127.879    0.000
##    .G4                0.559    0.005  118.510    0.000
##    .G5                0.497    0.004  118.740    0.000
##    .G6                0.636    0.005  119.461    0.000
##    .G7                0.773    0.006  124.456    0.000
##    .G8                0.895    0.007  129.064    0.000
##    .G9                0.741    0.006  124.052    0.000
##    .G10               0.697    0.006  124.695    0.000
##    .H1                0.194    0.004   47.101    0.000
##    .H2                0.752    0.006  128.930    0.000
##    .H3                0.344    0.005   66.883    0.000
##    .H4                1.314    0.010  132.473    0.000
##    .H5                1.753    0.013  132.336    0.000
##    .H6                1.173    0.009  131.990    0.000
##    .H7                1.560    0.012  132.263    0.000
##    .H8                1.315    0.010  130.839    0.000
##    .H9                0.834    0.006  129.930    0.000
##    .H10               1.222    0.009  132.241    0.000
##    .I1                0.962    0.008  126.938    0.000
##    .I2                0.760    0.006  124.967    0.000
##    .I3                0.783    0.006  127.345    0.000
##    .I4                0.428    0.004  104.018    0.000
##    .I5                0.905    0.007  128.913    0.000
##    .I6                0.782    0.006  125.363    0.000
##    .I7                0.517    0.004  118.647    0.000
##    .I8                0.437    0.004  107.988    0.000
##    .I9                0.456    0.004  117.546    0.000
##    .I10               0.727    0.006  126.246    0.000
##    .J1                0.817    0.007  122.881    0.000
##    .J2                0.582    0.005  108.442    0.000
##    .J3                0.787    0.007  116.769    0.000
##    .J4                0.570    0.005  110.701    0.000
##    .J5                0.849    0.007  128.340    0.000
##    .J6                0.906    0.007  128.764    0.000
##    .J7                0.932    0.007  129.006    0.000
##    .J8                0.944    0.007  129.632    0.000
##    .J9                0.923    0.008  121.356    0.000
##    .J10               0.939    0.008  123.139    0.000
##    .K1                0.552    0.005  114.506    0.000
##    .K2                0.980    0.008  125.535    0.000
##    .K3                1.074    0.008  127.126    0.000
##    .K4                0.905    0.007  125.957    0.000
##    .K5                0.747    0.006  124.095    0.000
##    .K6                0.470    0.004  105.537    0.000
##    .K7                0.432    0.004  105.217    0.000
##    .K8                1.146    0.009  130.236    0.000
##    .K9                0.752    0.006  120.881    0.000
##    .K10               0.737    0.006  124.970    0.000
##    .L1                0.827    0.007  122.550    0.000
##    .L2                0.728    0.006  122.121    0.000
##    .L3                0.748    0.006  119.483    0.000
##    .L4                0.612    0.005  121.010    0.000
##    .L5                0.830    0.007  125.339    0.000
##    .L6                1.209    0.009  129.213    0.000
##    .L7                0.759    0.006  123.570    0.000
##    .L8                1.071    0.008  127.183    0.000
##    .L9                0.806    0.006  125.072    0.000
##    .L10               0.754    0.006  123.978    0.000
##    .M1                0.946    0.007  127.367    0.000
##    .M2                0.655    0.005  126.498    0.000
##    .M3                0.344    0.003  123.375    0.000
##    .M4                0.653    0.005  123.979    0.000
##    .M5                1.116    0.009  130.827    0.000
##    .M6                0.669    0.006  115.570    0.000
##    .M7                0.924    0.007  125.032    0.000
##    .M8                0.696    0.006  117.557    0.000
##    .M9                0.604    0.005  120.094    0.000
##    .M10               0.830    0.007  123.851    0.000
##    .N1                0.639    0.006  113.871    0.000
##    .N2                0.692    0.006  122.230    0.000
##    .N3                0.451    0.004  118.559    0.000
##    .N4                0.504    0.005  111.790    0.000
##    .N5                1.046    0.008  128.549    0.000
##    .N6                0.693    0.006  122.614    0.000
##    .N7                0.440    0.004  124.078    0.000
##    .N8                0.626    0.005  122.509    0.000
##    .N9                0.892    0.007  125.668    0.000
##    .N10               1.127    0.009  128.456    0.000
##    .O1                0.922    0.007  129.814    0.000
##    .O2                0.804    0.007  112.203    0.000
##    .O3                0.561    0.005  119.511    0.000
##    .O4                0.898    0.007  126.096    0.000
##    .O5                0.683    0.005  125.654    0.000
##    .O6                1.065    0.009  125.120    0.000
##    .O7                1.004    0.008  123.295    0.000
##    .O8                0.808    0.008  104.138    0.000
##    .O9                1.028    0.009  119.036    0.000
##    .O10               0.859    0.007  122.110    0.000
##    .P1                0.401    0.004   90.771    0.000
##    .P2                0.582    0.005  107.074    0.000
##    .P3                0.983    0.008  127.851    0.000
##    .P4                0.899    0.007  127.744    0.000
##    .P5                0.746    0.006  128.331    0.000
##    .P6                0.980    0.008  128.617    0.000
##    .P7                1.118    0.009  130.233    0.000
##    .P8                0.657    0.006  114.394    0.000
##    .P9                0.912    0.007  129.128    0.000
##    .P10               0.958    0.007  129.879    0.000
##     Warmth            0.527    0.008   69.795    0.000
##     Intellect         0.233    0.005   51.196    0.000
##     EmotSta           0.314    0.007   46.901    0.000
##     Assert            0.622    0.008   82.607    0.000
##     Gregar            0.731    0.009   78.463    0.000
##     Duti              0.317    0.006   50.255    0.000
##     Friend            0.675    0.008   86.162    0.000
##     Sens              1.089    0.010  105.517    0.000
##     Dist              0.355    0.007   49.784    0.000
##     Imag              0.337    0.007   50.792    0.000
##     Reser             0.805    0.010   83.324    0.000
##     Anx               0.587    0.009   65.016    0.000
##     Complex           0.214    0.005   39.193    0.000
##     Intr              0.702    0.009   75.082    0.000
##     Order             0.093    0.004   26.438    0.000
##     Emot              0.994    0.011   93.936    0.000

The fit statistics mostly indicate poor fit. Our comparative fit indices, CFI = 0.678, TLI = 0.671, should be much higher, preferably in the 0.9 range. And GFI (0.664), a measure of absolute fit, is similarly low. But RMSEA, 0.042, is good, and SRMR (0.074) is acceptable. There a few potential issues that could explain poor fit, including the fact our observed variables aren’t normally distributed, but before we try a different estimation method or way of dealing with missing data, let’s look at what the model is telling us. All items load onto their factor significantly and positively. Factors are correlated with each other, except sensitivity and distrust, sensitivity and anxiety, and sensitivity and orderliness. Let’s extract parameter estimates, as well as standardized estimates, so we can dig in a bit more. Remember that for the unstandardized solution, the factors (the latent variables in our model) don’t have a metric, so the first item (observed variables) is always loaded at 1 to set the range. If we want to see how strongly that item loads on a factor, you’ll want the standardized values.

parameters <- parameterEstimates(CFA16)
standardizedsol <- standardizedSolution(CFA16)

unstandardized <- parameters[c(1:163), 4]
standardized <- standardizedsol[c(1:163), 4]

ByItem <- cbind(ByItem, unstandardized, standardized)

summary(ByItem$standardized)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
## 0.06515 0.44843 0.56886 0.55692 0.68524 0.92115

We have at least one factor loading that is quite low - about 0.07 standardized loading. Turns out, that’s item H4, the action movie item from the Sensitivity scale that we also flagged for low P, low discrimination, and poor scale function. Every indicator we’ve examined has show this item doesn’t really relate to the concept of “Sensitivity.” In fact, our four lowest factor ratings all belong to the Sensitivity scale. This scale also had our worst alpha. My recommendation would be, at the very least, to drop the 4th item from Sensitivity and rerun alpha.

Scores <- Items %>%
  filter(Item != "H4") %>%
  ungroup() %>%
  group_by(ID, Code) %>%
  summarise(Total = sum(Rating, na.rm = TRUE),
            NItems = sum(!is.na(Rating))) %>%
  ungroup() %>%
  left_join(Scales, by = "Code") %>%
  mutate(Items = ifelse(Name == "Sensitivity", 9, Items),
         Score = Total/NItems,
         CorTotal = Score*Items) 
## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by ID and Code.
## ℹ Output is grouped by ID.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(ID, Code))` for per-operation grouping
##   (`?dplyr::dplyr_by`) instead.
ByScale <- Scores %>%
  group_by(Code) %>%
  summarise(Variance = var(CorTotal, na.rm = TRUE))

ByScale2 <- ByItem %>%
  ungroup() %>%
  mutate(Code = substr(Column, 1, 1)) %>%
  group_by(Code) %>%
  summarise(EItemVar = sum(Variance))

Scales <- Scales %>%
  left_join(ByScale, by = "Code") %>%
  left_join(ByScale2, by = "Code")

Scales <- Scales %>%
  mutate(Items.y = ifelse(Name == "Sensitivity", 9, Items),
         Alpha.y = (Items.y/(Items.y-1)) * (1 - (EItemVar.y/Variance.y)))

Removing H4 actually makes alpha go down. Remember, though, that alpha rewards longer measures (more items increase reliability in CTT), so any gain from removing a poor item could be lost because now the measure is shorter. We could drop Sensitivity fully from the measure, but then that would break the factor structure Cattell theorized that included this concept of Sensitivity. If full removal is unacceptable, then the Sensitivity scale probably needs an overhaul with edited/new items.

Note that, while I said our CFA could handle some missing data, the estimation method (maximum likelihood, the default) uses only complete cases. An estimation method that allows missing data might get better results. Further, our data is not normally distributed, another requirement of this estimation method. So we want a different estimator that can handle some missing data and that works with nonnormal data. This paper recommends a weighted least squares method with rating scale items entered as ordinal. This approach definitely makes sense, but fair warning, this approach will use a lot of memory (to store the many matrices) and will take a while to run, especially if you don’t have a lot of RAM and/or are running programs like RStudio that use a lot of RAM. My current system has 48 gigs of RAM, but on my previous 16 gig system, I would sometimes need to close RStudio and run my code directly in R.

This is the code I ran: CFA2 <- cfa(model = Factor16, data = PF16R, ordered = TRUE, estimator = “WLSMV”)

Why am I telling you instead of showing you? Unfortunately, this analysis took over 1.5 hours and found similar results as the ML estimation model: fit statistics were in the 0.5-0.6 range, lack of correlation between Sensitivity and multiple other scales, and H4 had the lowest loading. So the issue isn’t that the data aren’t normally distributed. The issue is that the prescribed model does not fit the data well.

So, out of pure curiosity, I’m going to do something I typically do not recommend. I’m going to run an exploratory factor analysis, asking it to derive up to 16 factors. Normally I do not condone this kind of fishing. If you expect certain items to load on certain factors, you should use CFA. And if your CFA doesn’t work out, you should have some backup options - a different estimator, a different arrangement of factors, or even just accepting that your theoretical model isn’t it. But I don’t want to use this approach to see how the items should fall, to try to develop a new measure or personality theory. I want to use it for diagnosis - where do things fall apart? Do we have a small number of problem factors, or is the issue more widespread than that?

EFA in lavaan allows you to specify a number of factors, or even a range of factors, to extract. I definitely want to see what it does with the 16 factor solution. We could compare that to another personality theory, Big Five, which states that personality is made up of five independent but correlated factors: Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Emotional Stability. I could see how some of the 16 factors could map onto the Big Five, but the one exception I see is Imagination. It doesn’t have a clear place among the Big Five, and seems to be a unique contribution by Cattell. So let’s see what happens if I ask the EFA to extract 5, 6, and 16 factors.

ItemList <- grep("^[A-Za-z][0-9]+$", colnames(PF16R), value = TRUE)

EFA <- efa(data = PF16R, nfactors = c(5, 6, 16), rotation = "varimax", ov_names = ItemList)

I’ll start by looking at the 16 factor solution.

summary(EFA$nf16, fit.measures = TRUE)
## lavaan 0.7-2 ended normally after 1 iteration
## 
##   Estimator                                         ML
##   Optimization method                           NLMINB
##   Number of model parameters                      2771
##   Row rank of the constraints matrix               283
## 
##   Rotation method                   VARIMAX ORTHOGONAL
##   Rotation algorithm (rstarts)                GPA (30)
##   Standardized metric                             TRUE
##   Row weights                                   Kaiser
## 
##                                                   Used       Total
##   Number of observations                         35127       46427
## 
## Model Test User Model:
##                                                         
##   Test statistic                              261484.745
##   Degrees of freedom                               10715
##   P-value (Chi-square)                             0.000
## 
## Model Test Baseline Model:
## 
##   Test statistic                           2538331.585
##   Degrees of freedom                             13203
##   P-value                                        0.000
## 
## User Model versus Baseline Model:
## 
##   Comparative Fit Index (CFI)                    0.901
##   Tucker-Lewis Index (TLI)                       0.878
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)           -7427538.797
##   Loglikelihood unrestricted model (H1)   -7296796.425
##                                                       
##   Akaike (AIC)                            14860379.594
##   Bayesian (BIC)                          14882824.883
##   Sample-size adjusted Bayesian (SABIC)   14874400.014
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.026
##   90 Percent confidence interval - lower         0.026
##   90 Percent confidence interval - upper         0.026
##   P-value H_0: RMSEA <= 0.050                    1.000
##   P-value H_0: RMSEA >= 0.080                    0.000
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.015
## 
## Goodness of Fit Index:
## 
##   Goodness of Fit Index (GFI)                    0.899
##   90 Percent confidence interval - lower         0.899
##   90 Percent confidence interval - upper         0.900
## 
## Standardized loadings: (* = significant at 1% level)
## 
##         f1      f2      f3      f4      f5      f6      f7      f8      f9 
## A1   0.591*      .*       *       *       *      .*               *       *
## A2   0.502*       *      .*       *       *  0.357*       *      .*      .*
## A3   0.612*               *       *      .*       *               *      .*
## A4   0.489*               *       *       *      .*               *      .*
## A5   0.532*       *      .*       *      .*  0.363*               *      .*
## A6   0.510*       *      .*       *      .*  0.319*               *      .*
## A7   0.517*       *      .*      .*              .*       *       *      .*
## A8       .*       *               *       *      .*               *      .*
## A9   0.490*       *       *       *       *      .*       *       *      .*
## A10  0.359*       *       *       *       *       *               *       *
## B1        *      .*       *       *       *               *               *
## B2       .*      .*      .*       *       *               *               *
## B3        *      .*               *       *       *       *       *       *
## B4        *      .*       *       *      .*       *       *                
## B5        *      .*       *              .*       *       *       *       *
## B6       .*      .*      .*      .*       *       *               *      .*
## B7        *       *       *      .*      .*      .*       *       *       *
## B8        *      .*       *      .*      .*       *               *       *
## B9        *      .*      .*      .*      .*       *       *               *
## B10       *      .*       *       *  0.439*       *       *      .*       *
## B11       *      .*      .*       *      .*      .*       *       *       *
## B12       *       *      .*              .*              .*      .*       *
## B13       *       *       *              .*       *      .*      .*       *
## C1        *                       *  0.445*       *       *               *
## C2       .*      .*       *       *  0.615*      .*       *      .*      .*
## C3       .*      .*       *          0.465*       *               *       *
## C4        *       *      .*       *  0.514*       *       *      .*      .*
## C5        *       *       *       *  0.492*                       *      .*
## C6        *       *              .*  0.591*       *       *      .*      .*
## C7        *                      .*  0.680*      .*       *       *      .*
## C8       .*       *      .*       *  0.661*      .*       *       *      .*
## C9        *       *       *      .*  0.652*      .*       *      .*       *
## C10       *      .*       *          0.664*      .*       *       *       *
## D1        *  0.668*       *       *      .*      .*       *       *        
## D2        *  0.574*       *                      .*       *       *       *
## D3        *  0.324*      .*      .*      .*      .*       *       *       *
## D4       .*  0.336*      .*       *      .*      .*               *       *
## D5        *  0.616*               *      .*      .*       *       *       *
## D6        *  0.385*      .*       *      .*       *              .*       *
## D7        *  0.518*       *       *  0.322*      .*       *       *       *
## D8        *      .*       *      .*      .*       *       *       *       *
## D9           0.469*      .*       *      .*      .*               *       *
## D10       *      .*       *       *  0.471*      .*       *       *       *
## E1       .*      .*       *       *      .*  0.667*       *      .*       *
## E2        *       *      .*       *      .*  0.617*       *      .*       *
## E3       .*          0.474*       *          0.315*       *      .*       *
## E4        *       *      .*       *       *  0.550*              .*      .*
## E5       .*       *  0.398*               *  0.322*       *       *        
## E6        *       *      .*      .*       *  0.395*       *  0.372*       *
## E7       .*       *  0.311*       *       *      .*       *      .*        
## E8        *       *       *       *      .*  0.516*       *      .*      .*
## E9        *              .*              .*      .*               *       *
## E10       *       *      .*       *              .*       *      .*       *
## F1               .*          0.521*       *               *       *       *
## F2       .*               *  0.731*       *       *       *       *      .*
## F3       .*       *       *      .*       *       *              .*       *
## F4       .*       *       *  0.747*       *       *       *       *      .*
## F5       .*      .*       *  0.339*      .*      .*               *      .*
## F6       .*       *       *  0.738*       *       *       *       *      .*
## F7        *      .*      .*  0.646*       *       *              .*      .*
## F8        *       * -0.347*      .*      .*       *       *              .*
## F9       .*       *          0.742*       *       *       *       *      .*
## F10       *      .*      .*  0.409*       *       *              .*      .*
## G1       .*      .*       *       *  0.306*  0.591*       *              .*
## G2       .*       *       *       *      .*  0.694*               *       *
## G3        *      .*               *      .*  0.538*       *      .*       *
## G4       .*       *       *       *      .*  0.659*               *      .*
## G5       .*      .*       *              .*  0.653*       *       *       *
## G6       .*      .*       *       *  0.341*  0.619*       *      .*       *
## G7       .*       *       *       *  0.357*  0.521*       *       *      .*
## G8       .*      .*               *      .*  0.407*              .*        
## G9        *      .*       *       *      .*  0.617*              .*       *
## G10       *      .*       *       *      .*  0.583*       *       *       *
## H1        *               *       *               *  0.783*               *
## H2       .*       *       *       *       *       *  0.304*      .*       *
## H3        *       *       *                      .*  0.771*                
## H4                *      .*       *      .*       *       *      .*        
## H5   0.320*       *      .*       *      .*       *      .*       *       *
## H6   0.385*       *      .*       *       *       *      .*      .*       *
## H7       .*       *      .*       *       *      .*       *      .*       *
## H8       .*       *      .*       *       *       *      .*      .*        
## H9        *      .*       *       *       *       *      .*       *       *
## H10      .*       *       *       *                              .*       *
## I1       .*       *               *      .*       *       *          0.392*
## I2        *       *       *      .*      .*       *       *      .*  0.494*
## I3        *       *       *       *      .*      .*       *      .*  0.390*
## I4       .*       *       *      .*      .*      .*               *  0.687*
## I5        *       *       *       *      .*       *       *      .*  0.357*
## I6       .*       *              .*      .*       *              .*  0.477*
## I7       .*       *       *      .*       *      .*               *  0.677*
## I8       .*       *       *      .*      .*      .*               *  0.704*
## I9       .*               *      .*      .*       *               *  0.633*
## I10      .*       *       *      .*       *       *       *       *  0.505*
## J1        *       *      .*      .*      .*       *          0.366*      .*
## J2        *      .*              .*      .*      .*       *  0.305*       *
## J3        *       *       *      .*      .*       *       *  0.465*       *
## J4        *      .*       *      .*      .*       *          0.407*       *
## J5               .*       * -0.303*       *                      .*      .*
## J6       .*      .*       * -0.346*      .*                      .*      .*
## J7        *      .*      .* -0.301*       *      .*          0.398*       *
## J8        *       *         -0.559*       *       *       *       *       *
## J9                *       *       *      .*       *       *      .*       *
## J10       *       *       *      .*      .*      .*       *       *       *
## K1       .*                       *       *      .*       *       *      .*
## K2       .*       *       *      .*      .* -0.378*       *      .*      .*
## K3       .*      .*       *       *       * -0.602*              .*       *
## K4        *       *      .*       * -0.337*      .*               *      .*
## K5        *      .*               *      .*      .*       *      .*       *
## K6       .*       *       *       *       *      .*               *      .*
## K7       .*       *       *       *       *      .*       *       *      .*
## K8        *       *       *       *       *      .*       *       *       *
## K9       .*       *       *       *       *      .*       *       *      .*
## K10      .*       *               *       *      .*       *       *      .*
## L1        *      .*       *       * -0.586*      .*       *       *       *
## L2        *       *       *         -0.622*      .*       *       *      .*
## L3       .*       *       *         -0.648*       *       *       *        
## L4       .*       *       *       * -0.616*       *       *       *      .*
## L5        *       *       *       * -0.557*       *       *       *       *
## L6       .*      .*              .* -0.410*       *                      .*
## L7        *       *       *         -0.586*       *       *       *       *
## L8        *       *       *       * -0.484*       *       *      .*       *
## L9        *               *         -0.551*       *       *       *       *
## L10       *      .*       *         -0.604*       *              .*       *
## M1       .*       *      .*       *       *       *      .*      .*       *
## M2       .*      .*       *      .*      .*      .*       *  0.320*       *
## M3       .*       *       *       *      .*       *       *      .*       *
## M4        *      .*              .*       *      .*       *       *       *
## M5       .*       *       *      .*      .*      .*       *      .*       *
## M6                *       *      .*       *                      .*        
## M7       .*               *       *       *       *       *               *
## M8        *                       *       *       *       *       *        
## M9        *       *       *      .*       *       *       *      .*       *
## M10       *       *       *      .*       *       *       *       *        
## N1       .*               *       *      .* -0.301*       *       *      .*
## N2       .*      .*       *       *      .*      .*       *      .*      .*
## N3        *       *                       *      .*       *       *       *
## N4        *       *       *              .* -0.387*       *               *
## N5        *               *       *       *      .*               *       *
## N6        *       *       *       *              .*       *       *       *
## N7                        *       *       *      .*               *      .*
## N8       .*              .*      .*      .* -0.423*              .*      .*
## N9       .*       *       *      .*      .*      .*       *      .*      .*
## N10      .*       *       *       *      .*      .*               *      .*
## O1        *      .*              .*      .*       *       *       *       *
## O2       .*      .*       *      .*      .*       *       *       *       *
## O3        *      .*       *  0.332*       *                       *       *
## O4        *      .*       *      .*       *       *       *      .*       *
## O5        *      .*       *      .*      .*       *               *       *
## O6                *       *      .*       *       *       *      .*       *
## O7        *       *       *      .*      .*               *      .*       *
## O8        *               *       *      .*       *       *       *       *
## O9        *       *               *      .*       *       *       *       *
## O10       *      .*       *      .*  0.300*       *       *               *
## P1       .*      .*      .*      .* -0.519*       *       *       *      .*
## P2       .*      .*      .*      .* -0.454*               *       *      .*
## P3       .*      .*       *       *      .*               *       *      .*
## P4       .*      .*       *       *      .*       *               *      .*
## P5       .*       *               * -0.493*       *               *       *
## P6        *       *       *              .*       *       *       *      .*
## P7       .*       *       *       *      .*       *       *       *       *
## P8       .*      .*       *       * -0.415*       *       *       *      .*
## P9       .*       *       *      .*      .*       *       *      .*  0.355*
## P10 -0.400*                      .*       *      .*              .*      .*
##        f10     f11     f12     f13     f14     f15     f16       unique.var
## A1        *      .*       *       *               *       *           0.504
## A2        *      .*       *       *      .*       *       *           0.489
## A3               .*       *       *       *       *       *           0.550
## A4               .*       *      .*      .*       *                   0.613
## A5                *       *               *               *           0.488
## A6        *      .*      .*       *       *       *       *           0.537
## A7                *       *       *      .*       *       *           0.621
## A8        *      .*              .*      .*       *       *           0.799
## A9        *      .*       *      .*      .*               *           0.548
## A10       *       *       *      .*       *       *      .*           0.775
## B1        *       *       *  0.486*      .*              .*           0.665
## B2                        *  0.393*      .*              .*           0.667
## B3        *               *  0.456*      .*      .*      .*           0.651
## B4                *       *  0.476*      .*      .*      .*           0.636
## B5                *       *  0.382*       *       *       *           0.708
## B6                *          0.350*       *       *      .*           0.677
## B7        *       *      .*      .*      .*      .*       *           0.703
## B8                *      .*      .*       *      .*      .*           0.799
## B9        *       *       *  0.341*               *       *           0.724
## B10               *       *  0.353*       *      .*       *           0.570
## B11       *      .*      .*      .*               *      .*           0.695
## B12       *               *  0.515*       *       *                   0.635
## B13       *       *          0.443*       *       *       *           0.722
## C1       .*       *       *       *       *       *      .*           0.735
## C2        *      .*       *       *       *       *      .*           0.443
## C3        *       *       *      .*       *       *       *           0.661
## C4                       .*       *       *       *       *           0.596
## C5        *       * -0.511*       *               *       *           0.452
## C6                *      .*      .*      .*       *                   0.505
## C7                *       *       *      .*       *      .*           0.427
## C8        *      .*      .*               *      .*      .*           0.360
## C9        *       *      .*       *       *       *       *           0.488
## C10                              .*       *       *       *           0.472
## D1        *       *       *       *       *      .*       *           0.384
## D2        *       *       *       *       *       *      .*           0.565
## D3        *      .*       *       *               *                   0.676
## D4        *      .*       *      .*       *       *       *           0.720
## D5        *       *       *       *              .*       *           0.492
## D6        *       *       *      .*       *       *       *           0.712
## D7                *      .*      .*       *       *      .*           0.458
## D8        *       *       *  0.348*       *               *           0.684
## D9                *      .*      .*       *       *       *           0.608
## D10               *      .*      .*       *       *       *           0.590
## E1        *      .*       *       *      .*       *       *           0.415
## E2        *       *              .* -0.313*       *       *           0.410
## E3        *       *       *      .*       *      .*      .*           0.565
## E4        *       *              .* -0.339*       *       *           0.456
## E5        *       *       *      .*               *      .*           0.598
## E6                *       *       *       *      .*                   0.528
## E7   0.305*       *       *      .*       *      .*       *           0.658
## E8        *       *               * -0.353*               *           0.527
## E9        *               *      .*       *                           0.830
## E10               *      .*       *      .*       *       *           0.783
## F1        *       *       *      .*       *      .*       *           0.639
## F2        *       *       *       *       *      .*       *           0.390
## F3        *               *      .*               *       *           0.820
## F4        *       *       *      .*       *       *                   0.338
## F5        *       *              .*       *       *       *           0.737
## F6                        *       *       *       *      .*           0.385
## F7                        *       *       *      .*       *           0.471
## F8        *       *      .*       *       *      .*       *           0.717
## F9                *       *       *       *       *      .*           0.382
## F10       *       *       *       *       *       *      .*           0.650
## G1        *      .*       *       *      .*       *       *           0.368
## G2        *      .*       *       *      .*       *       *           0.415
## G3        *      .*       *       *      .*       *      .*           0.562
## G4        *      .*       *       *       *       *       *           0.362
## G5        *      .*       *       *       *       *                   0.390
## G6        *      .*                       *       *       *           0.382
## G7        *      .*       *              .*       *                   0.509
## G8        *      .*      .*  0.349*      .*       *                   0.548
## G9               .*       *       *      .*       *      .*           0.457
## G10       *      .*      .*       *      .*       *       *           0.460
## H1        *       *       *  0.420*      .*       *       *           0.161
## H2        *       *       *  0.400*       *       *       *           0.688
## H3        *       *          0.402*      .*               *           0.191
## H4        *       *       *       *       *       *       *           0.872
## H5        *      .*      .*       *       *       *                   0.738
## H6        *      .*       *       *       *       *       *           0.719
## H7                *      .*       *       *       *       *           0.809
## H8                *       *  0.346*       *              .*           0.691
## H9       .*       *      .*  0.408*       *      .*       *           0.702
## H10       *      .*      .*      .*       *       *       *           0.828
## I1        *      .*      .*       *       *       *      .*           0.645
## I2        *       *       *       *       *              .*           0.578
## I3        *      .*       *       *      .*               *           0.680
## I4               .*       *       *      .*               *           0.359
## I5       .*       *       *       *      .*       *      .*           0.758
## I6        *       *       *      .*      .*               *           0.609
## I7               .*       *       *       *       *       *           0.448
## I8        *      .*       *       *      .*       *       *           0.353
## I9        *      .*       *       *       *       *                   0.466
## I10       *       *       *       *       *       *       *           0.659
## J1        *       *       *      .*      .*      .*                   0.650
## J2       .*       *       *  0.333*      .*      .*       *           0.536
## J3       .*       *              .*      .*      .*       *           0.590
## J4   0.305*       *       *      .*      .*      .*      .*           0.538
## J5        *               *      .*      .*       *       *           0.709
## J6                *       *       *       *       *       *           0.702
## J7        *       *               *       *      .*       *           0.629
## J8        *       *       *       *       *      .*      .*           0.581
## J9   0.830*       *              .*       *       *                   0.189
## J10  0.760*       *              .*       *       *                   0.288
## K1        *  0.668*       *       *      .*       *       *           0.392
## K2           0.377*       *       *      .*       *       *           0.546
## K3        *      .*       *       *      .*       *       *           0.455
## K4           0.494*       *       *      .*       *       *           0.532
## K5        *  0.511*       *      .*      .*       *       *           0.508
## K6        *  0.745*       *       *       *       *                   0.287
## K7        *  0.785*       *               *       *                   0.250
## K8        *  0.395*       *       *       *       *       *           0.799
## K9           0.649*       *       *       *       *                   0.443
## K10          0.517*       *      .*       *       *      .*           0.589
## L1        *       *       *       *               *      .*           0.553
## L2        *                       *       *              .*           0.529
## L3        *       *      .*      .*       *              .*           0.474
## L4        *       *       *       *       *       *      .*           0.555
## L5                *               *       *       *      .*           0.643
## L6        *       *      .*       *       *       *      .*           0.692
## L7                *               *       *              .*           0.620
## L8        *       *      .*       *       *       *       *           0.688
## L9        *       *  0.306*                       *       *           0.577
## L10       *       *       *       *       *       *       *           0.555
## M1        *       *       *  0.343*      .*       *      .*           0.654
## M2        *       *       *      .*       *       *                   0.661
## M3        *       *       *  0.418*       *               *           0.679
## M4        *      .*       *  0.429*       *       *       *           0.626
## M5        *       *       *       *       *      .*       *           0.764
## M6        *       *       *  0.611*       *       *       *           0.590
## M7        *       *       *  0.505*       *               *           0.703
## M8        *               *  0.625*       *       *       *           0.585
## M9        *               *  0.534*       *       *      .*           0.625
## M10       *       *          0.544*       *       *      .*           0.640
## N1        *      .*               *  0.545*       *                   0.423
## N2               .*               *  0.494*       *       *           0.605
## N3        *       *       *      .*  0.651*               *           0.480
## N4        *      .*       *       *  0.613*       *       *           0.415
## N5                *       *      .*  0.423*               *           0.755
## N6                *               *  0.608*       *       *           0.550
## N7        *      .*       *      .*  0.505*       *       *           0.620
## N8        *       *       *       *  0.352*              .*           0.509
## N9        *       *       *      .*      .*       *                   0.595
## N10       *      .*       *      .*  0.388*       *      .*           0.712
## O1        *               *               *  0.313*      .*           0.699
## O2        *       *       *      .*       *  0.584*      .*           0.511
## O3                *       *       *       *  0.489*      .*           0.597
## O4        *       *      .*      .*       *  0.414*      .*           0.646
## O5        *               *      .*       *  0.361*      .*           0.676
## O6        *       *      .*               *  0.458*       *           0.724
## O7        *              .*                  0.497*       *           0.667
## O8        *       *               *       *  0.696*      .*           0.433
## O9        *       *       *       *          0.573*      .*           0.611
## O10      .*       *       *       *       *  0.441*      .*           0.595
## P1                *  0.507*      .*      .*              .*           0.308
## P2        *       *  0.485*      .*       *       *       *           0.384
## P3                *      .*       *       *       *  0.372*           0.670
## P4        *       *      .*               *       *  0.348*           0.649
## P5                        *      .*       *       *      .*           0.674
## P6        *       *      .*      .*       *              .*           0.756
## P7        *       *       *       *       *       *  0.402*           0.727
## P8        *       *  0.575*       *       *       *      .*           0.405
## P9                *      .*                               *           0.698
## P10       *       *      .*       *       *       *       *           0.650
##       communalities
## A1            0.496
## A2            0.511
## A3            0.450
## A4            0.387
## A5            0.512
## A6            0.463
## A7            0.379
## A8            0.201
## A9            0.452
## A10           0.225
## B1            0.335
## B2            0.333
## B3            0.349
## B4            0.364
## B5            0.292
## B6            0.323
## B7            0.297
## B8            0.201
## B9            0.276
## B10           0.430
## B11           0.305
## B12           0.365
## B13           0.278
## C1            0.265
## C2            0.557
## C3            0.339
## C4            0.404
## C5            0.548
## C6            0.495
## C7            0.573
## C8            0.640
## C9            0.512
## C10           0.528
## D1            0.616
## D2            0.435
## D3            0.324
## D4            0.280
## D5            0.508
## D6            0.288
## D7            0.542
## D8            0.316
## D9            0.392
## D10           0.410
## E1            0.585
## E2            0.590
## E3            0.435
## E4            0.544
## E5            0.402
## E6            0.472
## E7            0.342
## E8            0.473
## E9            0.170
## E10           0.217
## F1            0.361
## F2            0.610
## F3            0.180
## F4            0.662
## F5            0.263
## F6            0.615
## F7            0.529
## F8            0.283
## F9            0.618
## F10           0.350
## G1            0.632
## G2            0.585
## G3            0.438
## G4            0.638
## G5            0.610
## G6            0.618
## G7            0.491
## G8            0.452
## G9            0.543
## G10           0.540
## H1            0.839
## H2            0.312
## H3            0.809
## H4            0.128
## H5            0.262
## H6            0.281
## H7            0.191
## H8            0.309
## H9            0.298
## H10           0.172
## I1            0.355
## I2            0.422
## I3            0.320
## I4            0.641
## I5            0.242
## I6            0.391
## I7            0.552
## I8            0.647
## I9            0.534
## I10           0.341
## J1            0.350
## J2            0.464
## J3            0.410
## J4            0.462
## J5            0.291
## J6            0.298
## J7            0.371
## J8            0.419
## J9            0.811
## J10           0.712
## K1            0.608
## K2            0.454
## K3            0.545
## K4            0.468
## K5            0.492
## K6            0.713
## K7            0.750
## K8            0.201
## K9            0.557
## K10           0.411
## L1            0.447
## L2            0.471
## L3            0.526
## L4            0.445
## L5            0.357
## L6            0.308
## L7            0.380
## L8            0.312
## L9            0.423
## L10           0.445
## M1            0.346
## M2            0.339
## M3            0.321
## M4            0.374
## M5            0.236
## M6            0.410
## M7            0.297
## M8            0.415
## M9            0.375
## M10           0.360
## N1            0.577
## N2            0.395
## N3            0.520
## N4            0.585
## N5            0.245
## N6            0.450
## N7            0.380
## N8            0.491
## N9            0.405
## N10           0.288
## O1            0.301
## O2            0.489
## O3            0.403
## O4            0.354
## O5            0.324
## O6            0.276
## O7            0.333
## O8            0.567
## O9            0.389
## O10           0.405
## P1            0.692
## P2            0.616
## P3            0.330
## P4            0.351
## P5            0.326
## P6            0.244
## P7            0.273
## P8            0.595
## P9            0.302
## P10           0.350
## 
##                                f5    f6   f13    f4    f1    f9   f11    f2
## Sum of sq (ortho) loadings 11.086 8.592 6.852 5.226 5.170 4.574 4.296 4.016
## Proportion of total         0.160 0.124 0.099 0.076 0.075 0.066 0.062 0.058
## Proportion var              0.068 0.053 0.042 0.032 0.032 0.028 0.026 0.025
## Cumulative var              0.068 0.121 0.163 0.195 0.227 0.255 0.281 0.306
##                              f14   f15    f8   f12   f16    f3   f10    f7
## Sum of sq (ortho) loadings 3.992 3.378 2.728 2.111 1.894 1.796 1.769 1.636
## Proportion of total        0.058 0.049 0.039 0.031 0.027 0.026 0.026 0.024
## Proportion var             0.024 0.021 0.017 0.013 0.012 0.011 0.011 0.010
## Cumulative var             0.330 0.351 0.368 0.381 0.392 0.403 0.414 0.424
##                             total
## Sum of sq (ortho) loadings 69.117
## Proportion of total         1.000
## Proportion var              0.424
## Cumulative var              0.424

Our model fit is quite good, but that’s not too surprising since the data determines the model, rather than the other way around. I extracted the loadings into a data frame, then keep only the highest loading for each item. The gives me a dataset matching the standardized loading output from above. This was a bit easier to work with than scrolling through the output.

StLoadings16 <- as_tibble(EFA$loadings$nf16)

StLoadings16 <- StLoadings16 %>%
  mutate(Item = ItemList) %>%
  group_by(Item) %>%
  pivot_longer(cols = f1:f16,
               names_to = "factor",
               values_to = "loading") %>%
  filter(loading == max(loading))

StLoadings16 <- StLoadings16 %>%
  pivot_wider(id_cols = Item,
              names_from = factor,
              values_from = loading,
              values_fill = NA)

The results are, of course, quite a mess and somewhat hard to read. But I went through it and here’s the scales that had factors all to themselves: F, K, N, and O (or Dutifulness, Reserve, Introversion, and Orderliness). Some scales split across a couple factors, or shared factors with other scales. Our worst offenders were H, J, and L (Sensitivity, Imagination, and Anxiety). Each of these 3 scales split into 4 factors (with a total of 8 factors among them, since there was a bit of overlap); some of those factors were made up of a few items. H4, our worst item of the bunch, didn't load with any of the other H items. So we seem to have decent support for 4 of Cattell’s factors, little to no support for 3, and mediocre support for the rest.

I created a chart color-coding where items fell. It’s still messy and a bit overwhelming. But it gives you some idea of what scales seem to hold together and which ones fall apart.

Since this post is already overlong, I won’t be going through the 5 and 6 factor models, but look for them in a future post! Next time, let’s subject our 16 Factor Measure to Rasch measurement!

No comments:

Post a Comment