In my last post, I talked through how to clean the dataset of careless responders, responders with too much missing data, and people older than time itself. Today, I’ll go through a classical test theory approach to psychometric analysis. If you’d like to go through the cleaning process with the original dataset, I recommended going through the last post. If you want, you can even change the cutoffs I used for various flags and see how it changes the dataset and the results going forward.
If, on the other hand, you want to start with the exact dataset I’m using without going through the full cleaning process on your own, you can download my clean copy here. Just remember that inspecting and cleaning data is always part of the process when analyzing a dataset.
library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.2.1 ✔ readr 2.2.0
## ✔ forcats 1.0.1 ✔ stringr 1.6.0
## ✔ ggplot2 4.0.3 ✔ tibble 3.3.1
## ✔ lubridate 1.9.5 ✔ tidyr 1.3.2
## ✔ purrr 1.2.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
PF16 <- read_csv("Cleaned PF16 Data.csv")
## Rows: 46427 Columns: 171
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (2): country, age_group
## dbl (169): A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, B1, B2, B3, B4, B5, B6, ...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Classical test theory approaches are very much “what you see is what you get.” What I mean by that is, the range of abilities (or more specifically to a measure of traits, the range of how much of each trait people possess) is defined entirely by the sample we selected, and how well we can measure those traits is defined entirely by the items we used (though we recognize there are other, potentially better items). We can make assumptions that our sample is well representative of the full range of presence of these traits, but subjecting a different sample to this measure will likely produce very different results. This drawback is one of the reasons psychometricians gravitate to item response theory models, like Rasch, but the benefit of using CTT is that there are very few statistical assumptions we need to meet to use these analyses. And that sometimes makes CTT a better approach; it really depends on context.
I’ll talk more about the assumptions of IRT in my next post. But the key assumption in CTT is that a person’s observed score is the sum of their true score (their underlying ability or how much of the trait they actually posses - what we’re trying to measure) and error (caused by a variety of factors that get in the way of obtaining that true score, like distractions in the testing environment, a poorly written item, lack of attention by the respondent, and so on). We want that error to be 0, but we know in actual practice, it probably isn’t. So we work to try to minimize that error: we use multiple items to measure the same trait, hoping that will help cancel out some the error, and we conduct analyses that help us measure that error and separate it from the true score. We also use those analyses to identify the items that increase measurement error, so that we can edit or remove them.
Now, in some cases IRT has influenced CTT. For instance, someone who takes a purely CTT approach to psychometrics would treat rating scales as continuous variables. In fact, CTT literature often states that rating scales with at least 5 points can be treated as continuous regardless. IRT would consider rating scales to be ordinal (that is, until analysis converts them). Parametric statistics, like mean, standard deviation, and Pearson’s correlation coefficient (stats that assume normal distributions of the variable) are built specifically for continuous variables, whereas ordinal variables would require non-parametric approaches, like median, interquartile range, and Spearman’s rank-order correlation coefficient, which don’t make any assumptions about distributions. In my case, I’ll sometimes treat raw ratings scales as continuous, if the parametric test I’m using is robust with data that doesn’t quite meet assumptions, and sometimes as ordinal.
First, I want to reverse code the negatively worded items. Then we can start generating descriptive statistics by item and scale. The number of negatively worded items varies by scale, so we’ll need to specify the range for each scale, then use that list to reverse code the correct variables. As I did in the last post, I’m using mutate across, but this time we want it to use all items listed in ‘reverse’. Subtracting each rating from 6 (number of options + 1) will reverse the scale, so now 1 = 5, 2 = 4, and so on. 3, our midpoint ‘neutral’ option will remain unchanged.
reverse <- list(paste0("A", 8:10), paste0("B", 9:13), paste0("C", 6:10),
paste0("D", 7:10), paste0("E", 7:10), paste0("F", 6:10),
paste0("G", 6:10), paste0("H", 7:10), paste0("I", 7:10),
paste0("J", 8:10), paste0("K", 6:10), paste0("L", 8:10),
paste0("M", 6:10), paste0("N", 8:10), paste0("O", 6:10),
paste0("P", 8:10))
reverse <- unlist(reverse)
PF16R <- PF16 %>%
mutate(across(all_of(reverse), ~ 6 - .x))
I’d also like to have an index dataframe that contains the full name of each scale and how many items each scale contains. When I start generating statistics at the scale-level, I can add to this dataframe.
Scales <- tibble(Code = c(toupper(letters[1:16])),
Name = c("Warmth",
"Intellect",
"Emotional Stability",
"Assertiveness",
"Gregariousness",
"Dutifulness",
"Friendliness",
"Sensitivity",
"Distrust",
"Imagination",
"Reserve",
"Anxiety",
"Complexity",
"Introversion",
"Orderliness",
"Emotionality"),
Items = c(10, 13, rep(10, 14)))
I’ll start by generating means and standard deviations for each item. (This is an instance of treating the rating scales as continuous.) Since we want to aggregate, we’ll be using summarise instead of mutate, but we can use summarise in very similar ways as mutate: such as summarizing across a range of variables.
ByItem <- PF16R %>%
summarise(across(A1:P10, list(
Mean = ~mean(., na.rm = TRUE),
SD = ~sd(.x, na.rm = TRUE))
)) %>%
pivot_longer(everything(), names_to = c("Column", ".value"), names_sep = "_")
This first step of the code above creates a dataset with 1 row and 2 columns per item (163 * 2 = 326 columns), one with the item mean and the other with standard deviation. The default is to name each column as ColumnName_Function, so we’d have A1_Mean, A1_SD, A2_Mean, etc. Pivot longer moves the column names to become row names, and “.value”, combined with names_sep = “_” does two things: it removes _Mean and _SD from the columns (leaving 163 rows, 1 for each item), and instead creates 2 columns next to the list of items names, one for Mean and one for SD. We can then add additional summary data to this file.
One of the big metrics used in CTT analyses is p-value, typically the proportion of people responding to an item correctly or, if the measure is a checklist, the proportion of people endorsing that item. But this measure doesn’t have incorrect and correct answers, nor is it a checklist. We can still generate a p-value-like measure, by dividing the mean rating by the number of rating scale categories (5). Instead of measuring percentage correct, it measures degree of aggreement with the statment (on average). By itself, it isn’t extremely useful, but we can examine it alongside other analyses to flag items that may not fit as well on the measure.
ByItem <- ByItem %>%
mutate(pVal = Mean/5)
Next, I’ll add counts selecting each rating scale option to the ByItem data. Since the ByItem data is formatted as a row for each item, I’ll create my counts of categories using pivot functions to put it in the same format. In fact, for the next few analyses, I’ll want a set that pivots my item response data into long form, so I’ll first create that set.
This makes joining the two sets together very easy. I’ve also counted the number of people who skipped each item, though I didn’t include them when I calculated proportions selecting each option.
Items <- PF16R %>%
pivot_longer(
cols = A1:P10,
names_to = "Item",
values_to = "Rating"
) %>%
mutate(Code = substr(Item, 1, 1)) %>%
left_join(Scales, by = "Code")
CatCounts <- Items %>%
count(Item, Rating) %>%
complete(Item, Rating = 1:5, fill = list(n = 0)) %>%
group_by(Item) %>%
pivot_wider(names_from = Rating, values_from = n)
ByItem <- ByItem %>%
left_join(CatCounts, by = c("Column" = "Item"))
ByItem <- ByItem %>%
group_by(Column) %>%
mutate(TotalValid = sum(across(`1`:`5`)),
StronglyDis = `1`/TotalValid,
Disagree = `2`/TotalValid,
Neither = `3`/TotalValid,
Agree = `4`/TotalValid,
StronglyAgr = `5`/TotalValid) %>%
ungroup()
I’d also like to look at how well each item correlates to total scale score on the remaining items. For knowledge tests and checklists, this would be a point biserial, the correlation between a binary variable (0/1, or incorrect/correct) and a continuous variable (score on remaining items). But for a rating scale measure, we’ll be using polyserial correlations, the correlation between an ordinal variable (1-5 on the rating scale) and a continuous variable (score on the remaining items). Since some respondents are missing an item from a scale, we’ll compute total scores as averages of their ratings. Once we have our scores, we can use the CTT package to compute polyserial correlations by item.
library(CTT)
Polyserials <- Items %>%
group_by(ID, Code) %>%
mutate(Total = sum(Rating),
Count = sum(!is.na(Rating)),
Score = (Total - Rating)/(Count - 1)) %>%
group_by(Item) %>%
summarise(disc = polyserial(Score, Rating, ml = FALSE))
ByItem <- ByItem %>%
left_join(Polyserials, by = c("Column" = "Item"))
I called our polyserial variable “disc” for “discrimination”, a psychometric term for how well the item differentiates between people who are high in the trait being measured and those low in the trait.
Lastly, I want to compute our scale reliabilities, as measured by Cronbach’s alpha. This measures what is called internal consistency by determining how much variance is attributable to true score variance - that is, how much people naturally vary in the traits being measured - by separating the variance shared by items (covariance) from total variance, which would include both true scores and measurement error. Alpha ranges from 0 to 1, and equals the proportion of variance attributable to true scores; theoretically, an alpha of 1 means the scale has no measurement error. In practice, it typically means redundancy in items, multiple items that address the exact same concept within a scale. For example, if I had a bunch of items measuring how much you enjoy reading books as well as negatively worded items on how much you don’t enjoy reading books. The scale would probably have a high alpha, but it’s not giving you a lot of useful information, especially if book reading is only one aspect of the concept I’m measuring.
There are multiple R packages that compute alpha, but we'd need to reformat the data. However, the formula for alpha is pretty simple:
I can easily compute the required variances, add them to my Scales frame (which already includes number of items), and compute alpha for each scale. For individual item variance, I already have standard deviation, so I just need to square that value to get variance. Generating total scores is a bit more involved: I’ve retained respondents with up to 2 missing values, but I want a total score on the same scale as respondents without missing values, so I’ll create a corrected total score - the average score of the ratings they provided (sum divided by number of items completed), multiplied by the number of items on the full scale. Since I wanted to score my sample anyway, I created a new dataframe of individual scores by scale, then aggregated for the variance I need to generate my alphas.
ByItem <- ByItem %>%
mutate(Variance = SD^2)
Scores <- Items %>%
ungroup() %>%
group_by(ID, Code) %>%
summarise(Total = sum(Rating, na.rm = TRUE),
NItems = sum(!is.na(Rating))) %>%
ungroup() %>%
left_join(Scales, by = "Code") %>%
mutate(Score = Total/NItems,
CorTotal = Score*Items)
## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by ID and Code.
## ℹ Output is grouped by ID.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(ID, Code))` for per-operation grouping
## (`?dplyr::dplyr_by`) instead.
ByScale <- Scores %>%
group_by(Code) %>%
summarise(Variance = var(CorTotal, na.rm = TRUE))
ByScale2 <- ByItem %>%
ungroup() %>%
mutate(Code = substr(Column, 1, 1)) %>%
group_by(Code) %>%
summarise(EItemVar = sum(Variance))
Scales <- Scales %>%
left_join(ByScale, by = "Code") %>%
left_join(ByScale2, by = "Code")
Scales <- Scales %>%
mutate(Alpha = (Items/(Items-1)) * (1 - (EItemVar/Variance)))
Now let’s go through these item-level and scale-level indices and start flagging items that could be problematic.
I also have some information on the scales’ alphas from the validation study, where 501 respondents completed both the original 16PF and the IPIP version I’m using here. I can easily pull the table from the webpage, though it needs a bit of cleaning to be useful.
library(rvest)
##
## Attaching package: 'rvest'
## The following object is masked from 'package:readr':
##
## guess_encoding
url <- "https://ipip.ori.org/new16PFTable.htm"
page <- read_html(url)
all_tables <- html_table(page)
comptable <- all_tables[[2]]
comptable <- comptable %>%
rename(IPIPNames = X1,
PF16Names = X2,
IPIPItems = X3,
PF16Items = X4,
IPIPCor = X5,
PF16Cor = X6,
IPIPAlpha = X7,
PF16Alpha = X8,
Correlation = X9)
comptable <- comptable[c(5:20), ]
ComparisonAlpha <- comptable %>%
mutate(IPIPAlpha = as.numeric(IPIPAlpha)) %>%
select(IPIPNames, IPIPAlpha)
Scales <- Scales %>%
left_join(ComparisonAlpha, by = c("Name" = "IPIPNames"))
library(ggthemes)
Scales %>%
pivot_longer(
cols = c(Alpha, IPIPAlpha),
names_to = "source",
values_to = "alpha"
) %>%
ggplot(aes(x = Code, y = alpha, color = source)) +
geom_point() +
labs(y = "Alpha", color = "source") +
theme_economist_white()
For 11 of the scales, this sample produced higher alphas than the original validation sample. But what is a good alpha? The context of the measure is important here. If it’s being used to make high stakes decisions, like whether someone obtains a license or certificate or gets diagnosed with a condition, reliability should be high, preferably in the 0.9 range. But for a lower stakes measure, like this one, we can be a bit more generous in what alphas are acceptable. I’ll consider 0.90 and greater to be excellent, 0.80 to 0.89 good, 0.70 to 0.79 acceptable, and less than 0.70 questionable.
Scales <- Scales %>%
mutate(AlphaCat = cut(Alpha, breaks = c(0, 0.7, 0.8, 0.9, Inf),
labels = c("Questionable",
"Acceptable",
"Good",
"Excellent"),
right = FALSE))
Scales %>%
ggplot(aes(x = Code, y = Alpha, colour = AlphaCat)) +
geom_point() +
theme_economist_white()
Overall, our alphas are fine, with the one exception being scale H, sensitivity, which has an alpha of 0.67. Perhaps our item analysis will help us determine the issue with this scale.
Let’s start with the p-value-like measure. It’s measuring average degree of agreement with the item. Low agreement could just mean the trait is rare. But if most of the items are easy to endorse in a scale, and there are some exceptions, those items may not be measuring the trait well. Let’s look at the items with a “p-value” of less than 0.5.
ByItem %>%
filter(pVal < 0.5) %>%
select(Column, Mean, pVal, StronglyDis:StronglyAgr)
## # A tibble: 7 × 8
## Column Mean pVal StronglyDis Disagree Neither Agree StronglyAgr
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 F8 2.43 0.487 0.252 0.387 0.135 0.126 0.0992
## 2 H4 2.27 0.455 0.286 0.384 0.151 0.126 0.0528
## 3 I6 2.11 0.422 0.344 0.362 0.167 0.0940 0.0332
## 4 K10 2.44 0.489 0.178 0.453 0.168 0.150 0.0515
## 5 N8 2.35 0.471 0.170 0.492 0.195 0.103 0.0401
## 6 P9 2.32 0.465 0.202 0.479 0.151 0.129 0.0389
## 7 P10 2.50 0.499 0.166 0.404 0.236 0.157 0.0374
7 items showed low agreement, with average rating in the “disagree” range, and disagree was the most frequent rating. So let’s add our first flag to the item data.
ByItem <- ByItem %>%
mutate(Flag1 = ifelse(pVal < 0.50, "Low P", ""))
Next is discrimination, how well the item differentiates between people high and low in the trait, as measured by polyserials. We want all of our polyserials to be positive - and we would expect that since we reverse-coded the negatively worded items - so let’s see if any are 0 or negative. Since we probably don’t have any that are exactly 0, let’s also include discrimination that would essentialy round down to 0.
ByItem %>%
filter(disc <= 0.004) %>%
select(Column, disc)
## # A tibble: 0 × 2
## # ℹ 2 variables: Column <chr>, disc <dbl>
None of the items are negative or 0, which is good. But that doesn’t mean they provide meaningful information. So let’s look at items with positive, but low discrimination. We’ll use a cutoff of 0.20, or around 10% of the variance in scores.
ByItem %>%
filter(disc < 0.20) %>%
select(Column, disc)
## # A tibble: 1 × 2
## Column disc
## <chr> <dbl>
## 1 H4 0.0523
Only one item has low discrimination: item 4 on the Sensitivity scale, “Don’t like action movies”. This scale mostly deals with enjoyment of the arts, such as movies, books, poetry, and dance. Apparently sensitive people aren’t supposed to like action movies? But since this item has minimal correlation with sensitivity, apparently you can still be a sensitive artist who loves Die Hard.
ByItem <- ByItem %>%
mutate(Flag2 = ifelse(disc < .20, "Low Discrimination", ""))
Let’s also look at rating scale function. Respondents have 5 options to choose from for each statement: Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree. There’s disagreement in the survey literature of whether agreement scales should provide a neutral option or not, with strong arguments either way. But with psychometrics, we typically make these types of decisions empirically - are people using the neutral category and are they using it as intended? If so, we keep the category. If not, we should consider revising the rating scale.
Remember, we have 163 items, which gives us a lot of graphs to look at. But we can at least make it less overwhelming by creating a grid of each scale with all 10 (or 13, for scale B, intellect) items. I’d also like the items to appear in order, and since the Item in the data is alphanumeric, the 10+ item in the scale will appear after 1 instead of 9. So I’ll make a new numeric variable of item number. Then I’ll write a function to create the charts (one for each scale using the filter function, with a grid using facet_wrap of Item Number), and lastly, apply them to each scale. The map function allows me use a list of scales and apply them one-by-one to the plotting function. Note that the list is called scale_names (plural) and the function uses scale_name (singular).
Items <- Items %>%
mutate(Item_Num = substr(Item, 2, 3),
Item_Num = as.numeric(Item_Num))
histogram_scale <- function(scale_name) {
Items %>%
filter(Name == scale_name) %>%
ggplot(aes(factor(Rating, levels = 1:5))) +
geom_bar() +
scale_x_discrete(drop = FALSE) +
facet_wrap(vars(Item_Num)) +
labs(title = scale_name, x = "Rating", y = "Count") +
theme(plot.title = element_text(hjust = 0.5))
}
scale_names <- unique(Items$Name)
scale_histograms <- map(scale_names, histogram_scale) %>% set_names(scale_names)
Now I have a large list (scale_plots) containing the data for each plot, and I can view each plot by referencing it. For instance, to view scale B, Intellect:
scale_histograms$Intellect
Across all items and scales, all response options are being used, though in some cases strongly disagree and strongly agree aren’t heavily used. But the bigger question is whether the rating scale is being used as intended. That is, are people lower in the trait more likely to select Strongly Disagree, middle of the trait more likely to select Neither Agree nor Disagree, and so on? Our polyserial correlations tell us some of this, and I’ll use some of the same code I did to compute those correlations, but we can dig deeper into scale function by visualizing that relationship. Specifically, I’d like to chart average total score (excluding the current item) for each level of the rating scale. If the categories are functioning properly, we should see an increase in average score as we go up the rating scale. To create my groups of scale graphs, I’ll do something similar as I did above.
ScaleFunction <- Items %>%
group_by(ID, Code) %>%
mutate(Total = sum(Rating),
Count = sum(!is.na(Rating)),
Score = (Total - Rating)/(Count - 1)) %>%
ungroup() %>%
group_by(Item, Rating) %>%
summarise(Code = first(Code),
Name = first(Name),
Item_Num = first(Item_Num),
Score = mean(Score, na.rm = TRUE)) %>%
filter(!is.na(Rating)) %>%
ungroup()
## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by Item and Rating.
## ℹ Output is grouped by Item.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(Item, Rating))` for per-operation grouping
## (`?dplyr::dplyr_by`) instead.
plot_scalefunction <- function(scale_name) {
ScaleFunction %>%
filter(Name == scale_name) %>%
ggplot(aes(Rating, Score)) +
geom_point() +
geom_smooth(method = "lm") +
facet_wrap(vars(Item_Num)) +
labs(title = scale_name, x = "Rating", y = "Average Total Score") +
theme(plot.title = element_text(hjust = 0.5))
}
scale_scalefunctions <- map(scale_names, plot_scalefunction) %>% set_names(scale_names)
scale_scalefunctions$Intellect
## `geom_smooth()` using formula = 'y ~ x'
The steepness of the slope is, of course, the polyserial correlation for that item, so we could probably learn a lot from examining our discrimination indices. But these figures show where a point on a rating scale falls out of line - perhaps equal to or even lower than the point below it. Going through the figures scale by scale, the following variables show signs of scale dysfunction: A10, B1, B4, B8, D6, D8, E7, E9, F3, H4, H9, H10, I5, J9, J10, K8, L8, M3, M6, M7, M8, M9, M10, N7.
Flag3 <- list("A10", "B1", "B4", "B8", "D6", "D8", "E7", "E9", "F3", "H4", "H9", "H10", "I5", "J9", "J10", "K8", "L8", "M3", "M6", "M7", "M8", "M9", "M10", "N7")
ByItem <- ByItem %>%
mutate(Flag3 = ifelse(Column %in% Flag3, "Scale Dysfunction", ""))
Now it’s time to conduct our CFA. This isn’t necessarily required, especially if you’re conducting item and scale analysis on an established measure. It’s more necessary if you’re conducting the validation study of a measure. But let’s go ahead and conduct one to be thorough. I’ve blogged about using lavaan for CFA before, so check out that post for a bit more exposition. I just need to specify my model and feed it into the cfa function.
library(lavaan)
## This is lavaan 0.7-2
## lavaan is FREE software! Please report any bugs.
Factor16 <- '
"Warmth" =~ A1 + A2 + A3 + A4 + A5 + A6 + A7 + A8 + A9 + A10
"Intellect" =~ B1 + B2 + B3 + B4 + B5 + B6 + B7 + B8 + B9 + B10 + B11 + B12 + B13
"EmotSta" =~ C1 + C2 + C3 + C4 + C5 + C6 + C7 + C8 + C9 + C10
"Assert" =~ D1 + D2 + D3 + D4 + D5 + D6 + D7 + D8 + D9 + D10
"Gregar" =~ E1 + E2 + E3 + E4 + E5 + E6 + E7 + E8 + E9 + E10
"Duti" =~ F1 + F2 + F3 + F4 + F5 + F6 + F7 + F8 + F9 + F10
"Friend" =~ G1 + G2 + G3 + G4 + G5 + G6 + G7 + G8 + G9 + G10
"Sens" =~ H1 + H2 + H3 + H4 + H5 + H6 + H7 + H8 + H9 + H10
"Dist" =~ I1 + I2 + I3 + I4 + I5 + I6 + I7 + I8 + I9 + I10
"Imag" =~ J1 + J2 + J3 + J4 + J5 + J6 + J7 + J8 + J9 + J10
"Reser" =~ K1 + K2 + K3 + K4 + K5 + K6 + K7 + K8 + K9 + K10
"Anx" =~ L1 + L2 + L3 + L4 + L5 + L6 + L7 + L8 + L9 + L10
"Complex" =~ M1 + M2 + M3 + M4 + M5 + M6 + M7 + M8 + M9 + M10
"Intr" =~ N1 + N2 + N3 + N4 + N5 + N6 + N7 + N8 + N9 + N10
"Order" =~ O1 + O2 + O3 + O4 + O5 + O6 + O7 + O8 + O9 + O10
"Emot" =~ P1 + P2 + P3 + P4 + P5 + P6 + P7 + P8 + P9 + P10
'
CFA16 <- cfa(model = Factor16, data = PF16R)
summary(CFA16, fit.measures = TRUE)
## lavaan 0.7-2 ended normally after 229 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 446
##
## Used Total
## Number of observations 35127 46427
##
## Model Test User Model:
##
## Test statistic 825227.500
## Degrees of freedom 12920
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 2538331.585
## Degrees of freedom 13203
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.678
## Tucker-Lewis Index (TLI) 0.671
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -7709410.175
## Loglikelihood unrestricted model (H1) -7296796.425
##
## Akaike (AIC) 15419712.350
## Bayesian (BIC) 15423488.509
## Sample-size adjusted Bayesian (SABIC) 15422071.122
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.042
## 90 Percent confidence interval - lower 0.042
## 90 Percent confidence interval - upper 0.042
## P-value H_0: RMSEA <= 0.050 1.000
## P-value H_0: RMSEA >= 0.080 0.000
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.074
##
## Goodness of Fit Index:
##
## Goodness of Fit Index (GFI) 0.664
## 90 Percent confidence interval - lower 0.663
## 90 Percent confidence interval - upper 0.664
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## Warmth =~
## A1 1.000
## A2 0.907 0.008 116.796 0.000
## A3 0.799 0.008 97.549 0.000
## A4 0.819 0.008 104.850 0.000
## A5 0.854 0.007 114.200 0.000
## A6 0.788 0.007 108.222 0.000
## A7 0.730 0.007 100.615 0.000
## A8 0.620 0.009 70.406 0.000
## A9 0.864 0.008 106.308 0.000
## A10 0.501 0.008 60.932 0.000
## Intellect =~
## B1 1.000
## B2 1.107 0.014 76.995 0.000
## B3 0.810 0.011 72.482 0.000
## B4 0.887 0.010 85.535 0.000
## B5 0.993 0.012 80.909 0.000
## B6 0.760 0.013 59.640 0.000
## B7 0.723 0.013 54.657 0.000
## B8 0.692 0.012 57.775 0.000
## B9 1.002 0.016 62.204 0.000
## B10 1.190 0.016 75.870 0.000
## B11 0.826 0.016 52.107 0.000
## B12 1.136 0.014 80.092 0.000
## B13 1.112 0.016 69.476 0.000
## EmotSta =~
## C1 1.000
## C2 1.294 0.015 84.880 0.000
## C3 0.892 0.012 74.627 0.000
## C4 1.007 0.014 74.003 0.000
## C5 1.107 0.015 74.148 0.000
## C6 1.401 0.017 82.245 0.000
## C7 1.585 0.018 89.142 0.000
## C8 1.568 0.018 87.897 0.000
## C9 1.407 0.016 85.686 0.000
## C10 1.351 0.016 86.058 0.000
## Assert =~
## D1 1.000
## D2 0.788 0.007 107.111 0.000
## D3 0.595 0.007 82.615 0.000
## D4 0.587 0.007 80.777 0.000
## D5 0.832 0.006 130.873 0.000
## D6 0.506 0.006 83.606 0.000
## D7 0.953 0.007 134.242 0.000
## D8 0.494 0.006 79.039 0.000
## D9 0.787 0.007 113.375 0.000
## D10 0.779 0.008 97.128 0.000
## Gregar =~
## E1 1.000
## E2 1.195 0.008 147.997 0.000
## E3 0.502 0.007 76.939 0.000
## E4 1.075 0.008 135.968 0.000
## E5 0.389 0.005 72.376 0.000
## E6 0.737 0.008 92.805 0.000
## E7 0.243 0.007 34.327 0.000
## E8 1.009 0.008 126.263 0.000
## E9 0.182 0.007 27.905 0.000
## E10 0.498 0.008 64.492 0.000
## Duti =~
## F1 1.000
## F2 1.281 0.014 94.733 0.000
## F3 0.790 0.016 48.169 0.000
## F4 1.526 0.016 97.453 0.000
## F5 0.958 0.015 64.289 0.000
## F6 1.485 0.015 96.367 0.000
## F7 1.357 0.015 90.393 0.000
## F8 0.716 0.014 52.375 0.000
## F9 1.489 0.015 96.937 0.000
## F10 0.864 0.012 70.283 0.000
## Friend =~
## G1 1.000
## G2 1.104 0.008 146.184 0.000
## G3 0.850 0.008 111.975 0.000
## G4 1.122 0.007 158.342 0.000
## G5 1.049 0.007 157.691 0.000
## G6 1.156 0.007 155.565 0.000
## G7 1.003 0.007 135.526 0.000
## G8 0.709 0.007 99.902 0.000
## G9 1.007 0.007 137.614 0.000
## G10 0.939 0.007 134.239 0.000
## Sens =~
## H1 1.000
## H2 0.436 0.005 89.926 0.000
## H3 1.068 0.005 203.172 0.000
## H4 0.072 0.006 11.678 0.000
## H5 0.155 0.007 21.820 0.000
## H6 0.212 0.006 36.384 0.000
## H7 0.172 0.007 25.600 0.000
## H8 0.397 0.006 63.360 0.000
## H9 0.392 0.005 77.523 0.000
## H10 0.158 0.006 26.655 0.000
## Dist =~
## I1 1.000
## I2 1.033 0.013 81.293 0.000
## I3 0.869 0.012 74.463 0.000
## I4 1.499 0.015 97.554 0.000
## I5 0.780 0.012 67.351 0.000
## I6 1.020 0.013 80.370 0.000
## I7 1.152 0.013 90.327 0.000
## I8 1.405 0.015 96.341 0.000
## I9 1.124 0.012 91.269 0.000
## I10 0.922 0.012 78.039 0.000
## Imag =~
## J1 1.000
## J2 1.328 0.014 91.827 0.000
## J3 1.252 0.015 85.968 0.000
## J4 1.252 0.014 90.596 0.000
## J5 0.673 0.011 60.718 0.000
## J6 0.659 0.011 58.507 0.000
## J7 0.647 0.011 57.134 0.000
## J8 0.590 0.011 53.147 0.000
## J9 1.144 0.014 80.221 0.000
## J10 1.058 0.014 77.027 0.000
## Reser =~
## K1 1.000
## K2 0.833 0.007 113.944 0.000
## K3 0.767 0.007 103.698 0.000
## K4 0.776 0.007 111.451 0.000
## K5 0.798 0.007 121.476 0.000
## K6 1.128 0.007 164.339 0.000
## K7 1.087 0.007 164.699 0.000
## K8 0.516 0.007 72.865 0.000
## K9 0.940 0.007 134.376 0.000
## K10 0.751 0.006 117.066 0.000
## Anx =~
## L1 1.000
## L2 0.958 0.009 106.417 0.000
## L3 1.085 0.010 111.895 0.000
## L4 0.923 0.008 108.930 0.000
## L5 0.852 0.009 96.666 0.000
## L6 0.700 0.009 74.979 0.000
## L7 0.908 0.009 102.565 0.000
## L8 0.835 0.009 88.412 0.000
## L9 0.855 0.009 97.660 0.000
## L10 0.885 0.009 101.337 0.000
## Complex =~
## M1 1.000
## M2 0.899 0.015 60.116 0.000
## M3 0.801 0.012 65.360 0.000
## M4 1.068 0.017 64.564 0.000
## M5 0.625 0.015 40.985 0.000
## M6 1.515 0.021 71.353 0.000
## M7 1.190 0.019 62.960 0.000
## M8 1.453 0.021 70.317 0.000
## M9 1.235 0.018 68.609 0.000
## M10 1.212 0.019 64.739 0.000
## Intr =~
## N1 1.000
## N2 0.775 0.007 108.927 0.000
## N3 0.727 0.006 119.027 0.000
## N4 0.936 0.007 131.165 0.000
## N5 0.593 0.008 77.295 0.000
## N6 0.761 0.007 107.645 0.000
## N7 0.560 0.005 102.222 0.000
## N8 0.727 0.007 107.999 0.000
## N9 0.719 0.008 95.143 0.000
## N10 0.623 0.008 78.019 0.000
## Order =~
## O1 1.000
## O2 2.543 0.050 50.422 0.000
## O3 1.703 0.035 48.881 0.000
## O4 1.518 0.034 45.129 0.000
## O5 1.368 0.030 45.559 0.000
## O6 1.773 0.039 46.024 0.000
## O7 1.921 0.041 47.273 0.000
## O8 3.013 0.059 51.243 0.000
## O9 2.347 0.048 49.026 0.000
## O10 1.887 0.039 47.884 0.000
## Emot =~
## P1 1.000
## P2 0.940 0.006 163.104 0.000
## P3 0.530 0.006 88.216 0.000
## P4 0.512 0.006 89.093 0.000
## P5 0.437 0.005 84.117 0.000
## P6 0.484 0.006 81.521 0.000
## P7 0.396 0.006 63.895 0.000
## P8 0.846 0.006 148.026 0.000
## P9 0.435 0.006 76.558 0.000
## P10 0.394 0.006 68.305 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## Warmth ~~
## Intellect 0.045 0.002 19.331 0.000
## EmotSta 0.119 0.003 40.813 0.000
## Assert 0.139 0.004 36.779 0.000
## Gregar 0.311 0.005 65.131 0.000
## Duti 0.117 0.003 40.744 0.000
## Friend 0.376 0.005 77.040 0.000
## Sens 0.034 0.005 7.444 0.000
## Dist -0.225 0.004 -59.546 0.000
## Imag -0.034 0.003 -12.601 0.000
## Reser -0.369 0.005 -72.007 0.000
## Anx -0.035 0.003 -10.163 0.000
## Complex 0.084 0.002 34.019 0.000
## Intr -0.297 0.005 -63.609 0.000
## Order 0.031 0.002 20.261 0.000
## Emot -0.267 0.005 -53.438 0.000
## Intellect ~~
## EmotSta 0.090 0.002 42.220 0.000
## Assert 0.197 0.003 60.419 0.000
## Gregar -0.023 0.003 -8.550 0.000
## Duti -0.026 0.002 -14.976 0.000
## Friend 0.063 0.003 24.553 0.000
## Sens 0.232 0.004 58.667 0.000
## Dist 0.006 0.002 3.482 0.000
## Imag 0.056 0.002 28.282 0.000
## Reser -0.029 0.003 -10.569 0.000
## Anx -0.114 0.003 -41.711 0.000
## Complex 0.163 0.003 55.379 0.000
## Intr 0.074 0.003 27.102 0.000
## Order 0.031 0.001 26.585 0.000
## Emot -0.086 0.003 -26.908 0.000
## EmotSta ~~
## Assert 0.211 0.004 56.972 0.000
## Gregar 0.132 0.003 39.308 0.000
## Duti 0.072 0.002 33.265 0.000
## Friend 0.247 0.004 61.511 0.000
## Sens -0.018 0.004 -5.172 0.000
## Dist -0.156 0.003 -51.590 0.000
## Imag -0.104 0.003 -41.240 0.000
## Reser -0.155 0.004 -43.925 0.000
## Anx -0.376 0.005 -68.570 0.000
## Complex 0.036 0.002 20.334 0.000
## Intr -0.146 0.003 -43.165 0.000
## Order 0.048 0.001 32.170 0.000
## Emot -0.359 0.005 -66.888 0.000
## Assert ~~
## Gregar 0.235 0.005 51.078 0.000
## Duti -0.020 0.003 -7.516 0.000
## Friend 0.370 0.005 75.878 0.000
## Sens 0.047 0.005 9.370 0.000
## Dist -0.024 0.003 -8.180 0.000
## Imag -0.059 0.003 -19.623 0.000
## Reser -0.219 0.005 -47.302 0.000
## Anx -0.277 0.005 -60.925 0.000
## Complex 0.106 0.003 38.450 0.000
## Intr -0.132 0.004 -31.046 0.000
## Order 0.078 0.002 36.130 0.000
## Emot -0.040 0.005 -8.186 0.000
## Gregar ~~
## Duti -0.020 0.003 -6.772 0.000
## Friend 0.538 0.006 88.407 0.000
## Sens -0.184 0.006 -33.177 0.000
## Dist -0.140 0.004 -39.876 0.000
## Imag -0.016 0.003 -4.913 0.000
## Reser -0.345 0.005 -63.469 0.000
## Anx -0.141 0.004 -32.927 0.000
## Complex 0.016 0.003 6.375 0.000
## Intr -0.485 0.006 -79.410 0.000
## Order -0.015 0.002 -9.087 0.000
## Emot -0.093 0.005 -17.515 0.000
## Duti ~~
## Friend 0.056 0.003 19.773 0.000
## Sens -0.023 0.004 -6.588 0.000
## Dist -0.126 0.003 -47.210 0.000
## Imag -0.158 0.003 -53.202 0.000
## Reser -0.076 0.003 -24.417 0.000
## Anx -0.007 0.003 -2.646 0.008
## Complex -0.068 0.002 -34.325 0.000
## Intr -0.082 0.003 -27.119 0.000
## Order 0.075 0.002 39.158 0.000
## Emot -0.146 0.004 -38.826 0.000
## Friend ~~
## Sens -0.073 0.005 -14.513 0.000
## Dist -0.211 0.004 -56.194 0.000
## Imag -0.114 0.003 -35.699 0.000
## Reser -0.482 0.006 -83.318 0.000
## Anx -0.271 0.005 -59.695 0.000
## Complex 0.057 0.002 22.944 0.000
## Intr -0.452 0.006 -80.620 0.000
## Order 0.042 0.002 23.978 0.000
## Emot -0.247 0.005 -47.273 0.000
## Sens ~~
## Dist 0.005 0.004 1.229 0.219
## Imag 0.173 0.004 41.725 0.000
## Reser 0.033 0.006 6.045 0.000
## Anx 0.006 0.005 1.219 0.223
## Complex 0.229 0.004 53.692 0.000
## Intr 0.252 0.006 44.922 0.000
## Order 0.003 0.002 1.718 0.086
## Emot -0.072 0.006 -11.450 0.000
## Dist ~~
## Imag 0.094 0.003 37.328 0.000
## Reser 0.268 0.004 60.801 0.000
## Anx 0.143 0.003 43.392 0.000
## Complex -0.010 0.002 -5.710 0.000
## Intr 0.229 0.004 56.477 0.000
## Order -0.018 0.001 -14.858 0.000
## Emot 0.318 0.005 63.357 0.000
## Imag ~~
## Reser 0.087 0.003 25.777 0.000
## Anx 0.118 0.003 37.386 0.000
## Complex 0.135 0.003 48.962 0.000
## Intr 0.170 0.004 46.729 0.000
## Order -0.076 0.002 -38.556 0.000
## Emot 0.118 0.004 30.514 0.000
## Reser ~~
## Anx 0.114 0.004 26.597 0.000
## Complex -0.056 0.003 -20.535 0.000
## Intr 0.401 0.006 70.747 0.000
## Order -0.026 0.002 -14.528 0.000
## Emot 0.171 0.006 30.948 0.000
## Anx ~~
## Complex -0.056 0.002 -23.138 0.000
## Intr 0.138 0.004 32.817 0.000
## Order -0.038 0.002 -22.601 0.000
## Emot 0.459 0.006 73.932 0.000
## Complex ~~
## Intr 0.039 0.003 15.348 0.000
## Order -0.009 0.001 -9.319 0.000
## Emot -0.103 0.003 -31.667 0.000
## Intr ~~
## Order -0.006 0.002 -3.931 0.000
## Emot 0.218 0.005 40.239 0.000
## Order ~~
## Emot -0.044 0.002 -20.728 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .A1 0.590 0.005 116.592 0.000
## .A2 0.463 0.004 115.829 0.000
## .A3 0.693 0.006 123.897 0.000
## .A4 0.573 0.005 121.544 0.000
## .A5 0.452 0.004 117.342 0.000
## .A6 0.473 0.004 120.215 0.000
## .A7 0.524 0.004 122.985 0.000
## .A8 1.025 0.008 129.023 0.000
## .A9 0.608 0.005 120.991 0.000
## .A10 0.948 0.007 130.051 0.000
## .B1 0.558 0.005 122.992 0.000
## .B2 0.731 0.006 123.616 0.000
## .B3 0.491 0.004 125.438 0.000
## .B4 0.292 0.002 118.132 0.000
## .B5 0.477 0.004 121.535 0.000
## .B6 0.800 0.006 128.707 0.000
## .B7 0.920 0.007 129.524 0.000
## .B8 0.725 0.006 129.034 0.000
## .B9 1.230 0.010 128.209 0.000
## .B10 0.895 0.007 124.119 0.000
## .B11 1.360 0.010 129.878 0.000
## .B12 0.654 0.005 122.019 0.000
## .B13 1.072 0.008 126.409 0.000
## .C1 1.000 0.008 128.625 0.000
## .C2 0.654 0.005 122.467 0.000
## .C3 0.639 0.005 127.664 0.000
## .C4 0.845 0.007 127.844 0.000
## .C5 1.013 0.008 127.803 0.000
## .C6 0.947 0.008 124.395 0.000
## .C7 0.652 0.006 117.267 0.000
## .C8 0.728 0.006 119.182 0.000
## .C9 0.722 0.006 121.731 0.000
## .C10 0.644 0.005 121.360 0.000
## .D1 0.410 0.004 105.119 0.000
## .D2 0.755 0.006 123.351 0.000
## .D3 0.837 0.007 127.820 0.000
## .D4 0.859 0.007 128.066 0.000
## .D5 0.454 0.004 115.444 0.000
## .D6 0.586 0.005 127.682 0.000
## .D7 0.544 0.005 113.808 0.000
## .D8 0.641 0.005 128.290 0.000
## .D9 0.640 0.005 121.729 0.000
## .D10 0.958 0.008 125.477 0.000
## .E1 0.585 0.005 112.127 0.000
## .E2 0.573 0.006 102.590 0.000
## .E3 0.817 0.006 128.902 0.000
## .E4 0.694 0.006 112.680 0.000
## .E5 0.567 0.004 129.391 0.000
## .E6 1.103 0.009 126.725 0.000
## .E7 1.110 0.008 131.905 0.000
## .E8 0.815 0.007 117.714 0.000
## .E9 0.952 0.007 132.121 0.000
## .E10 1.211 0.009 130.122 0.000
## .F1 0.840 0.007 126.688 0.000
## .F2 0.417 0.004 113.030 0.000
## .F3 2.153 0.016 131.109 0.000
## .F4 0.446 0.004 106.579 0.000
## .F5 1.433 0.011 129.389 0.000
## .F6 0.476 0.004 109.547 0.000
## .F7 0.671 0.006 118.994 0.000
## .F8 1.429 0.011 130.771 0.000
## .F9 0.450 0.004 108.066 0.000
## .F10 0.869 0.007 128.323 0.000
## .G1 0.426 0.004 117.899 0.000
## .G2 0.728 0.006 122.143 0.000
## .G3 0.962 0.008 127.879 0.000
## .G4 0.559 0.005 118.510 0.000
## .G5 0.497 0.004 118.740 0.000
## .G6 0.636 0.005 119.461 0.000
## .G7 0.773 0.006 124.456 0.000
## .G8 0.895 0.007 129.064 0.000
## .G9 0.741 0.006 124.052 0.000
## .G10 0.697 0.006 124.695 0.000
## .H1 0.194 0.004 47.101 0.000
## .H2 0.752 0.006 128.930 0.000
## .H3 0.344 0.005 66.883 0.000
## .H4 1.314 0.010 132.473 0.000
## .H5 1.753 0.013 132.336 0.000
## .H6 1.173 0.009 131.990 0.000
## .H7 1.560 0.012 132.263 0.000
## .H8 1.315 0.010 130.839 0.000
## .H9 0.834 0.006 129.930 0.000
## .H10 1.222 0.009 132.241 0.000
## .I1 0.962 0.008 126.938 0.000
## .I2 0.760 0.006 124.967 0.000
## .I3 0.783 0.006 127.345 0.000
## .I4 0.428 0.004 104.018 0.000
## .I5 0.905 0.007 128.913 0.000
## .I6 0.782 0.006 125.363 0.000
## .I7 0.517 0.004 118.647 0.000
## .I8 0.437 0.004 107.988 0.000
## .I9 0.456 0.004 117.546 0.000
## .I10 0.727 0.006 126.246 0.000
## .J1 0.817 0.007 122.881 0.000
## .J2 0.582 0.005 108.442 0.000
## .J3 0.787 0.007 116.769 0.000
## .J4 0.570 0.005 110.701 0.000
## .J5 0.849 0.007 128.340 0.000
## .J6 0.906 0.007 128.764 0.000
## .J7 0.932 0.007 129.006 0.000
## .J8 0.944 0.007 129.632 0.000
## .J9 0.923 0.008 121.356 0.000
## .J10 0.939 0.008 123.139 0.000
## .K1 0.552 0.005 114.506 0.000
## .K2 0.980 0.008 125.535 0.000
## .K3 1.074 0.008 127.126 0.000
## .K4 0.905 0.007 125.957 0.000
## .K5 0.747 0.006 124.095 0.000
## .K6 0.470 0.004 105.537 0.000
## .K7 0.432 0.004 105.217 0.000
## .K8 1.146 0.009 130.236 0.000
## .K9 0.752 0.006 120.881 0.000
## .K10 0.737 0.006 124.970 0.000
## .L1 0.827 0.007 122.550 0.000
## .L2 0.728 0.006 122.121 0.000
## .L3 0.748 0.006 119.483 0.000
## .L4 0.612 0.005 121.010 0.000
## .L5 0.830 0.007 125.339 0.000
## .L6 1.209 0.009 129.213 0.000
## .L7 0.759 0.006 123.570 0.000
## .L8 1.071 0.008 127.183 0.000
## .L9 0.806 0.006 125.072 0.000
## .L10 0.754 0.006 123.978 0.000
## .M1 0.946 0.007 127.367 0.000
## .M2 0.655 0.005 126.498 0.000
## .M3 0.344 0.003 123.375 0.000
## .M4 0.653 0.005 123.979 0.000
## .M5 1.116 0.009 130.827 0.000
## .M6 0.669 0.006 115.570 0.000
## .M7 0.924 0.007 125.032 0.000
## .M8 0.696 0.006 117.557 0.000
## .M9 0.604 0.005 120.094 0.000
## .M10 0.830 0.007 123.851 0.000
## .N1 0.639 0.006 113.871 0.000
## .N2 0.692 0.006 122.230 0.000
## .N3 0.451 0.004 118.559 0.000
## .N4 0.504 0.005 111.790 0.000
## .N5 1.046 0.008 128.549 0.000
## .N6 0.693 0.006 122.614 0.000
## .N7 0.440 0.004 124.078 0.000
## .N8 0.626 0.005 122.509 0.000
## .N9 0.892 0.007 125.668 0.000
## .N10 1.127 0.009 128.456 0.000
## .O1 0.922 0.007 129.814 0.000
## .O2 0.804 0.007 112.203 0.000
## .O3 0.561 0.005 119.511 0.000
## .O4 0.898 0.007 126.096 0.000
## .O5 0.683 0.005 125.654 0.000
## .O6 1.065 0.009 125.120 0.000
## .O7 1.004 0.008 123.295 0.000
## .O8 0.808 0.008 104.138 0.000
## .O9 1.028 0.009 119.036 0.000
## .O10 0.859 0.007 122.110 0.000
## .P1 0.401 0.004 90.771 0.000
## .P2 0.582 0.005 107.074 0.000
## .P3 0.983 0.008 127.851 0.000
## .P4 0.899 0.007 127.744 0.000
## .P5 0.746 0.006 128.331 0.000
## .P6 0.980 0.008 128.617 0.000
## .P7 1.118 0.009 130.233 0.000
## .P8 0.657 0.006 114.394 0.000
## .P9 0.912 0.007 129.128 0.000
## .P10 0.958 0.007 129.879 0.000
## Warmth 0.527 0.008 69.795 0.000
## Intellect 0.233 0.005 51.196 0.000
## EmotSta 0.314 0.007 46.901 0.000
## Assert 0.622 0.008 82.607 0.000
## Gregar 0.731 0.009 78.463 0.000
## Duti 0.317 0.006 50.255 0.000
## Friend 0.675 0.008 86.162 0.000
## Sens 1.089 0.010 105.517 0.000
## Dist 0.355 0.007 49.784 0.000
## Imag 0.337 0.007 50.792 0.000
## Reser 0.805 0.010 83.324 0.000
## Anx 0.587 0.009 65.016 0.000
## Complex 0.214 0.005 39.193 0.000
## Intr 0.702 0.009 75.082 0.000
## Order 0.093 0.004 26.438 0.000
## Emot 0.994 0.011 93.936 0.000
The fit statistics mostly indicate poor fit. Our comparative fit indices, CFI = 0.678, TLI = 0.671, should be much higher, preferably in the 0.9 range. And GFI (0.664), a measure of absolute fit, is similarly low. But RMSEA, 0.042, is good, and SRMR (0.074) is acceptable. There a few potential issues that could explain poor fit, including the fact our observed variables aren’t normally distributed, but before we try a different estimation method or way of dealing with missing data, let’s look at what the model is telling us. All items load onto their factor significantly and positively. Factors are correlated with each other, except sensitivity and distrust, sensitivity and anxiety, and sensitivity and orderliness. Let’s extract parameter estimates, as well as standardized estimates, so we can dig in a bit more. Remember that for the unstandardized solution, the factors (the latent variables in our model) don’t have a metric, so the first item (observed variables) is always loaded at 1 to set the range. If we want to see how strongly that item loads on a factor, you’ll want the standardized values.
parameters <- parameterEstimates(CFA16)
standardizedsol <- standardizedSolution(CFA16)
unstandardized <- parameters[c(1:163), 4]
standardized <- standardizedsol[c(1:163), 4]
ByItem <- cbind(ByItem, unstandardized, standardized)
summary(ByItem$standardized)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.06515 0.44843 0.56886 0.55692 0.68524 0.92115
We have at least one factor loading that is quite low - about 0.07 standardized loading. Turns out, that’s item H4, the action movie item from the Sensitivity scale that we also flagged for low P, low discrimination, and poor scale function. Every indicator we’ve examined has show this item doesn’t really relate to the concept of “Sensitivity.” In fact, our four lowest factor ratings all belong to the Sensitivity scale. This scale also had our worst alpha. My recommendation would be, at the very least, to drop the 4th item from Sensitivity and rerun alpha.
Scores <- Items %>%
filter(Item != "H4") %>%
ungroup() %>%
group_by(ID, Code) %>%
summarise(Total = sum(Rating, na.rm = TRUE),
NItems = sum(!is.na(Rating))) %>%
ungroup() %>%
left_join(Scales, by = "Code") %>%
mutate(Items = ifelse(Name == "Sensitivity", 9, Items),
Score = Total/NItems,
CorTotal = Score*Items)
## `summarise()` has regrouped the output.
## ℹ Summaries were computed grouped by ID and Code.
## ℹ Output is grouped by ID.
## ℹ Use `summarise(.groups = "drop_last")` to silence this message.
## ℹ Use `summarise(.by = c(ID, Code))` for per-operation grouping
## (`?dplyr::dplyr_by`) instead.
ByScale <- Scores %>%
group_by(Code) %>%
summarise(Variance = var(CorTotal, na.rm = TRUE))
ByScale2 <- ByItem %>%
ungroup() %>%
mutate(Code = substr(Column, 1, 1)) %>%
group_by(Code) %>%
summarise(EItemVar = sum(Variance))
Scales <- Scales %>%
left_join(ByScale, by = "Code") %>%
left_join(ByScale2, by = "Code")
Scales <- Scales %>%
mutate(Items.y = ifelse(Name == "Sensitivity", 9, Items),
Alpha.y = (Items.y/(Items.y-1)) * (1 - (EItemVar.y/Variance.y)))
Removing H4 actually makes alpha go down. Remember, though, that alpha rewards longer measures (more items increase reliability in CTT), so any gain from removing a poor item could be lost because now the measure is shorter. We could drop Sensitivity fully from the measure, but then that would break the factor structure Cattell theorized that included this concept of Sensitivity. If full removal is unacceptable, then the Sensitivity scale probably needs an overhaul with edited/new items.
Note that, while I said our CFA could handle some missing data, the estimation method (maximum likelihood, the default) uses only complete cases. An estimation method that allows missing data might get better results. Further, our data is not normally distributed, another requirement of this estimation method. So we want a different estimator that can handle some missing data and that works with nonnormal data. This paper recommends a weighted least squares method with rating scale items entered as ordinal. This approach definitely makes sense, but fair warning, this approach will use a lot of memory (to store the many matrices) and will take a while to run, especially if you don’t have a lot of RAM and/or are running programs like RStudio that use a lot of RAM. My current system has 48 gigs of RAM, but on my previous 16 gig system, I would sometimes need to close RStudio and run my code directly in R.
This is the code I ran: CFA2 <- cfa(model = Factor16, data = PF16R, ordered = TRUE, estimator = “WLSMV”)
Why am I telling you instead of showing you? Unfortunately, this analysis took over 1.5 hours and found similar results as the ML estimation model: fit statistics were in the 0.5-0.6 range, lack of correlation between Sensitivity and multiple other scales, and H4 had the lowest loading. So the issue isn’t that the data aren’t normally distributed. The issue is that the prescribed model does not fit the data well.
So, out of pure curiosity, I’m going to do something I typically do not recommend. I’m going to run an exploratory factor analysis, asking it to derive up to 16 factors. Normally I do not condone this kind of fishing. If you expect certain items to load on certain factors, you should use CFA. And if your CFA doesn’t work out, you should have some backup options - a different estimator, a different arrangement of factors, or even just accepting that your theoretical model isn’t it. But I don’t want to use this approach to see how the items should fall, to try to develop a new measure or personality theory. I want to use it for diagnosis - where do things fall apart? Do we have a small number of problem factors, or is the issue more widespread than that?
EFA in lavaan allows you to specify a number of factors, or even a range of factors, to extract. I definitely want to see what it does with the 16 factor solution. We could compare that to another personality theory, Big Five, which states that personality is made up of five independent but correlated factors: Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Emotional Stability. I could see how some of the 16 factors could map onto the Big Five, but the one exception I see is Imagination. It doesn’t have a clear place among the Big Five, and seems to be a unique contribution by Cattell. So let’s see what happens if I ask the EFA to extract 5, 6, and 16 factors.
ItemList <- grep("^[A-Za-z][0-9]+$", colnames(PF16R), value = TRUE)
EFA <- efa(data = PF16R, nfactors = c(5, 6, 16), rotation = "varimax", ov_names = ItemList)
I’ll start by looking at the 16 factor solution.
summary(EFA$nf16, fit.measures = TRUE)
## lavaan 0.7-2 ended normally after 1 iteration
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 2771
## Row rank of the constraints matrix 283
##
## Rotation method VARIMAX ORTHOGONAL
## Rotation algorithm (rstarts) GPA (30)
## Standardized metric TRUE
## Row weights Kaiser
##
## Used Total
## Number of observations 35127 46427
##
## Model Test User Model:
##
## Test statistic 261484.745
## Degrees of freedom 10715
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 2538331.585
## Degrees of freedom 13203
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.901
## Tucker-Lewis Index (TLI) 0.878
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -7427538.797
## Loglikelihood unrestricted model (H1) -7296796.425
##
## Akaike (AIC) 14860379.594
## Bayesian (BIC) 14882824.883
## Sample-size adjusted Bayesian (SABIC) 14874400.014
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.026
## 90 Percent confidence interval - lower 0.026
## 90 Percent confidence interval - upper 0.026
## P-value H_0: RMSEA <= 0.050 1.000
## P-value H_0: RMSEA >= 0.080 0.000
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.015
##
## Goodness of Fit Index:
##
## Goodness of Fit Index (GFI) 0.899
## 90 Percent confidence interval - lower 0.899
## 90 Percent confidence interval - upper 0.900
##
## Standardized loadings: (* = significant at 1% level)
##
## f1 f2 f3 f4 f5 f6 f7 f8 f9
## A1 0.591* .* * * * .* * *
## A2 0.502* * .* * * 0.357* * .* .*
## A3 0.612* * * .* * * .*
## A4 0.489* * * * .* * .*
## A5 0.532* * .* * .* 0.363* * .*
## A6 0.510* * .* * .* 0.319* * .*
## A7 0.517* * .* .* .* * * .*
## A8 .* * * * .* * .*
## A9 0.490* * * * * .* * * .*
## A10 0.359* * * * * * * *
## B1 * .* * * * * *
## B2 .* .* .* * * * *
## B3 * .* * * * * * *
## B4 * .* * * .* * *
## B5 * .* * .* * * * *
## B6 .* .* .* .* * * * .*
## B7 * * * .* .* .* * * *
## B8 * .* * .* .* * * *
## B9 * .* .* .* .* * * *
## B10 * .* * * 0.439* * * .* *
## B11 * .* .* * .* .* * * *
## B12 * * .* .* .* .* *
## B13 * * * .* * .* .* *
## C1 * * 0.445* * * *
## C2 .* .* * * 0.615* .* * .* .*
## C3 .* .* * 0.465* * * *
## C4 * * .* * 0.514* * * .* .*
## C5 * * * * 0.492* * .*
## C6 * * .* 0.591* * * .* .*
## C7 * .* 0.680* .* * * .*
## C8 .* * .* * 0.661* .* * * .*
## C9 * * * .* 0.652* .* * .* *
## C10 * .* * 0.664* .* * * *
## D1 * 0.668* * * .* .* * *
## D2 * 0.574* * .* * * *
## D3 * 0.324* .* .* .* .* * * *
## D4 .* 0.336* .* * .* .* * *
## D5 * 0.616* * .* .* * * *
## D6 * 0.385* .* * .* * .* *
## D7 * 0.518* * * 0.322* .* * * *
## D8 * .* * .* .* * * * *
## D9 0.469* .* * .* .* * *
## D10 * .* * * 0.471* .* * * *
## E1 .* .* * * .* 0.667* * .* *
## E2 * * .* * .* 0.617* * .* *
## E3 .* 0.474* * 0.315* * .* *
## E4 * * .* * * 0.550* .* .*
## E5 .* * 0.398* * 0.322* * *
## E6 * * .* .* * 0.395* * 0.372* *
## E7 .* * 0.311* * * .* * .*
## E8 * * * * .* 0.516* * .* .*
## E9 * .* .* .* * *
## E10 * * .* * .* * .* *
## F1 .* 0.521* * * * *
## F2 .* * 0.731* * * * * .*
## F3 .* * * .* * * .* *
## F4 .* * * 0.747* * * * * .*
## F5 .* .* * 0.339* .* .* * .*
## F6 .* * * 0.738* * * * * .*
## F7 * .* .* 0.646* * * .* .*
## F8 * * -0.347* .* .* * * .*
## F9 .* * 0.742* * * * * .*
## F10 * .* .* 0.409* * * .* .*
## G1 .* .* * * 0.306* 0.591* * .*
## G2 .* * * * .* 0.694* * *
## G3 * .* * .* 0.538* * .* *
## G4 .* * * * .* 0.659* * .*
## G5 .* .* * .* 0.653* * * *
## G6 .* .* * * 0.341* 0.619* * .* *
## G7 .* * * * 0.357* 0.521* * * .*
## G8 .* .* * .* 0.407* .*
## G9 * .* * * .* 0.617* .* *
## G10 * .* * * .* 0.583* * * *
## H1 * * * * 0.783* *
## H2 .* * * * * * 0.304* .* *
## H3 * * * .* 0.771*
## H4 * .* * .* * * .*
## H5 0.320* * .* * .* * .* * *
## H6 0.385* * .* * * * .* .* *
## H7 .* * .* * * .* * .* *
## H8 .* * .* * * * .* .*
## H9 * .* * * * * .* * *
## H10 .* * * * .* *
## I1 .* * * .* * * 0.392*
## I2 * * * .* .* * * .* 0.494*
## I3 * * * * .* .* * .* 0.390*
## I4 .* * * .* .* .* * 0.687*
## I5 * * * * .* * * .* 0.357*
## I6 .* * .* .* * .* 0.477*
## I7 .* * * .* * .* * 0.677*
## I8 .* * * .* .* .* * 0.704*
## I9 .* * .* .* * * 0.633*
## I10 .* * * .* * * * * 0.505*
## J1 * * .* .* .* * 0.366* .*
## J2 * .* .* .* .* * 0.305* *
## J3 * * * .* .* * * 0.465* *
## J4 * .* * .* .* * 0.407* *
## J5 .* * -0.303* * .* .*
## J6 .* .* * -0.346* .* .* .*
## J7 * .* .* -0.301* * .* 0.398* *
## J8 * * -0.559* * * * * *
## J9 * * * .* * * .* *
## J10 * * * .* .* .* * * *
## K1 .* * * .* * * .*
## K2 .* * * .* .* -0.378* * .* .*
## K3 .* .* * * * -0.602* .* *
## K4 * * .* * -0.337* .* * .*
## K5 * .* * .* .* * .* *
## K6 .* * * * * .* * .*
## K7 .* * * * * .* * * .*
## K8 * * * * * .* * * *
## K9 .* * * * * .* * * .*
## K10 .* * * * .* * * .*
## L1 * .* * * -0.586* .* * * *
## L2 * * * -0.622* .* * * .*
## L3 .* * * -0.648* * * *
## L4 .* * * * -0.616* * * * .*
## L5 * * * * -0.557* * * * *
## L6 .* .* .* -0.410* * .*
## L7 * * * -0.586* * * * *
## L8 * * * * -0.484* * * .* *
## L9 * * -0.551* * * * *
## L10 * .* * -0.604* * .* *
## M1 .* * .* * * * .* .* *
## M2 .* .* * .* .* .* * 0.320* *
## M3 .* * * * .* * * .* *
## M4 * .* .* * .* * * *
## M5 .* * * .* .* .* * .* *
## M6 * * .* * .*
## M7 .* * * * * * *
## M8 * * * * * *
## M9 * * * .* * * * .* *
## M10 * * * .* * * * *
## N1 .* * * .* -0.301* * * .*
## N2 .* .* * * .* .* * .* .*
## N3 * * * .* * * *
## N4 * * * .* -0.387* * *
## N5 * * * * .* * *
## N6 * * * * .* * * *
## N7 * * * .* * .*
## N8 .* .* .* .* -0.423* .* .*
## N9 .* * * .* .* .* * .* .*
## N10 .* * * * .* .* * .*
## O1 * .* .* .* * * * *
## O2 .* .* * .* .* * * * *
## O3 * .* * 0.332* * * *
## O4 * .* * .* * * * .* *
## O5 * .* * .* .* * * *
## O6 * * .* * * * .* *
## O7 * * * .* .* * .* *
## O8 * * * .* * * * *
## O9 * * * .* * * * *
## O10 * .* * .* 0.300* * * *
## P1 .* .* .* .* -0.519* * * * .*
## P2 .* .* .* .* -0.454* * * .*
## P3 .* .* * * .* * * .*
## P4 .* .* * * .* * * .*
## P5 .* * * -0.493* * * *
## P6 * * * .* * * * .*
## P7 .* * * * .* * * * *
## P8 .* .* * * -0.415* * * * .*
## P9 .* * * .* .* * * .* 0.355*
## P10 -0.400* .* * .* .* .*
## f10 f11 f12 f13 f14 f15 f16 unique.var
## A1 * .* * * * * 0.504
## A2 * .* * * .* * * 0.489
## A3 .* * * * * * 0.550
## A4 .* * .* .* * 0.613
## A5 * * * * 0.488
## A6 * .* .* * * * * 0.537
## A7 * * * .* * * 0.621
## A8 * .* .* .* * * 0.799
## A9 * .* * .* .* * 0.548
## A10 * * * .* * * .* 0.775
## B1 * * * 0.486* .* .* 0.665
## B2 * 0.393* .* .* 0.667
## B3 * * 0.456* .* .* .* 0.651
## B4 * * 0.476* .* .* .* 0.636
## B5 * * 0.382* * * * 0.708
## B6 * 0.350* * * .* 0.677
## B7 * * .* .* .* .* * 0.703
## B8 * .* .* * .* .* 0.799
## B9 * * * 0.341* * * 0.724
## B10 * * 0.353* * .* * 0.570
## B11 * .* .* .* * .* 0.695
## B12 * * 0.515* * * 0.635
## B13 * * 0.443* * * * 0.722
## C1 .* * * * * * .* 0.735
## C2 * .* * * * * .* 0.443
## C3 * * * .* * * * 0.661
## C4 .* * * * * 0.596
## C5 * * -0.511* * * * 0.452
## C6 * .* .* .* * 0.505
## C7 * * * .* * .* 0.427
## C8 * .* .* * .* .* 0.360
## C9 * * .* * * * * 0.488
## C10 .* * * * 0.472
## D1 * * * * * .* * 0.384
## D2 * * * * * * .* 0.565
## D3 * .* * * * 0.676
## D4 * .* * .* * * * 0.720
## D5 * * * * .* * 0.492
## D6 * * * .* * * * 0.712
## D7 * .* .* * * .* 0.458
## D8 * * * 0.348* * * 0.684
## D9 * .* .* * * * 0.608
## D10 * .* .* * * * 0.590
## E1 * .* * * .* * * 0.415
## E2 * * .* -0.313* * * 0.410
## E3 * * * .* * .* .* 0.565
## E4 * * .* -0.339* * * 0.456
## E5 * * * .* * .* 0.598
## E6 * * * * .* 0.528
## E7 0.305* * * .* * .* * 0.658
## E8 * * * -0.353* * 0.527
## E9 * * .* * 0.830
## E10 * .* * .* * * 0.783
## F1 * * * .* * .* * 0.639
## F2 * * * * * .* * 0.390
## F3 * * .* * * 0.820
## F4 * * * .* * * 0.338
## F5 * * .* * * * 0.737
## F6 * * * * .* 0.385
## F7 * * * .* * 0.471
## F8 * * .* * * .* * 0.717
## F9 * * * * * .* 0.382
## F10 * * * * * * .* 0.650
## G1 * .* * * .* * * 0.368
## G2 * .* * * .* * * 0.415
## G3 * .* * * .* * .* 0.562
## G4 * .* * * * * * 0.362
## G5 * .* * * * * 0.390
## G6 * .* * * * 0.382
## G7 * .* * .* * 0.509
## G8 * .* .* 0.349* .* * 0.548
## G9 .* * * .* * .* 0.457
## G10 * .* .* * .* * * 0.460
## H1 * * * 0.420* .* * * 0.161
## H2 * * * 0.400* * * * 0.688
## H3 * * 0.402* .* * 0.191
## H4 * * * * * * * 0.872
## H5 * .* .* * * * 0.738
## H6 * .* * * * * * 0.719
## H7 * .* * * * * 0.809
## H8 * * 0.346* * .* 0.691
## H9 .* * .* 0.408* * .* * 0.702
## H10 * .* .* .* * * * 0.828
## I1 * .* .* * * * .* 0.645
## I2 * * * * * .* 0.578
## I3 * .* * * .* * 0.680
## I4 .* * * .* * 0.359
## I5 .* * * * .* * .* 0.758
## I6 * * * .* .* * 0.609
## I7 .* * * * * * 0.448
## I8 * .* * * .* * * 0.353
## I9 * .* * * * * 0.466
## I10 * * * * * * * 0.659
## J1 * * * .* .* .* 0.650
## J2 .* * * 0.333* .* .* * 0.536
## J3 .* * .* .* .* * 0.590
## J4 0.305* * * .* .* .* .* 0.538
## J5 * * .* .* * * 0.709
## J6 * * * * * * 0.702
## J7 * * * * .* * 0.629
## J8 * * * * * .* .* 0.581
## J9 0.830* * .* * * 0.189
## J10 0.760* * .* * * 0.288
## K1 * 0.668* * * .* * * 0.392
## K2 0.377* * * .* * * 0.546
## K3 * .* * * .* * * 0.455
## K4 0.494* * * .* * * 0.532
## K5 * 0.511* * .* .* * * 0.508
## K6 * 0.745* * * * * 0.287
## K7 * 0.785* * * * 0.250
## K8 * 0.395* * * * * * 0.799
## K9 0.649* * * * * 0.443
## K10 0.517* * .* * * .* 0.589
## L1 * * * * * .* 0.553
## L2 * * * .* 0.529
## L3 * * .* .* * .* 0.474
## L4 * * * * * * .* 0.555
## L5 * * * * .* 0.643
## L6 * * .* * * * .* 0.692
## L7 * * * .* 0.620
## L8 * * .* * * * * 0.688
## L9 * * 0.306* * * 0.577
## L10 * * * * * * * 0.555
## M1 * * * 0.343* .* * .* 0.654
## M2 * * * .* * * 0.661
## M3 * * * 0.418* * * 0.679
## M4 * .* * 0.429* * * * 0.626
## M5 * * * * * .* * 0.764
## M6 * * * 0.611* * * * 0.590
## M7 * * * 0.505* * * 0.703
## M8 * * 0.625* * * * 0.585
## M9 * * 0.534* * * .* 0.625
## M10 * * 0.544* * * .* 0.640
## N1 * .* * 0.545* * 0.423
## N2 .* * 0.494* * * 0.605
## N3 * * * .* 0.651* * 0.480
## N4 * .* * * 0.613* * * 0.415
## N5 * * .* 0.423* * 0.755
## N6 * * 0.608* * * 0.550
## N7 * .* * .* 0.505* * * 0.620
## N8 * * * * 0.352* .* 0.509
## N9 * * * .* .* * 0.595
## N10 * .* * .* 0.388* * .* 0.712
## O1 * * * 0.313* .* 0.699
## O2 * * * .* * 0.584* .* 0.511
## O3 * * * * 0.489* .* 0.597
## O4 * * .* .* * 0.414* .* 0.646
## O5 * * .* * 0.361* .* 0.676
## O6 * * .* * 0.458* * 0.724
## O7 * .* 0.497* * 0.667
## O8 * * * * 0.696* .* 0.433
## O9 * * * * 0.573* .* 0.611
## O10 .* * * * * 0.441* .* 0.595
## P1 * 0.507* .* .* .* 0.308
## P2 * * 0.485* .* * * * 0.384
## P3 * .* * * * 0.372* 0.670
## P4 * * .* * * 0.348* 0.649
## P5 * .* * * .* 0.674
## P6 * * .* .* * .* 0.756
## P7 * * * * * * 0.402* 0.727
## P8 * * 0.575* * * * .* 0.405
## P9 * .* * 0.698
## P10 * * .* * * * * 0.650
## communalities
## A1 0.496
## A2 0.511
## A3 0.450
## A4 0.387
## A5 0.512
## A6 0.463
## A7 0.379
## A8 0.201
## A9 0.452
## A10 0.225
## B1 0.335
## B2 0.333
## B3 0.349
## B4 0.364
## B5 0.292
## B6 0.323
## B7 0.297
## B8 0.201
## B9 0.276
## B10 0.430
## B11 0.305
## B12 0.365
## B13 0.278
## C1 0.265
## C2 0.557
## C3 0.339
## C4 0.404
## C5 0.548
## C6 0.495
## C7 0.573
## C8 0.640
## C9 0.512
## C10 0.528
## D1 0.616
## D2 0.435
## D3 0.324
## D4 0.280
## D5 0.508
## D6 0.288
## D7 0.542
## D8 0.316
## D9 0.392
## D10 0.410
## E1 0.585
## E2 0.590
## E3 0.435
## E4 0.544
## E5 0.402
## E6 0.472
## E7 0.342
## E8 0.473
## E9 0.170
## E10 0.217
## F1 0.361
## F2 0.610
## F3 0.180
## F4 0.662
## F5 0.263
## F6 0.615
## F7 0.529
## F8 0.283
## F9 0.618
## F10 0.350
## G1 0.632
## G2 0.585
## G3 0.438
## G4 0.638
## G5 0.610
## G6 0.618
## G7 0.491
## G8 0.452
## G9 0.543
## G10 0.540
## H1 0.839
## H2 0.312
## H3 0.809
## H4 0.128
## H5 0.262
## H6 0.281
## H7 0.191
## H8 0.309
## H9 0.298
## H10 0.172
## I1 0.355
## I2 0.422
## I3 0.320
## I4 0.641
## I5 0.242
## I6 0.391
## I7 0.552
## I8 0.647
## I9 0.534
## I10 0.341
## J1 0.350
## J2 0.464
## J3 0.410
## J4 0.462
## J5 0.291
## J6 0.298
## J7 0.371
## J8 0.419
## J9 0.811
## J10 0.712
## K1 0.608
## K2 0.454
## K3 0.545
## K4 0.468
## K5 0.492
## K6 0.713
## K7 0.750
## K8 0.201
## K9 0.557
## K10 0.411
## L1 0.447
## L2 0.471
## L3 0.526
## L4 0.445
## L5 0.357
## L6 0.308
## L7 0.380
## L8 0.312
## L9 0.423
## L10 0.445
## M1 0.346
## M2 0.339
## M3 0.321
## M4 0.374
## M5 0.236
## M6 0.410
## M7 0.297
## M8 0.415
## M9 0.375
## M10 0.360
## N1 0.577
## N2 0.395
## N3 0.520
## N4 0.585
## N5 0.245
## N6 0.450
## N7 0.380
## N8 0.491
## N9 0.405
## N10 0.288
## O1 0.301
## O2 0.489
## O3 0.403
## O4 0.354
## O5 0.324
## O6 0.276
## O7 0.333
## O8 0.567
## O9 0.389
## O10 0.405
## P1 0.692
## P2 0.616
## P3 0.330
## P4 0.351
## P5 0.326
## P6 0.244
## P7 0.273
## P8 0.595
## P9 0.302
## P10 0.350
##
## f5 f6 f13 f4 f1 f9 f11 f2
## Sum of sq (ortho) loadings 11.086 8.592 6.852 5.226 5.170 4.574 4.296 4.016
## Proportion of total 0.160 0.124 0.099 0.076 0.075 0.066 0.062 0.058
## Proportion var 0.068 0.053 0.042 0.032 0.032 0.028 0.026 0.025
## Cumulative var 0.068 0.121 0.163 0.195 0.227 0.255 0.281 0.306
## f14 f15 f8 f12 f16 f3 f10 f7
## Sum of sq (ortho) loadings 3.992 3.378 2.728 2.111 1.894 1.796 1.769 1.636
## Proportion of total 0.058 0.049 0.039 0.031 0.027 0.026 0.026 0.024
## Proportion var 0.024 0.021 0.017 0.013 0.012 0.011 0.011 0.010
## Cumulative var 0.330 0.351 0.368 0.381 0.392 0.403 0.414 0.424
## total
## Sum of sq (ortho) loadings 69.117
## Proportion of total 1.000
## Proportion var 0.424
## Cumulative var 0.424
Our model fit is quite good, but that’s not too surprising since the data determines the model, rather than the other way around. I extracted the loadings into a data frame, then keep only the highest loading for each item. The gives me a dataset matching the standardized loading output from above. This was a bit easier to work with than scrolling through the output.
StLoadings16 <- as_tibble(EFA$loadings$nf16)
StLoadings16 <- StLoadings16 %>%
mutate(Item = ItemList) %>%
group_by(Item) %>%
pivot_longer(cols = f1:f16,
names_to = "factor",
values_to = "loading") %>%
filter(loading == max(loading))
StLoadings16 <- StLoadings16 %>%
pivot_wider(id_cols = Item,
names_from = factor,
values_from = loading,
values_fill = NA)
The results are, of course, quite a mess and somewhat hard to read. But I went through it and here’s the scales that had factors all to themselves: F, K, N, and O (or Dutifulness, Reserve, Introversion, and Orderliness). Some scales split across a couple factors, or shared factors with other scales. Our worst offenders were H, J, and L (Sensitivity, Imagination, and Anxiety). Each of these 3 scales split into 4 factors (with a total of 8 factors among them, since there was a bit of overlap); some of those factors were made up of a few items. H4, our worst item of the bunch, didn't load with any of the other H items. So we seem to have decent support for 4 of Cattell’s factors, little to no support for 3, and mediocre support for the rest.
I created a chart color-coding where items fell. It’s still messy and a bit overwhelming. But it gives you some idea of what scales seem to hold together and which ones fall apart.
Since this post is already overlong, I won’t be going through the 5 and 6 factor models, but look for them in a future post! Next time, let’s subject our 16 Factor Measure to Rasch measurement!





No comments:
Post a Comment