Reliability Deep Dive: Beyond Cronbach’s Alpha

Psychometrics

Reliability

Alpha

Omega

SEM

CFA

Composite Reliability

AVE

HTMT

Measurement

Comprehensive reliability analysis on the Big Five Inventory (BFI) dataset (N ≈ 2,800). Covers Cronbach’s α with confidence intervals, McDonald’s ω (total and hierarchical), split-half reliability (Spearman-Brown corrected), standard error of measurement (SEM), test–retest simulation, and factor-specific reliability, plus an SEM (structural equation modeling) addendum on composite reliability (CR), average variance extracted (AVE), and convergent/discriminant validity. Includes critical discussion of when α misleads and ω is preferred.

Published

March 23, 2026

0.1 Context

Reliability — the consistency of measurement — is arguably the most fundamental psychometric property of any assessment instrument. Without adequate reliability, validity evidence is uninterpretable, group comparisons are attenuated, and individual scores carry excessive uncertainty.

Yet the most commonly reported reliability coefficient, Cronbach’s α (1951), is widely misunderstood and frequently misapplied. Alpha estimates internal consistency under the assumption of essential tau-equivalence — that all items measure the same construct with equal true-score weights and uncorrelated errors. When this assumption is violated (as it almost always is in practice), α can either over- or underestimate true reliability (Sijtsma, 2009; McNeish, 2018).

McDonald’s ω (1999) provides a more general alternative. Omega total (ωt) estimates the proportion of total variance attributable to all common factors (without requiring equal loadings), while omega hierarchical (ωh) isolates the variance attributable to the general factor alone. The ωt/ωh distinction is critical for deciding whether a total score is justified or whether subscale scores should be reported (Reise et al., 2013).

This project demonstrates a comprehensive reliability analysis toolkit on the Big Five Inventory (BFI) dataset from the psych package (N ≈ 2,800; 25 items across 5 personality factors). We cover:

Cronbach’s α with bootstrap confidence intervals and α-if-item- deleted diagnostics.
McDonald’s ω — total and hierarchical, estimated via both the Schmid-Leiman decomposition and CFA (lavaan).
Split-half reliability with Spearman-Brown correction and all possible splits.
Standard Error of Measurement (SEM) and score confidence intervals.
Factor-specific reliability — per-subscale α and ω, revealing which factors are measured with acceptable precision.
SEM-based measurement quality — Composite Reliability (CR), Average Variance Extracted (AVE), Fornell-Larcker checks, and HTMT as an addendum for reflective CFA/SEM workflows.

The BFI is ideal for this demonstration because its 5-factor structure produces heterogeneous factor loadings (violating tau-equivalence) and correlated factors (complicating total-score interpretation), making the α/ω comparison maximally informative.

0.2 Key Results

Note: The results below are theoretical expectations based on published analyses of the BFI and the psychometric literature. They will be confirmed or refined once the code is executed.

Expected findings:

Cronbach’s α for the full 25-item BFI should be moderate (≈ 0.70–0.80), reflecting the multidimensionality of the item set. Per-factor α should vary: Neuroticism and Extraversion typically show α ≈ 0.80–0.85, while Openness and Agreeableness may be lower (0.70–0.75).
ωt (total) should exceed α because it does not require equal loadings. The difference (ωt − α) will be small if loadings are relatively homogeneous within factors, and larger if they are not.
ωh (hierarchical) should be substantially lower than ωt, because the BFI is designed to measure five distinct factors, not a single general factor. A large ωt/ωh gap confirms that the total score captures mostly factor-specific variance and that subscale scores are more interpretable than a total score.
Split-half reliability should vary across random splits, illustrating why a single split-half estimate is unstable. The Spearman-Brown corrected estimate should approximate α for essentially tau-equivalent scales.
SEM should be in the range of 2–4 raw score points (on a 25-item scale scored 1–5), translating to 95% confidence intervals of ±4–8 points around individual scores.
In the 5-factor CFA addendum, standardised loadings should mostly exceed .50, Composite Reliability (CR) should generally exceed .70, and AVE should be strongest for the clearest factors, with weaker domains occasionally falling just below .50 despite acceptable CR.

What to look for in the output:

The α vs. ωt comparison: how much does tau-equivalence violation matter for this dataset?
The ωh value: does a total BFI score make sense, or should subscales be reported separately?
The SEM-based confidence intervals: how wide are they? Are they wide enough to change practical decisions (e.g., classification)?
In the SEM addendum: do CR and AVE support convergent validity, and do Fornell-Larcker / HTMT suggest acceptable discriminant validity across the Big Five factors?

What we learn from analysing the dataset:

Part A — Data (BFI, N = 2,436 complete cases): After listwise deletion (87.0% retention) and reverse-scoring of 7 items (A1, C4, C5, E1, E2, O2, O5), the mean inter-item correlation is 0.093 — very low, confirming the strong multidimensionality of the 25-item set.

Part B — Cronbach’s α (full scale): Full-scale α = 0.698 [95% CI: 0.678, 0.716]. This value is substantially below the 0.80 threshold typically expected for individual-level use, but this is a diagnostic success, not a failure: α applied to a multidimensional scale is expected to be low and is not a meaningful indicator of measurement quality. The psych::alpha() function correctly flags items N1–N5 and O4 as negatively correlated with the first principal component — because Neuroticism points in the opposite direction from the other four factors at the total-score level. The α-if-item-deleted analysis confirms this: N4 (r_corrected = −0.106) and N5 (r_corrected = 0.013) would increase full-scale α if removed. This is the textbook scenario where full-scale α is inappropriate and factor-specific analysis is required.

Part C — McDonald’s ω: ωt = 0.879 (Schmid-Leiman), dramatically higher than α = 0.698. The gap of 0.181 — the largest in this project — reflects both tau-equivalence violation and the inappropriate application of α to multidimensional data. ωh = 0.475, yielding ωh/ωt = 0.54: the general factor accounts for just over half the reliable variance, confirming that subscale scores are more interpretable than a total score but that a general factor is not negligible.

CFA model fit is below conventional thresholds: CFI = 0.782, TLI = 0.754, RMSEA = 0.078, SRMR = 0.075. This is a well-documented limitation of the BFI-25: 5 items per factor is too few for strict CFA fit criteria (Soto & John, 2017). The model is adequate for demonstrating the reliability workflow but would require modification (e.g., correlated residuals) for publication-grade CFA.

Part D — Factor-specific reliability: Per-factor ωt ranges from 0.608 (Openness) to 0.820 (Neuroticism). Neuroticism and Extraversion (ω ≥ 0.77) are adequate for individual- level interpretation; Agreeableness and Conscientiousness (ω ≈ 0.73– 0.74) are borderline; Openness (ω = 0.608) falls short. The α–ω gap is minimal per factor (0.000–0.016), indicating that tau-equivalence violation is modest within each factor — the large full-scale gap is driven by multidimensionality, not by loading heterogeneity.

Part E — Standard Error of Measurement: SEM (ω-based) ranges from 2.37 (Agreeableness) to 2.57 (Extraversion), translating to 95% CI half-widths of 4.64–5.04 points on a 5–30 scale. This represents approximately ±17–20% of the scale range — substantial uncertainty for individual decisions.

Part H — SEM Addendum (CR, AVE, discriminant validity): CR values confirm the factor-specific ωt pattern (CR ≡ ωt, as expected). All AVE values fall below the 0.50 threshold (range: 0.257–0.485), indicating weak convergent validity — a known property of short personality inventories. The Fornell-Larcker criterion is violated for Agreeableness–Extraversion (latent r = 0.683 > √AVE for both factors), suggesting these constructs are not fully discriminable with 5 items each. HTMT values are all below 0.85, with the highest being A–E (0.646) — passing the conservative threshold despite the Fornell-Larcker violation.

0.3 Setup

# ── Core packages ──────────────────────────────────────────────────────────────
library(psych)       # BFI data + alpha, omega, splitHalf
library(lavaan)      # CFA-based omega
library(ggplot2)
library(dplyr)
library(tidyr)
library(tibble)
library(knitr)
library(purrr)

# ── Global plot theme ──────────────────────────────────────────────────────────
theme_set(
  theme_minimal(base_size = 11) +
    theme(
      plot.title    = element_text(face = "bold", size = 13),
      plot.subtitle = element_text(colour = "grey40", size = 10),
      legend.position = "bottom"
    )
)

palette_rel <- c("#2C3E50", "#E74C3C", "#3498DB", "#2ECC71", "#F39C12")

set.seed(2026)

1 PART A — Data Preparation

1.1 Load and inspect the BFI

data("bfi", package = "psych")

# ── Select the 25 personality items (drop age, gender, education) ────────────
item_names <- c(paste0("A", 1:5), paste0("C", 1:5), paste0("E", 1:5),
                paste0("N", 1:5), paste0("O", 1:5))
bfi_items <- bfi[, item_names]

# ── Handle missing data (listwise for simplicity) ───────────────────────────
n_complete <- sum(complete.cases(bfi_items))
bfi_complete <- na.omit(bfi_items)

cat(sprintf("BFI dataset: %d respondents, %d personality items\n",
            nrow(bfi), length(item_names)))

BFI dataset: 2800 respondents, 25 personality items

cat(sprintf("Complete cases: %d (%.1f%%)\n",
            n_complete, 100 * n_complete / nrow(bfi)))

Complete cases: 2436 (87.0%)

cat(sprintf("Response scale: 1 (Very Inaccurate) to 6 (Very Accurate)\n\n"))

Response scale: 1 (Very Inaccurate) to 6 (Very Accurate)

# ── Factor structure (for subscale analyses) ─────────────────────────────────
factor_map <- tibble(
  item   = item_names,
  factor = rep(c("Agreeableness", "Conscientiousness", "Extraversion",
                  "Neuroticism", "Openness"), each = 5),
  factor_short = rep(c("A", "C", "E", "N", "O"), each = 5)
)

kable(
  factor_map |> group_by(factor) |> summarise(items = paste(item, collapse = ", ")),
  caption = "BFI factor structure (5 items per factor)"
)

BFI factor structure (5 items per factor)
factor	items
Agreeableness	A1, A2, A3, A4, A5
Conscientiousness	C1, C2, C3, C4, C5
Extraversion	E1, E2, E3, E4, E5
Neuroticism	N1, N2, N3, N4, N5
Openness	O1, O2, O3, O4, O5

1.2 Reverse-score items

Several BFI items are reverse-keyed. We reverse-score them so that higher values consistently indicate more of the trait.

# ── BFI reverse-keyed items (from psych documentation) ───────────────────────
# A1 (-), C4 (-), C5 (-), E1 (-), E2 (-), O2 (-), O5 (-)
reverse_items <- c("A1", "C4", "C5", "E1", "E2", "O2", "O5")

bfi_scored <- bfi_complete
for (item in reverse_items) {
  bfi_scored[[item]] <- 7 - bfi_scored[[item]]  # Reverse on 1–6 scale
}

cat(sprintf("Reverse-scored items: %s\n", paste(reverse_items, collapse = ", ")))

Reverse-scored items: A1, C4, C5, E1, E2, O2, O5

cat(sprintf("After reverse scoring, all items keyed in the positive direction.\n"))

After reverse scoring, all items keyed in the positive direction.

# ── Quick sanity check: inter-item correlations should now be mostly positive ─
avg_r <- mean(cor(bfi_scored, use = "pairwise.complete.obs")[lower.tri(cor(bfi_scored, use = "pairwise.complete.obs"))])
cat(sprintf("Mean inter-item correlation (after reverse scoring): %.3f\n", avg_r))

Mean inter-item correlation (after reverse scoring): 0.093

2 PART B — Cronbach’s Alpha

2.1 Full-scale alpha

# ── Full-scale alpha with item diagnostics ───────────────────────────────────
alpha_full <- psych::alpha(bfi_scored, check.keys = FALSE)

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

cat(sprintf("=== Full-scale BFI (25 items, N = %d) ===\n\n", nrow(bfi_scored)))

=== Full-scale BFI (25 items, N = 2436) ===

cat(sprintf("Cronbach's α (raw):          %.3f\n", alpha_full$total$raw_alpha))

Cronbach's α (raw):          0.698

cat(sprintf("Cronbach's α (standardised): %.3f\n", alpha_full$total$std.alpha))

Cronbach's α (standardised): 0.719

cat(sprintf("Guttman's λ6:                %.3f\n", alpha_full$total$`G6(smc)`))

Guttman's λ6:                0.803

cat(sprintf("Average inter-item r:        %.3f\n", alpha_full$total$average_r))

Average inter-item r:        0.093

cat(sprintf("Signal/Noise ratio:          %.2f\n", alpha_full$total$`S/N`))

Signal/Noise ratio:          2.56

2.1.1 Interpretation

The full-scale α = 0.698 is low but entirely expected:

This is a multidimensional scale. The 25 BFI items measure 5 distinct factors, not one. Applying α to the full set treats them as unidimensional — a violated assumption that depresses α. The average inter-item r of 0.093 confirms this: items from different factors barely correlate.
The N-item warnings are correct. psych::alpha() flags N1–N5 and O4 as negatively correlated with the first principal component. This is because Neuroticism (high = anxious, unstable) points conceptually opposite to Agreeableness, Conscientiousness, Extraversion, and Openness (high = positive trait). Our reverse- scoring (A1, C4, C5, E1, E2, O2, O5) is correct within each factor — we should NOT additionally reverse N items, because within the Neuroticism subscale the items are correctly aligned.
Guttman’s λ6 = 0.803 is substantially higher than α, because it uses squared multiple correlations (SMC) rather than average inter-item r, making it more robust to multidimensionality.
Pedagogical value: This is the canonical example of when α misleads. A naive interpretation (“α < 0.70, the scale is unreliable”) would be wrong. The factor-specific analysis (Part D) will show that individual factors are adequately reliable.

2.2 Alpha-if-item-deleted

# ── Item diagnostics table ───────────────────────────────────────────────────
item_diag <- tibble(
  item           = rownames(alpha_full$alpha.drop),
  alpha_if_drop  = alpha_full$alpha.drop$raw_alpha,
  r_corrected    = alpha_full$item.stats$r.drop,
  r_total        = alpha_full$item.stats$r.cor
) |>
  left_join(factor_map, by = "item")

kable(
  item_diag |> select(item, factor, r_corrected, alpha_if_drop) |> arrange(factor, item),
  digits = 3,
  caption = "Item diagnostics: corrected item-total correlation and alpha-if-deleted"
)

Item diagnostics: corrected item-total correlation and alpha-if-deleted
item	factor	r_corrected	alpha_if_drop
A1	Agreeableness	0.138	0.698
A2	Agreeableness	0.440	0.676
A3	Agreeableness	0.452	0.673
A4	Agreeableness	0.295	0.685
A5	Agreeableness	0.395	0.678
C1	Conscientiousness	0.333	0.683
C2	Conscientiousness	0.366	0.680
C3	Conscientiousness	0.258	0.689
C4	Conscientiousness	0.277	0.687
C5	Conscientiousness	0.214	0.693
E1	Extraversion	0.303	0.684
E2	Extraversion	0.323	0.682
E3	Extraversion	0.472	0.671
E4	Extraversion	0.376	0.678
E5	Extraversion	0.472	0.671
N1	Neuroticism	0.051	0.707
N2	Neuroticism	0.067	0.705
N3	Neuroticism	0.094	0.703
N4	Neuroticism	-0.106	0.720
N5	Neuroticism	0.013	0.711
O1	Openness	0.317	0.685
O2	Openness	0.106	0.702
O3	Openness	0.419	0.677
O4	Openness	0.124	0.698
O5	Openness	0.187	0.694

Alpha-if-item-deleted for each BFI item. The dashed line shows the full-scale alpha. Items above this line reduce reliability when included.

# ── Plot alpha-if-deleted ────────────────────────────────────────────────────
ggplot(item_diag, aes(x = reorder(item, alpha_if_drop), y = alpha_if_drop,
                       fill = factor)) +
  geom_col(alpha = 0.85, width = 0.7) +
  geom_hline(yintercept = alpha_full$total$raw_alpha,
             linetype = "dashed", colour = "grey40") +
  annotate("text", x = 1.5, y = alpha_full$total$raw_alpha + 0.005,
           label = sprintf("Full α = %.3f", alpha_full$total$raw_alpha),
           size = 3, colour = "grey40", hjust = 0) +
  scale_fill_manual(values = c("Agreeableness" = palette_rel[1],
                                "Conscientiousness" = palette_rel[2],
                                "Extraversion" = palette_rel[3],
                                "Neuroticism" = palette_rel[4],
                                "Openness" = palette_rel[5])) +
  coord_flip() +
  labs(
    title    = "Alpha-if-Item-Deleted",
    subtitle = "Items above the dashed line reduce full-scale reliability",
    x = NULL, y = "Alpha if Item Deleted", fill = "Factor"
  )

2.2.1 Interpretation

The alpha-if-deleted diagnostic identifies items that weaken the overall scale:

Items above the dashed line would increase α if removed. In a multidimensional scale like the BFI, this is expected — items from one factor may correlate weakly with items from other factors.
Low corrected item-total correlations (< 0.20) suggest items that are weakly related to the total score. However, in a multidimensional context, this does not mean the item is “bad” — it may be an excellent marker of its own factor while correlating poorly with the other four.
This is precisely the scenario where α is misleading: applying α to a multidimensional scale treats all 25 items as measuring one construct, which they do not. Factor-specific reliability (Part D) will provide a more accurate picture.

In view of the analysis of the dataset:

The alpha-if-deleted diagnostic confirms the multidimensionality problem:

N4 (r_corrected = −0.106) is the only item with a negative corrected item-total correlation. Removing it would increase α from 0.698 to 0.720. This does not mean N4 is a bad item — it is the strongest evidence that full-scale α is the wrong metric. N4 measures Neuroticism well (CFA loading = 0.573) but correlates negatively with the total because the total is dominated by the other four factors.
All N items (r_corrected: 0.013–0.094) and O2, O4 (r_corrected: 0.106, 0.124) show weak or near-zero item-total correlations — consistent with items that are strong within their own factor but orthogonal to the general trend.
The strongest items for the total score are E3 and E5 (r_corrected ≈ 0.47), A2, A3 (r_corrected ≈ 0.44–0.45) — items from factors that happen to correlate positively with the first principal component.

2.3 Bootstrap confidence interval for α

# ── Bootstrap alpha ──────────────────────────────────────────────────────────
n_boot <- 1000
alpha_boot <- numeric(n_boot)

for (b in 1:n_boot) {
  idx <- sample(nrow(bfi_scored), replace = TRUE)
  alpha_boot[b] <- psych::alpha(bfi_scored[idx, ], check.keys = FALSE)$total$raw_alpha
}

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( N1 N2 N3 N4 N5 O4 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

alpha_ci <- quantile(alpha_boot, probs = c(0.025, 0.975))

cat(sprintf("Bootstrap 95%% CI for α: [%.3f, %.3f]\n", alpha_ci[1], alpha_ci[2]))

Bootstrap 95% CI for α: [0.678, 0.716]

cat(sprintf("Bootstrap SE: %.4f\n", sd(alpha_boot)))

Bootstrap SE: 0.0094

cat(sprintf("Point estimate: %.3f\n", alpha_full$total$raw_alpha))

Point estimate: 0.698

3 PART C — McDonald’s Omega

3.1 Omega via Schmid-Leiman decomposition

The psych::omega() function estimates ωt and ωh by first extracting a factor solution, then applying the Schmid-Leiman transformation to decompose each item’s variance into general factor and group factor components.

# ── Omega with 5 group factors ──────────────────────────────────────────────
omega_result <- psych::omega(bfi_scored, nfactors = 5, fm = "ml",
                              plot = FALSE, n.obs = nrow(bfi_scored))

cat("=== McDonald's Omega (Schmid-Leiman) ===\n\n")

=== McDonald's Omega (Schmid-Leiman) ===

cat(sprintf("Omega total (ωt):        %.3f\n", omega_result$omega.tot))

Omega total (ωt):        0.879

cat(sprintf("Omega hierarchical (ωh): %.3f\n", omega_result$omega_h))

Omega hierarchical (ωh): 0.475

cat(sprintf("Cronbach's alpha:        %.3f\n", omega_result$alpha))

Cronbach's alpha:        0.823

cat(sprintf("\nωt − α = %.3f (tau-equivalence violation effect)\n",
            omega_result$omega.tot - omega_result$alpha))


ωt − α = 0.056 (tau-equivalence violation effect)

cat(sprintf("ωh / ωt = %.2f (proportion of reliable variance due to general factor)\n",
            omega_result$omega_h / omega_result$omega.tot))

ωh / ωt = 0.54 (proportion of reliable variance due to general factor)

3.2 Omega via CFA (lavaan)

For a confirmatory approach, we specify the 5-factor CFA model and compute ω from the standardised loadings.

# ── Specify 5-factor CFA model ──────────────────────────────────────────────
cfa_model <- '
  A =~ A1 + A2 + A3 + A4 + A5
  C =~ C1 + C2 + C3 + C4 + C5
  E =~ E1 + E2 + E3 + E4 + E5
  N =~ N1 + N2 + N3 + N4 + N5
  O =~ O1 + O2 + O3 + O4 + O5
'

fit_cfa <- cfa(cfa_model, data = bfi_scored, estimator = "MLR")

cat("=== CFA Model Fit ===\n")

=== CFA Model Fit ===

fit_indices <- fitMeasures(fit_cfa, c("chisq", "df", "pvalue",
                                       "cfi", "tli", "rmsea",
                                       "rmsea.ci.lower", "rmsea.ci.upper",
                                       "srmr"))
print(round(fit_indices, 3))

         chisq             df         pvalue            cfi            tli 
      4165.467        265.000          0.000          0.782          0.754 
         rmsea rmsea.ci.lower rmsea.ci.upper           srmr 
         0.078          0.076          0.080          0.075

# ── Extract standardised loadings ────────────────────────────────────────────
std_loadings <- standardizedSolution(fit_cfa) |>
  filter(op == "=~") |>
  select(lhs, rhs, est.std)

cat("\n=== Standardised Factor Loadings ===\n")


=== Standardised Factor Loadings ===

kable(std_loadings, digits = 3, col.names = c("Factor", "Item", "Loading"))

Factor	Item	Loading
A	A1	0.344
A	A2	0.648
A	A3	0.749
A	A4	0.510
A	A5	0.687
C	C1	0.551
C	C2	0.592
C	C3	0.546
C	C4	0.702
C	C5	0.620
E	E1	0.564
E	E2	0.699
E	E3	0.627
E	E4	0.703
E	E5	0.553
N	N1	0.825
N	N2	0.803
N	N3	0.721
N	N4	0.573
N	N5	0.503
O	O1	0.564
O	O2	0.418
O	O3	0.724
O	O4	0.233
O	O5	0.461

# ── Compute omega total per factor from CFA ──────────────────────────────────
compute_omega_factor <- function(loadings_vec, error_vars) {
  (sum(loadings_vec))^2 / ((sum(loadings_vec))^2 + sum(error_vars))
}

omega_per_factor <- std_loadings |>
  group_by(lhs) |>
  summarise(
    n_items     = n(),
    mean_lambda = mean(est.std),
    sum_lambda  = sum(est.std),
    sum_lambda2 = sum(est.std^2),
    .groups     = "drop"
  ) |>
  mutate(
    omega_t = sum_lambda^2 / (sum_lambda^2 + n_items - sum_lambda2)
  )

kable(
  omega_per_factor |> select(lhs, n_items, mean_lambda, omega_t),
  digits = 3,
  col.names = c("Factor", "Items", "Mean λ", "ωt (CFA)"),
  caption = "Factor-specific omega total from CFA standardised loadings"
)

Factor-specific omega total from CFA standardised loadings
Factor	Items	Mean λ	ωt (CFA)
A	5	0.588	0.732
C	5	0.602	0.741
E	5	0.629	0.767
N	5	0.685	0.820
O	5	0.480	0.608

3.2.1 Interpretation

The comparison of α and ω reveals key insights about the BFI’s measurement properties:

ωt > α: This confirms that tau-equivalence is violated — item loadings are unequal within factors. Alpha underestimates the true reliability when some items are stronger indicators than others.
ωh << ωt: The hierarchical omega is much lower than total omega. This means most of the reliable variance is captured by the five group factors, not by a general “personality” factor. A total BFI score (summing all 25 items) is poorly justified — the subscale scores carry most of the interpretable information.
The ωh/ωt ratio: Values below 0.50 indicate that less than half the reliable variance is due to the general factor. This is the key criterion for deciding between total scores and subscale scores (Reise et al., 2013).
CFA-based ω per factor: This provides the most accurate reliability estimate for each subscale, because it accounts for unequal loadings without the tau-equivalence assumption.

Our results :

ωt > α, but the gap varies by level of analysis. At the full-scale level, ωt (0.879) exceeds α (0.698) by 0.181 — driven primarily by multidimensionality. At the factor level, the gap shrinks to 0.000–0.016, indicating that within each factor, loadings are relatively homogeneous and tau-equivalence is approximately met. This pattern is maximally informative: it demonstrates that the α–ω discrepancy is not a universal constant but depends on whether the analysis is conducted at the appropriate level of aggregation.
ωh = 0.475, ωh/ωt = 0.54. The general factor captures just over half the reliable variance. This is a boundary case: the ratio is above the 0.50 threshold sometimes cited for justifying a total score, but only marginally. In practice, given the clear 5-factor structure and the poor full-scale α, subscale scores remain preferable for substantive interpretation, even if a weak general factor exists.
CFA fit is below conventional thresholds (CFI = 0.782, TLI = 0.754, RMSEA = 0.078, SRMR = 0.075). This is a known limitation of the BFI with only 5 items per factor — the strict proportional constraints of a simple-structure CFA are too rigid for personality items that often have minor cross-loadings. This does not invalidate the reliability analysis (ω is robust to moderate misfit) but means the CFA-derived indices (CR, AVE in Part H) should be interpreted with caution.
Factor-specific ωt (CFA) ranges from 0.608 (Openness) to 0.820 (Neuroticism). Openness is the weakest factor, driven by O4 (λ = 0.233) — an item that may be poorly understood by respondents or that taps a different facet of Openness than the other items.

4 PART D — Factor-Specific Reliability

4.1 Per-factor alpha and omega comparison

# ── Compute alpha per factor ────────────────────────────────────────────────
factor_names <- c("A", "C", "E", "N", "O")
factor_labels <- c("Agreeableness", "Conscientiousness", "Extraversion",
                    "Neuroticism", "Openness")

alpha_by_factor <- purrr::map_dfr(seq_along(factor_names), function(i) {
  items <- paste0(factor_names[i], 1:5)
  a <- psych::alpha(bfi_scored[, items], check.keys = FALSE)
  tibble(
    factor = factor_labels[i],
    alpha  = a$total$raw_alpha,
    mean_r = a$total$average_r
  )
})

# ── Merge with CFA omega ────────────────────────────────────────────────────
factor_rel <- alpha_by_factor |>
  left_join(
    omega_per_factor |> transmute(factor = case_when(
      lhs == "A" ~ "Agreeableness",
      lhs == "C" ~ "Conscientiousness",
      lhs == "E" ~ "Extraversion",
      lhs == "N" ~ "Neuroticism",
      lhs == "O" ~ "Openness"
    ), omega_t = omega_t),
    by = "factor"
  ) |>
  mutate(
    omega_alpha_diff = omega_t - alpha
  )

kable(
  factor_rel,
  digits = 3,
  caption = "Factor-specific reliability: α vs. ωt"
)

Factor-specific reliability: α vs. ωt
factor	alpha	mean_r	omega_t	omega_alpha_diff
Agreeableness	0.716	0.345	0.732	0.016
Conscientiousness	0.737	0.364	0.741	0.004
Extraversion	0.765	0.395	0.767	0.002
Neuroticism	0.817	0.473	0.820	0.003
Openness	0.608	0.241	0.608	0.000

Reliability coefficients by BFI factor. Alpha (circles) vs. Omega total (triangles). The gap reveals tau-equivalence violation per factor.

# ── Plot comparison ──────────────────────────────────────────────────────────
rel_plot_df <- factor_rel |>
  select(factor, alpha, omega_t) |>
  pivot_longer(-factor, names_to = "metric", values_to = "value") |>
  mutate(metric = ifelse(metric == "alpha", "Cronbach's α", "McDonald's ωt"))

ggplot(rel_plot_df, aes(x = factor, y = value, shape = metric, colour = metric)) +
  geom_point(size = 4) +
  geom_line(aes(group = metric), linetype = "dashed", alpha = 0.4) +
  geom_hline(yintercept = 0.70, linetype = "dotted", colour = "grey50") +
  annotate("text", x = 0.6, y = 0.705, label = "0.70 threshold",
           size = 3, colour = "grey50", hjust = 0) +
  scale_colour_manual(values = c("Cronbach's α" = "#3498DB",
                                  "McDonald's ωt" = "#E74C3C")) +
  scale_shape_manual(values = c("Cronbach's α" = 16, "McDonald's ωt" = 17)) +
  ylim(0.55, 0.95) +
  labs(
    title    = "Factor-Specific Reliability: α vs. ω",
    subtitle = "Gap between markers = tau-equivalence violation effect",
    x = NULL, y = "Reliability Coefficient",
    colour = NULL, shape = NULL
  )

4.1.1 Interpretation

The factor-specific comparison reveals:

Which factors are best measured: Factors with α and ω ≥ 0.80 provide scores suitable for individual-level interpretation. Factors below 0.70 should be interpreted cautiously at the individual level (though they may still be adequate for group comparisons).
Where α misleads most: Factors with the largest ω − α gap have the most heterogeneous loadings. For these, reporting α alone understates the scale’s reliability.
Practical recommendations:
- Report ω (not α) as the primary reliability coefficient.
- If ω < 0.70 for a factor, consider whether the 5 items adequately represent the construct or whether additional items are needed.
- Report the SEM alongside reliability to communicate measurement precision in score-scale units (see Part E).

Taking into account the results of our analysis:

The factor-specific comparison yields the project’s key insight:

Neuroticism (ω = 0.820, α = 0.817) is the most reliably measured factor, with the highest mean loading (0.685) and smallest loading heterogeneity. The α–ω gap of 0.003 confirms near-perfect tau-equivalence within this subscale.
Extraversion (ω = 0.767) and Conscientiousness (ω = 0.741) are adequate for group-level research (≥ 0.70) but fall short of the 0.80 threshold for individual decisions.
Agreeableness (ω = 0.732) is pulled down by A1 (λ = 0.344), which is notably weaker than the other A items (λ = 0.510–0.749).
Openness (ω = 0.608) is the clear outlier, below 0.70 for both α and ω. The problem is O4 (λ = 0.233) and to a lesser extent O2 (λ = 0.418). With 5 items, even one weak item substantially depresses reliability. If O4 were removed, the 4-item Openness ω would likely exceed 0.65.
The α–ω gap is negligible within each factor (0.000–0.016). This confirms that tau-equivalence holds approximately at the subscale level — the large full-scale discrepancy is entirely a multidimensionality artefact.

5 PART E — Standard Error of Measurement

5.1 SEM and confidence intervals

# ── Compute SEM per factor ───────────────────────────────────────────────────
# SEM = SD × sqrt(1 − reliability)
# Using omega as the reliability estimate

factor_sem <- factor_rel |>
  mutate(
    SD = map_dbl(seq_along(factor_names), function(i) {
      items <- paste0(factor_names[i], 1:5)
      sd(rowSums(bfi_scored[, items]))
    }),
    SEM_alpha = SD * sqrt(1 - alpha),
    SEM_omega = SD * sqrt(1 - omega_t),
    CI95_half_alpha = 1.96 * SEM_alpha,
    CI95_half_omega = 1.96 * SEM_omega
  )

kable(
  factor_sem |> select(factor, SD, alpha, omega_t, SEM_alpha, SEM_omega,
                        CI95_half_alpha, CI95_half_omega),
  digits = 2,
  col.names = c("Factor", "SD", "α", "ωt", "SEM (α)", "SEM (ω)",
                 "95% CI half-width (α)", "95% CI half-width (ω)"),
  caption = "Standard Error of Measurement by factor"
)

Standard Error of Measurement by factor
Factor	SD	α	ωt	SEM (α)	SEM (ω)	95% CI half-width (α)	95% CI half-width (ω)
Agreeableness	4.57	0.72	0.73	2.43	2.37	4.77	4.64
Conscientiousness	4.81	0.74	0.74	2.47	2.45	4.83	4.80
Extraversion	5.34	0.77	0.77	2.59	2.57	5.07	5.04
Neuroticism	6.00	0.82	0.82	2.57	2.55	5.03	4.99
Openness	4.02	0.61	0.61	2.52	2.52	4.94	4.94

5.2 Score uncertainty visualisation

# ── Hypothetical respondent at each factor mean ──────────────────────────────
mean_scores <- map_dbl(seq_along(factor_names), function(i) {
  items <- paste0(factor_names[i], 1:5)
  mean(rowSums(bfi_scored[, items]))
})

ci_df <- tibble(
  factor     = factor_labels,
  observed   = round(mean_scores, 1),
  ci_lower   = observed - factor_sem$CI95_half_omega,
  ci_upper   = observed + factor_sem$CI95_half_omega,
  SEM        = factor_sem$SEM_omega
)

ggplot(ci_df, aes(x = factor, y = observed)) +
  geom_pointrange(aes(ymin = ci_lower, ymax = ci_upper),
                  colour = "#2C3E50", size = 0.8, linewidth = 0.8) +
  coord_flip() +
  labs(
    title    = "Score Confidence Intervals (95%, SEM-Based)",
    subtitle = "For a respondent at the sample mean on each factor",
    x = NULL, y = "Subscale Score (sum of 5 items, range 5–30)"
  )

Score confidence intervals (95%) for a respondent with mean subscale scores. Band width reflects measurement precision.

5.2.1 Interpretation

The SEM translates abstract reliability coefficients into concrete score-scale units:

A SEM of 2 points on a 5–30 scale means that a score of 20 should be interpreted as “somewhere between 16 and 24” at 95% confidence. This is a ±20% band — non-trivial for individual decisions.
Factors with larger SEM provide less precise measurement. When using these scores for classification or high-stakes decisions, the confidence interval should be reported alongside the point estimate.
Comparison with reliability: A factor with α = 0.80 and SD = 5 has SEM = 2.24; the same SD with α = 0.70 has SEM = 2.74. The 0.10 improvement in reliability translates to ~0.5 points of SEM — meaningful at the individual level.

Our results :

The SEM converts reliability into practical score-unit uncertainty:

SEM ranges from 2.37 to 2.57 across factors — a relatively narrow range, because the factors with higher reliability also have higher SD (the two effects partially cancel in SEM = SD × √(1−r)).
95% CI half-widths of 4.6–5.0 points on a 5–30 scale represent ±16–20% of the scale range. For a respondent scoring 20, the true score plausibly lies between 15 and 25. This band is wide enough to cross category boundaries in most classification systems.
Openness has the same SEM as better-measured factors (2.52) despite lower ω, because its SD (4.02) is the smallest. This illustrates why SEM is more informative than ω alone: a factor can have low reliability but still produce acceptable precision if score variability is also low.
Practical recommendation: For any individual-level decision (screening, selection, feedback), report the confidence interval alongside the point estimate. The SEM makes it impossible to ignore measurement uncertainty.

6 PART F — Split-Half Reliability

6.1 Random and systematic splits

# ── Split-half on Neuroticism (as an example factor) ────────────────────────
n_items <- paste0("N", 1:5)
n_data  <- bfi_scored[, n_items]

# ── Method 1: Odd-even split ────────────────────────────────────────────────
odd_items  <- n_items[c(1, 3, 5)]
even_items <- n_items[c(2, 4)]

r_odd_even <- cor(rowSums(n_data[, odd_items]), rowSums(n_data[, even_items]))
# Spearman-Brown correction (unequal splits)
sb_odd_even <- (2 * r_odd_even) / (1 + r_odd_even)

cat(sprintf("Odd-even split (Neuroticism):\n"))

Odd-even split (Neuroticism):

cat(sprintf("  Half-test correlation: %.3f\n", r_odd_even))

  Half-test correlation: 0.735

cat(sprintf("  Spearman-Brown corrected: %.3f\n\n", sb_odd_even))

  Spearman-Brown corrected: 0.848

# ── Method 2: Random splits (500 iterations) ────────────────────────────────
n_splits <- 500
sh_estimates <- numeric(n_splits)

for (s in 1:n_splits) {
  split1_idx <- sample(1:5, size = 2)  # 2 items in one half
  split2_idx <- setdiff(1:5, split1_idx)  # 3 items in the other

  r_split <- cor(rowSums(n_data[, split1_idx]),
                 rowSums(n_data[, split2_idx]))
  # Spearman-Brown for unequal lengths
  k <- 5 / length(split1_idx)  # full test / half test ratio
  sh_estimates[s] <- (k * r_split) / (1 + (k - 1) * r_split)
}

cat(sprintf("Random split-half (500 splits, Neuroticism):\n"))

Random split-half (500 splits, Neuroticism):

cat(sprintf("  Mean: %.3f\n", mean(sh_estimates)))

  Mean: 0.847

cat(sprintf("  SD:   %.3f\n", sd(sh_estimates)))

  SD:   0.035

cat(sprintf("  Range: [%.3f, %.3f]\n", min(sh_estimates), max(sh_estimates)))

  Range: [0.772, 0.880]

# ── Plot distribution of split-half estimates ────────────────────────────────
sh_df <- tibble(reliability = sh_estimates)

ggplot(sh_df, aes(x = reliability)) +
  geom_histogram(bins = 30, fill = palette_rel[4], alpha = 0.7, colour = "white") +
  geom_vline(xintercept = mean(sh_estimates), linetype = "dashed", colour = "grey30") +
  geom_vline(xintercept = alpha_by_factor$alpha[4], linetype = "dotted",
             colour = "#3498DB", linewidth = 0.8) +
  annotate("text", x = alpha_by_factor$alpha[4] + 0.01,
           y = Inf, vjust = 2, hjust = 0,
           label = sprintf("α = %.3f", alpha_by_factor$alpha[4]),
           colour = "#3498DB", size = 3) +
  labs(
    title    = "Split-Half Reliability Distribution (Neuroticism)",
    subtitle = sprintf("500 random splits — Mean = %.3f, SD = %.3f",
                        mean(sh_estimates), sd(sh_estimates)),
    x = "Spearman-Brown Corrected Reliability", y = "Count"
  )

Distribution of split-half reliability estimates across 500 random splits (Neuroticism subscale). The spread illustrates why a single split-half estimate is unreliable.

6.1.1 Interpretation

The split-half analysis demonstrates a key limitation of the method:

Split-half reliability varies by partition. With 5 items, there are only \(\binom{5}{2} = 10\) possible splits, but even this small number produces a range of estimates. The arbitrary choice of partition introduces instability — this is why α (the mean of all possible split-halves for standardised items) was introduced as a more stable alternative.
Spearman-Brown correction compensates for the shorter half-test length, projecting half-test reliability to full-test reliability. The correction assumes parallel halves (equal means and variances), which may not hold for all splits.
Mean split-half ≈ α: For essentially tau-equivalent items, the average across all possible splits converges to Cronbach’s α. Discrepancies indicate heterogeneous loadings (tau-equivalence violation) — consistent with the α/ω comparison.

7 PART G — Reliability Comparison Summary

summary_table <- tibble(
  Coefficient = c(
    "Cronbach's α (full scale, 25 items)",
    "Cronbach's α (standardised)",
    "Guttman's λ6",
    "McDonald's ωt (total, Schmid-Leiman)",
    "McDonald's ωh (hierarchical)",
    "ωh / ωt ratio",
    "Average inter-item r",
    sprintf("Split-half mean (N scale, %d splits)", n_splits)
  ),
  Estimate = c(
    sprintf("%.3f [%.3f, %.3f]", alpha_full$total$raw_alpha, alpha_ci[1], alpha_ci[2]),
    sprintf("%.3f", alpha_full$total$std.alpha),
    sprintf("%.3f", alpha_full$total$`G6(smc)`),
    sprintf("%.3f", omega_result$omega.tot),
    sprintf("%.3f", omega_result$omega_h),
    sprintf("%.2f", omega_result$omega_h / omega_result$omega.tot),
    sprintf("%.3f", alpha_full$total$average_r),
    sprintf("%.3f", mean(sh_estimates))
  ),
  Interpretation = c(
    "Internal consistency under tau-equivalence assumption",
    "Alpha on standardised (z-scored) items",
    "Lower bound based on SMC; more robust than α",
    "Total reliable variance (allows unequal loadings)",
    "Variance due to general factor only",
    "< 0.50 → subscale scores preferred over total score",
    "Low due to multidimensionality (5 factors in 25 items)",
    "Unstable — depends on partition"
  )
)

kable(summary_table, caption = "Comprehensive reliability comparison — BFI (25 items)")

Comprehensive reliability comparison — BFI (25 items)
Coefficient	Estimate	Interpretation
Cronbach’s α (full scale, 25 items)	0.698 [0.678, 0.716]	Internal consistency under tau-equivalence assumption
Cronbach’s α (standardised)	0.719	Alpha on standardised (z-scored) items
Guttman’s λ6	0.803	Lower bound based on SMC; more robust than α
McDonald’s ωt (total, Schmid-Leiman)	0.879	Total reliable variance (allows unequal loadings)
McDonald’s ωh (hierarchical)	0.475	Variance due to general factor only
ωh / ωt ratio	0.54	< 0.50 → subscale scores preferred over total score
Average inter-item r	0.093	Low due to multidimensionality (5 factors in 25 items)
Split-half mean (N scale, 500 splits)	0.847	Unstable — depends on partition

7.0.1 Key takeaways

Report ω, not just α. The ωt estimate accounts for unequal loadings and is the more appropriate measure of composite reliability. Alpha can be reported as a familiar reference but should not be the sole indicator.
Total score vs. subscale scores. The ωh/ωt ratio is the decisive statistic. If ωh is low relative to ωt (< 0.50), the total score captures mostly factor-specific variance and is not recommended for interpretation. Report and interpret subscale scores instead.
Always report the SEM. Reliability coefficients are abstractions; the SEM translates them into practical score-unit terms that stakeholders can understand. A score of 20 ± 4 (95% CI) communicates uncertainty more directly than “α = 0.80.”
Context matters. Reliability ≥ 0.80 is often cited as a threshold, but this depends on the decision stakes. For individual clinical or selection decisions, ≥ 0.90 is recommended (Nunnally & Bernstein, 1994). For group-level research, ≥ 0.70 may suffice.

Specifically, regarding our analysis data :

Full-scale α is misleading for multidimensional scales. The BFI α = 0.698 is not evidence of poor measurement — it is evidence that α is the wrong statistic. ωt = 0.879 tells a completely different (and correct) story when multidimensionality is modelled.
The α–ω gap is context-dependent. At the full-scale level, the gap is massive (0.181); at the factor level, it is negligible (0.000–0.016). Always match the reliability analysis to the level of score interpretation.
ωh/ωt = 0.54 is a boundary value. It marginally supports a general factor, but the combination of low full-scale α, clear 5-factor structure, and factor-specific ωt patterns all point toward reporting and interpreting subscale scores, not a total score.
The SEM is the deliverable. Reliability coefficients are methodological tools; the SEM is what matters for stakeholders. A score of 20 ± 5 (95% CI) is immediately interpretable.
One weak item can undermine a short scale. O4 (λ = 0.233) drags Openness reliability to 0.608. With only 5 items, there is no redundancy to absorb a weak indicator — item quality control is critical for short-form instruments.

The reliability evidence accumulated in Parts B–G addresses the question “how consistently does this instrument measure?” from a classical and omega-based perspective. But in contemporary SEM-based research, reliability and validity are often evaluated jointly through the measurement model itself. Part H extends the analysis into this framework, demonstrating how the same CFA model fitted in Part C yields Composite Reliability, convergent validity (AVE), and discriminant validity (Fornell-Larcker, HTMT) — metrics that are standard reporting requirements in fields using SEM (Cheung et al., 2024; Hair et al., 2022).

A key insight that will emerge: CR and ωt are algebraically equivalent under the congeneric model (Raykov, 1997; Peterson & Kim, 2013). This is not a coincidence — both coefficients estimate the same quantity (the proportion of composite variance due to the common factor), just derived from different computational routes. Part H’s CR should therefore converge with Part D’s factor-specific ωt, and any discrepancy would signal a computation error rather than a substantive finding.

8 PART H — SEM Measurement-Model Addendum

8.1 Reliability and validity in SEM (Structural Equation Modeling)

Important disambiguation: in Part E, SEM referred to the Standard Error of Measurement. In this addendum, SEM refers to Structural Equation Modeling.

This section addresses the user’s linked question directly: beyond α, ω, and the score-level SEM, researchers often evaluate measurement-model quality in a reflective CFA/SEM framework using:

Composite Reliability (CR) — a congeneric reliability coefficient based on the factor loadings and residual variances. In practice, this is conceptually much closer to ω than to α.
Average Variance Extracted (AVE) — the average proportion of item variance explained by the latent factor; commonly used as an index of convergent validity.
Fornell-Larcker checks — compare each factor’s √AVE with its correlations with other factors.
HTMT (heterotrait-monotrait ratio) — a complementary screen for discriminant validity.

These criteria are intended for reflective measurement models. They are not the right diagnostics for formative indicators.

library(semPlot)

semPaths(fit_cfa,
         what       = "std",        # standardised estimates
         whatLabels = "std",        # display standardised values
         layout     = "tree2",      # tree layout, factors on top
         style      = "lisrel",     # LISREL convention (ovals + rectangles)
         residuals  = TRUE,         # show residual variances
         intercepts = FALSE,        # omit intercepts
         nCharNodes = 3,            # abbreviate node labels to 3 chars
         sizeMan    = 5,            # observed variable box size
         sizeLat    = 9,            # latent variable oval size
         edge.label.cex = 0.65,     # edge label size
         fade       = FALSE,        # no fading by weight
         color      = list(lat = "#3498DB", man = "#ECF0F1"),
         border.color = "#2C3E50",
         edge.color = "#2C3E50",
         mar        = c(2, 2, 2, 2))

Standardised path diagram for the BFI 5-factor CFA. Ovals = latent factors; rectangles = observed items; single-headed arrows = standardised loadings; double-headed curves = factor correlations; short arrows = residual variances.

8.2 CR, AVE, Fornell-Larcker, and HTMT

# ── Standardised loadings and residual variances from the CFA model ──────────
std_sol_sem <- standardizedSolution(fit_cfa)

loading_df <- std_sol_sem |>
  filter(op == "=~") |>
  transmute(factor_code = lhs, item = rhs, lambda = est.std)

# ── Residual variances from standardised solution ────────────────────────────
# Under a standardised congeneric model: θ_ii = 1 − λ_i²
# This is algebraically equivalent to extracting diag(lavInspect(fit_cfa, "std")$theta)
# but avoids dependency on lavInspect's internal list structure, which can vary
# across lavaan versions.
resid_df <- loading_df |>
  transmute(item = item, theta = 1 - lambda^2)

factor_key <- tibble(
  factor_code = c("A", "C", "E", "N", "O"),
  factor = c("Agreeableness", "Conscientiousness", "Extraversion",
             "Neuroticism", "Openness")
)

# ── Construct-level SEM quality indices ──────────────────────────────────────
sem_quality <- loading_df |>
  left_join(resid_df, by = "item") |>
  group_by(factor_code) |>
  summarise(
    n_items      = n(),
    mean_loading = mean(lambda),
    min_loading  = min(lambda),
    CR           = (sum(lambda)^2) / ((sum(lambda)^2) + sum(theta)),
    AVE          = sum(lambda^2) / (sum(lambda^2) + sum(theta)),
    .groups      = "drop"
  ) |>
  left_join(factor_key, by = "factor_code") |>
  mutate(
    sqrt_AVE     = sqrt(AVE),
    loading_flag = ifelse(min_loading >= 0.50, "All λ ≥ .50", "Some λ < .50"),
    CR_flag      = ifelse(CR >= 0.70, "OK (≥ .70)", "Low (< .70)"),
    AVE_flag     = ifelse(AVE >= 0.50, "OK (≥ .50)", "Low (< .50)")
  ) |>
  select(factor, n_items, mean_loading, min_loading, CR, AVE, sqrt_AVE,
         loading_flag, CR_flag, AVE_flag)

kable(
  sem_quality,
  digits = 3,
  col.names = c("Factor", "Items", "Mean λ", "Min λ", "CR", "AVE", "√AVE",
                "Loadings", "CR check", "AVE check"),
  caption = "SEM addendum: composite reliability and convergent validity indices"
)

SEM addendum: composite reliability and convergent validity indices
Factor	Items	Mean λ	Min λ	CR	AVE	√AVE	Loadings	CR check	AVE check
Agreeableness	5	0.588	0.344	0.732	0.367	0.605	Some λ < .50	OK (≥ .70)	Low (< .50)
Conscientiousness	5	0.602	0.546	0.741	0.366	0.605	All λ ≥ .50	OK (≥ .70)	Low (< .50)
Extraversion	5	0.629	0.553	0.767	0.400	0.633	All λ ≥ .50	OK (≥ .70)	Low (< .50)
Neuroticism	5	0.685	0.503	0.820	0.485	0.696	All λ ≥ .50	OK (≥ .70)	Low (< .50)
Openness	5	0.480	0.233	0.608	0.257	0.507	Some λ < .50	Low (< .70)	Low (< .50)

# ── Fornell-Larcker matrix: diagonal = √AVE, off-diagonal = latent correlations ─
latent_cor <- lavInspect(fit_cfa, "cor.lv")
fl_matrix <- latent_cor
diag(fl_matrix) <- sem_quality$sqrt_AVE[match(colnames(fl_matrix), factor_key$factor_code)]
dimnames(fl_matrix) <- list(factor_key$factor, factor_key$factor)

kable(
  round(fl_matrix, 3),
  caption = "Fornell-Larcker matrix (diagonal = √AVE; off-diagonal = latent correlations)"
)

Fornell-Larcker matrix (diagonal = √AVE; off-diagonal = latent correlations)
	Agreeableness	Conscientiousness	Extraversion	Neuroticism	Openness
Agreeableness	0.605	0.334	0.683	-0.223	0.303
Conscientiousness	0.334	0.605	0.357	-0.283	0.301
Extraversion	0.683	0.357	0.633	-0.244	0.453
Neuroticism	-0.223	-0.283	-0.244	0.696	-0.112
Openness	0.303	0.301	0.453	-0.112	0.507

# ── HTMT from observed item correlations ─────────────────────────────────────
item_blocks <- list(
  Agreeableness     = paste0("A", 1:5),
  Conscientiousness = paste0("C", 1:5),
  Extraversion      = paste0("E", 1:5),
  Neuroticism       = paste0("N", 1:5),
  Openness          = paste0("O", 1:5)
)

R_items <- cor(bfi_scored, use = "pairwise.complete.obs")

compute_htmt <- function(items_x, items_y, R) {
  heterotrait <- abs(R[items_x, items_y])
  monotrait_x <- abs(R[items_x, items_x][lower.tri(R[items_x, items_x])])
  monotrait_y <- abs(R[items_y, items_y][lower.tri(R[items_y, items_y])])

  mean(heterotrait) / sqrt(mean(monotrait_x) * mean(monotrait_y))
}

constructs <- names(item_blocks)
htmt_mat <- matrix(NA_real_,
                   nrow = length(constructs),
                   ncol = length(constructs),
                   dimnames = list(constructs, constructs))
diag(htmt_mat) <- NA

for (i in seq_along(constructs)) {
  for (j in seq_along(constructs)) {
    if (i < j) {
      val <- compute_htmt(item_blocks[[i]], item_blocks[[j]], R_items)
      htmt_mat[i, j] <- val
      htmt_mat[j, i] <- val
    }
  }
}

kable(
  round(htmt_mat, 3),
  caption = "HTMT matrix (values < .85 are conservative evidence for discriminant validity; < .90 is a looser rule)"
)

HTMT matrix (values < .85 are conservative evidence for discriminant validity; < .90 is a looser rule)
	Agreeableness	Conscientiousness	Extraversion	Neuroticism	Openness
Agreeableness	NA	0.351	0.646	0.247	0.272
Conscientiousness	0.351	NA	0.369	0.295	0.345
Extraversion	0.646	0.369	NA	0.299	0.469
Neuroticism	0.247	0.295	0.299	NA	0.265
Openness	0.272	0.345	0.469	0.265	NA

8.2.1 Interpretation

This SEM-based addendum adds a measurement-model lens to the earlier reliability analysis:

CR ≡ ω under the congeneric model. The Composite Reliability formula — CR = (Σλ)² / [(Σλ)² + Σθ] — is algebraically identical to McDonald’s ωt when both are computed from the same CFA standardised solution (Raykov, 1997; Peterson & Kim, 2013). Any convergence between Parts D and H is therefore expected and confirms computational consistency. The practical implication: researchers can report either CR or ω (or both for audiences from different traditions), knowing they estimate the same reliability parameter.
Convergent validity via CR and AVE. The conventional thresholds are CR ≥ .70 and AVE ≥ .50, alongside standardised loadings mostly ≥ .50 (preferably ≥ .70) (Fornell & Larcker, 1981; Hair et al., 2022). AVE can be slightly conservative for short scales (5 items per factor), so a factor may show acceptable CR yet borderline AVE. That pattern usually means the construct is measured reliably enough overall, but some indicators are only moderately strong — a common finding with personality inventories where items sample diverse behavioural facets within a broad trait.
Discriminant validity — two complementary checks:
- Fornell-Larcker criterion: the diagonal entries (√AVE) should exceed the off-diagonal latent correlations in each row and column. Violations indicate that two constructs share more variance with each other than each captures from its own indicators — a threat to construct distinctness.
- HTMT: values below .85 provide conservative evidence for discriminant validity; below .90 is sometimes used when constructs are conceptually adjacent (Henseler et al., 2015). HTMT is generally considered a more sensitive criterion than Fornell-Larcker, especially when loadings are heterogeneous (Voorhees et al., 2016).
Integration with earlier findings. The Big Five factors are theoretically distinct but empirically correlated — especially Agreeableness–Conscientiousness and Extraversion–Agreeableness. If some Fornell-Larcker or HTMT checks are borderline for these pairs, this is a known property of the BFI rather than a measurement failure. The pattern should be interpreted in light of the substantive theory (personality dimensions are correlated, not orthogonal) and the brevity of the scale (5 items per factor limits the achievable AVE).
Best-practice caveat. No single index should be used as a dichotomous pass/fail gate. Modern guidance recommends combining CR, loadings, AVE, and discriminant-validity checks, ideally supplemented by bootstrap confidence intervals for HTMT and by examining the pattern holistically rather than relying on mechanical thresholding (Cheung et al., 2024).

Thus, SEM-based addendum adds a measurement-model lens to the earlier reliability analysis, and the results highlight the known limitations of the BFI-25:

CR ≡ ωt confirmed. The CR values match the CFA-based ωt to three decimal places (e.g., Neuroticism CR = 0.820, ωt = 0.820), confirming the algebraic identity under the congeneric model (Raykov, 1997; Peterson & Kim, 2013). This is not a coincidence but a mathematical necessity.
Convergent validity is weak. All five AVE values fall below the 0.50 threshold (range: 0.257–0.485). This means that, on average, each latent factor captures less than half the variance of its own indicators — the error variance exceeds the explained variance. This is a known property of short personality scales: items are designed to sample diverse behavioural facets within a broad trait, so they correlate moderately (mean r ≈ 0.25–0.47) rather than strongly. The CR ≥ 0.70 threshold is still met for 4 of 5 factors, creating the classic pattern “acceptable CR but low AVE” that signals reliable composites built from individually moderate indicators.
Openness fails both CR (0.608) and AVE (0.257). The combination of O4 (λ = 0.233) and O2 (λ = 0.418) depresses both metrics below any reasonable threshold. This factor would need revision — either replacing O4 or adding items — before use in an SEM context.
Discriminant validity — mixed evidence:
- Fornell-Larcker violation for Agreeableness–Extraversion: the latent correlation (0.683) exceeds √AVE for both factors (0.605 and 0.633). This pair is not fully discriminable with 5 items per construct — a known concern with Big Five measures where A and E share variance through prosocial/interpersonal content.
- HTMT is clean. The highest HTMT is A–E (0.646), well below the conservative 0.85 threshold. All other pairs are below 0.50. This illustrates why HTMT is often preferred over Fornell-Larcker: it is less sensitive to low AVE values that inflate the diagonal (Henseler et al., 2015; Voorhees et al., 2016).
Best-practice caveat. The poor CFA fit (CFI = 0.782) means all SEM-derived indices are conditioned on a model that does not fit well. In a publication context, one would either (a) improve the model (correlated residuals, exploratory SEM) before computing CR/AVE, or (b) report these indices with an explicit caveat about model fit. For this portfolio demonstration, the pattern is informative because it shows how to interpret imperfect results — a more valuable skill than reporting textbook-perfect numbers.

8.2.2 Practical reporting template

A compact SEM-style write-up for this dataset would look like:

For the reflective 5-factor CFA model, we evaluated construct reliability and validity using Composite Reliability (CR), Average Variance Extracted (AVE), the Fornell-Larcker criterion, and HTMT. CR values ≥ .70 were treated as acceptable evidence of construct reliability; AVE ≥ .50 and salient standardised loadings (≥ .50) as evidence of convergent validity; and discriminant validity was examined via √AVE-versus-latent-correlation comparisons and HTMT (< .85 conservative, < .90 liberal). These SEM-based indices complement the α, ω, and Standard Error of Measurement reported in Parts B–F, and are appropriate for reflective latent-variable models where both reliability and construct validity must be demonstrated jointly. —

9 Technical Summary

cat("─── Reproducibility Information ───\n")

─── Reproducibility Information ───

cat(sprintf("Date: %s\n", Sys.Date()))

Date: 2026-04-07

cat(sprintf("R version: %s\n", R.version.string))

R version: R version 4.4.1 (2024-06-14 ucrt)

cat(sprintf("psych version: %s\n", packageVersion("psych")))

psych version: 2.4.6.26

cat(sprintf("lavaan version: %s\n", packageVersion("lavaan")))

lavaan version: 0.6.19

cat(sprintf("Platform: %s\n", sessionInfo()$platform))

Platform: x86_64-w64-mingw32/x64

9.1 References

Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024). Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pacific Journal of Management, 41, 745–783.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412.
Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484–501.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.
Hair, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S. (2022). Evaluation of Reflective Measurement Models. In Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R (2nd ed.). Springer.
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135.
McDonald, R. P. (1999). Test Theory: A Unified Treatment. Lawrence Erlbaum Associates.
McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). McGraw-Hill.
Peterson, R. A., & Kim, Y. (2013). On the relationship between coefficient alpha and composite reliability. Journal of Applied Psychology, 98(1), 194–198.
Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173–184.
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140.
Revelle, W., & Condon, D. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31(12), 1395–1411.
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120.
Voorhees, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016). Discriminant validity testing in marketing: An analysis, causes for concern, and proposed remedies. Journal of the Academy of Marketing Science, 34(1), 119–134.