Initial Psychometric Validation of a Self-Report Measure of the Symptoms of Mild Hypoxic Hypoxia

Hypoxia is the absence of an adequate supply of oxygen in the arterial and capillary blood, resulting in a rapid deterioration of body function, especially in the central nervous system. In turn, hypoxic hypoxia results from reduced alveolar oxygen tension in the lungs caused by low oxygen partial pressure at altitude. Whereas the physiological and psychological effects of hypoxia at altitudes higher than 15000 ft (4572 m) are well understood,¹ the effects of unprotected exposure to hypoxic hypoxia in altitudes up to 14000 ft (4267 m) are less well known. As such, considerable research has focused on the effects of mild hypoxic hypoxia on physiological and psychological functioning in altitudes less than 14000 ft, with mixed results. The inconsistent pattern of findings across studies may be due to many factors, including subtlety of alterations compared to higher altitudes, heterogenous central nervous system response, individual differences in compensatory mechanisms, duration of exposure, presence/absence of exercise, and variation in the methods and measures used to quantify the effects of hypoxic hypoxia.^2,3 The aim of this study is to improve reliability across studies by developing a psychometrically valid self-report scale for the symptoms of mild hypoxia that can be used consistently by researchers going forward—ideally in conjunction with other valid measures of physiological and psychological functioning. This will likely contribute to a reduction in methodological variability in future studies.

Typically, previous studies assessing the effects of mild hypoxic hypoxia (hereafter mild hypoxia) on aircrew performance have included combinations of physiological and psychological measures. For example, in our laboratory, we have assessed the physiological effects of mild hypoxia, via cerebral regional and finger pulse oxyhemoglobin saturation levels and heart and respiration rates, and its psychological effects using computerized cognitive tasks and self-reports of mood, fatigue, and symptoms of mild hypoxia.^4,5 The use of a multitude of measures is predicated on the assumption that the relatively subtle effects of mild hypoxia may be difficult to capture using a single measure, and that a broad set of measures is more likely to provide a comprehensive picture of its effects on human functioning and performance. Sound as this assumption may be, the confidence that one might have about the findings will nevertheless depend on the reliability and validity of the individual measures incorporated into a study design. One family of measures that is frequently included involves self-report measures of the symptoms of mild hypoxia, essentially instructing aircrew to report their subjective experiences of its effects on functioning and performance. Critically, there has been a wide range of symptoms that have been assessed across studies, including dizziness, lightheadedness, mental confusion, tingling in fingers/toes, visual impairment, apprehension, cyanosis, euphoria, fatigue, headache, hot/cold flashes, increased heart rate/palpitations, increased respiration, muscle weakness, nausea, numbness, tetany (i.e., involuntary muscle twitching), changes in personality, difficulty with mathematical calculations, impaired judgment, impaired memory/recall, slips or lapses in aircraft operating procedures, difficulty with communications, impaired manual dexterity, and slowed reaction time, among others.^6–8 Nevertheless, despite this variability, certain symptoms have been reported more consistently across studies than others. For example, Deussing et al. found that the three most commonly recorded hypoxia symptoms were tingling (54%), difficulty concentrating (32%), and dizziness (30%).⁶ The most commonly-reported symptoms by Nishi et al. included a feeling of warmth, poor concentration, and diminished vision.⁷ In turn, the five most common symptoms observed by Woodrow et al. were lightheaded/dizzy, dizziness, mental confusion, visual impairment, and tingling.⁸ Overall, these findings suggest that aircrew are most likely to report symptoms that reflect problems in the cognitive (e.g., poor concentration, mental confusion) and sensory (e.g., dizziness, poor vision) domains, although difficulties can extend to other domains and exhibit considerable individual differences.

A particularly important series of studies were conducted to explore not only the symptoms reported by aircrew, but also the veracity of the core methodological assumptions that underlie our interpretations of retrospective self-report data about episodes characterized by mild hypoxia. Specifically, Smith administered a survey to Australian Army helicopter aircrew who had operated at altitudes up to 10,000 ft (3048 m), which revealed that the most common symptoms associated with mild hypoxia were difficulty with calculations (45%), feeling light-headed (38%), delayed reaction time (38%), and mental confusion (36%).⁹ That study also found that the relative impact of symptoms that fell into the cognitive, psychomotor, vision, and behavioral categories remained the same regardless of whether aircrew were asked to indicate if they had experienced the features themselves (i.e., “symptoms”) or whether they were aware of their presence in colleagues (i.e., “observations”). Smith concluded that “Rather than providing authoritative data, this study should be seen as a preliminary survey which identifies the need for a structured, objective, rigorous scientific study to evaluate the potential for helicopter aircrew to experience operationally significant hypoxia below 10,000 ft, and to consider the emphasis that is given to it during aviation medicine training of helicopter aircrew” (pp. 797–798).⁹ In a follow-up study conducted in Saudi Arabia, Smith collected survey data on the symptoms of mild hypoxia from aircrew on two occasions—first during aviation physiology training at the beginning of the hypoxia lecture, and again after hypoxia awareness training.¹⁰ In Saudi Arabia the hypoxia awareness training is administered to all aircrew as part of a 5-d aviation physiology course during initial aircrew training, followed at 3-yr intervals by a 2-d refresher course. When aircrew completed the survey at the beginning of the hypoxia lecture, they indicated the presence and severity of the symptoms they remembered from their previous hypoxia training. In turn, when they completed the survey after hypoxia awareness training, they indicated the presence and severity of symptoms they had experienced during the training just completed. The key finding was that there was a great deal of agreement between the symptoms reported during experience of acute hypoxia and those they remembered from training up to 3 yr earlier, such as impairment of cognitive function (e.g., poor concentration), psychomotor impairment (e.g., slowing of responses), visual impairment (e.g., blurred vision), and psychological disturbance (e.g., anxiety). Critically, this study demonstrated that aircrew can be relied upon to “remember the symptoms they experience when hypoxic; that the symptom-complex aircrew attribute to hypoxia reflects the symptoms they actually experience during hypoxia; and that the accuracy with which aircrew remember their hypoxia signature does not decline during the interval between refresher training courses” (pp. 54–55)—even up to 3 yr later.¹⁰

The literature above suggests that aircrew do experience subjective symptoms associated with mild hypoxia, and that they are able to recall and report them accurately. However, across studies, researchers have used variable instruments for assessing the symptoms of mild hypoxia, relying on surveys generated for specific contexts with unknown psychometric properties. We believe that this methodological variability is one of the sources that has contributed to the inconsistency of findings on the effects of mild hypoxia on psychological functioning.^2,3 Our review of this literature suggests that the variety of reported symptoms can be largely grouped into three dimensions: cognitive (e.g., impaired memory, impaired judgment), sensory (e.g., blurred vision, dizziness), and affective (e.g., euphoria, fatigue).

The aim of the present study was to develop and take the initial steps in the psychometric validation of a novel self-report measure of the symptoms of mild hypoxia. We hypothesized that the range of symptoms would be represented by a three-dimensional structure (i.e., cognitive, sensory, affective), tested via factor analysis and reliability analysis. If successful, the adoption of a psychometrically valid scale for self-reporting of the symptoms of mild hypoxia should reduce some of the measurement variability that has plagued studies in this area in the past.³

METHODS

Subjects

The protocol for this study was approved by Defense Research and Development Canada’s Human Research Ethics Committee (protocol #2018-074). The subjects were recruited via an e-mail distributed by the Royal Canadian Air Force (RCAF) Association of Canada to its members. As stated on its website, the mission statement of the RCAF Association is to serve as “a national aerospace and community service organization established to commemorate the noble achievements of the men and women who have served as members of Canada’s air force since its inception, advocate for a proficient and well-equipped air force, and support the Royal Canadian Air Cadet program.” As such, its membership consists of active and retired members of RCAF, and represented an ideal opportunity to survey a large number of active and retired aircrew with flight experience to probe the effects of mild hypoxia on cognitive, sensory, and affective symptoms. In cases where the recipients of the solicitation e-mail agreed to participate in a study on the “signs and symptoms of mild hypoxia,” they were directed to a link to complete the survey anonymously.

Materials

The survey consisted of 15 items—5 items each for assessing the symptoms associated with the cognitive, sensory, and affective aspects of mild hypoxia (Appendix A, found in the online version of this article). The specific items themselves were selected by the authors from previous studies that had examined the self-report symptoms associated with mild hypoxia, reviewed above. For each item (i.e., symptom), the subjects were asked to indicate the extent to which they had experienced it while performing their duties as aircrew in aircraft with unpressurized cabins using a 5-point scale ranging from 1 (Never) to 5 (Always). The survey also included a series of demographic questions for characterizing the sample. The demographics of the sample appear in Table I.

Table I. Demographics.

View Fullscreen

Procedure

The survey was administered online via LimeSurvey. Informed consent was obtained from all subjects prior to data collection.

Statistical Analysis

We performed an exploratory factor analysis using maximum likelihood estimation in the MPlus software (Version 8.4, Los Angeles, CA, United States) to identify any problematic items and to determine the underlying factor structure of the measure. We chose maximum likelihood estimation for the added benefit of the various fit indices that it produces, which are useful in guiding factor structure selection. Due to elevated levels of skewness (>2) and kurtosis (>7) in certain items (Table II), we used the robust maximum likelihood estimator within MPlus, which compensates for nonnormal data when conducting maximum likelihood estimation. Additionally, observing the findings that oblique rotation (counterintuitively) produces better simple structure, and that uncorrelated factors are commonly theoretically implausible,¹¹ we specified an oblique rotation to allow factors to correlate freely with one another. Subsequent to exploratory factor analysis, we carried out a reliability analysis on each subscale using the Statistical Package for the Social Sciences (Version 28, IBM, Armonk, NY, United States) to determine the cohesiveness of associated items.

Table II. Item Descriptive Statistics.

View Fullscreen

RESULTS

In terms of factor rotation, exploratory factor analysis was carried out using maximum likelihood estimation with oblique rotation. Given that our measure contained 15 items, we extracted 1-, 2-, and 3-factor solutions to allow for comparison of the largest number of factored solutions while still ensuring a sufficient number of potential items for any given factor.¹² Results indicated that not only did the 3-factor solution produce better fit indices [root mean square error of approximation (RMSEA) = 0.044, comparative fit index (CFI) = 0.976, standardized root mean square residual (SRMR) = 0.028] than the 1-factor (RMSEA = 0.123, CFI = 0.736, SRMR = 0.088) and 2-factor (RMSEA = 0.065, CFI = 0.938, SRMR = 0.039) solutions, but it also met conventional criteria of absolute model fit,¹³ whereas the other two models fell somewhat short.

In further refining the 3-factor solution, we sought to strike a balance between empirical plausibility and factor interoperability. Problematic items were identified as: 1) items that did not have salient loadings on any factor; 2) items that loaded onto more than one factor; and 3) items whose loadings on a factor did not make theoretical sense. We considered a salient loading as above 0.30.¹⁴ After identifying and removing an item, we re-evaluated the pattern of loadings and identified any additional items for removal. Undertaking this process led us to remove three items in the following order: euphoria, depressed mood, and impaired memory. “Euphoria” did not show salient loading onto any factor (loadings ranged from 0.05–0.28); “depressed mood” loaded onto the wrong factor (i.e., sensory rather than affective); and “impaired memory” showed small salient loadings onto two factors: cognitive (0.427) and sensory (0.418). The remaining 12 items displayed simple structure in forming three factors (Table III). The first factor contained items that we had hypothesized would form a cognitive factor: confusion, slowed reaction time, difficulty concentrating, and impaired judgment. These items reflect symptoms associated with cognitive impairments related to mild hypoxia. Similarly, the second factor contained five items (dizziness/light-headedness, hot/cold flashes, increased heart rate/palpitation/respiration, headache, and vision problems) that we had hypothesized would form a sensory factor. Finally, the three items of the third factor—fatigue, nervousness, and stress—formed the hypothesized affective factor. Finally, factor correlations were 0.438 between the cognitive and affective factors, 0.723 between cognitive and sensory factors, and 0.572 between the sensory and affective factors.

Table III. Loadings for the Cognitive (Factor 1), Sensory (Factor 2), and Affective (Factor 3) Factors.

View Fullscreen

Subsequent to performing the Exploratory Factor Analysis, we undertook a reliability analysis to ensure that items formed internally consistent and homogenous factors. The cognitive factor showed an internal consistency reliability of 0.90. Interitem correlations ranged from 0.649–0.788. Three of the four items (i.e., slowed reaction time, difficulty concentrating, and impaired judgment) showed corrected item-total correlations in the low 0.80 range, while the remaining item (i.e., confusion) showed a corrected item-total correlation in the 0.7 range (Table IV). The sensory factor had an internal consistency reliability of 0.75. Interitem correlations ranged from 0.343–0.500. Four items (hot/cold flashes, increased heart rate/palpitation/respiration, headache, and vision problems) showed corrected item-total correlations in the 0.50 range, whereas one item (i.e., dizziness/light-headedness) showed a corrected item-total correlation of 0.49 (Table IV). The affective factor had an internal consistency reliability of 0.85. Interitem correlations ranged from 0.590–0.684. All corrected item-total correlations fell in the range between 0.70–0.77 (Table IV).

Table IV. Corrected Item Total Correlations and Item Endorsements.

View Fullscreen

DISCUSSION

This study was conducted to test the psychometric properties of a new self-report measure of the symptoms of mild hypoxia in a large sample of active and retired aircrew (N = 354) with substantial years of military service (M = 25.04 yr, SD = 11.27). Our review of the literature on hypoxic hypoxia showed that although many past studies had probed its symptoms, there was to date no psychometrically validated self-report instrument for measuring such symptoms. The development of such an instrument had been motivated in the literature,⁹ at least in part as a potentially useful approach to reducing some of the methodological variability that has plagued past research.^1–3 Our review of the literature suggested further that most self-reported symptoms tend to fall into three categories: cognitive, sensory, and affective. As such, to fill this gap in the literature, we developed a 15-item scale based largely on items used in past studies to probe the symptoms of mild hypoxia and subjected the data to exploratory factor analysis, followed by reliability analysis on each subscale to determine cohesiveness of associated items. The results provided initial psychometric validation for a 12-item scale that conforms to our predicted three-dimensional structure (cognitive, sensory, affective) (Tables II–IV).

Our factor structure is consistent with reports from the literature. For example, cognitive impairments have been one of the most consistent findings reported by aircrew in relation to mild hypoxia. A systematic meta-regression analysis of the effects of acute hypoxia on cognitive performance, including data from 22 experiments, demonstrated a moderate, negative mean effect size (Hodges g = −0.49, 95% CI −0.64 to −0.34, P < 0.001).¹⁵ Consistent with McMorris et al.,¹⁵ we included elementary (e.g., reaction time) and higher-order cognitive (e.g., judgment) symptoms under the common umbrella of cognitive effects. The justification for this was twofold: first, McMorris et al.’s meta-analysis revealed no difference between executive and nonexecutive tasks, indicating that there is no empirical reason to separate them.¹⁵ Second, from a theoretical perspective, one can view the tasks ranging from psychomotor vigilance to executive functions as representing a continuum of tasks that cover elementary to higher-order cognition, all of which have been shown to be impacted by mild hypoxia.¹ The next most commonly reported set of symptoms appear to be sensory in nature, in particular difficulties with vision, dizziness, tingling, and hot/cold flashes, among others. Finally, the third set of symptoms reported by aircrew are affective in nature, representing mood and emotional states, including nervousness and stress.^3,9,10 Indeed, experimental work from our laboratory has demonstrated that acute hypoxic hypoxia can increase the subjective experience of negative mood.^4,5

Our study had several limitations. First, although we followed statistical best practices for the exploratory factor analysis and reliability analysis,^11–13 further validation work is necessary for the maturation of this instrument, including additional psychometric research (e.g., confirmatory factor analysis in an independent sample of aircrew), as well as validation in situ (i.e., assessing the sensitivity of the instrument following exposure to various levels of altitude). Second, the majority (67%) of the survey responders were 61 yr of age or older, and not active members of the military at time of data collection (Table I). Also, they were asked to consider their history of flight rather than a specific episode of flight. These are important considerations regarding the accuracy of their retrospective memory recall in relation to the symptoms of mild hypoxia, although there is also evidence to suggest that aircrew can recognize and recall the symptoms of mild hypoxia accurately up to 3 yr after experiencing them.¹⁰ This concern can be addressed by investigating the structure of this survey in active aircrew directly after completing a flight session in the future. Despite these limitations, this instrument offers a promising tool for measuring the symptoms of mild hypoxia going forward, negating the need to develop and administer ad hoc questionnaires. Finally, as noted at the outset, the current standard in studies of mild hypoxia is to use a comprehensive set of physiological and psychological measures to capture the full breadth of its effects^4,5 and we see this instrument as one of the components of this multifactorial approach to assessing the effects of this multifaceted phenomenon.