People with schizophrenia generally show various clinical symptoms, such as paranoid delusional thoughts, auditory hallucinations, and disorganized thinking, according to the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; American Psychiatric Association, 2013). Impaired recognition of facial emotions, which contributes to poor social functioning (Kohler et al., 2010; McCleery et al., 2015), is also common among people with schizophrenia. Facial expressions that show emotion play an important role in interpersonal communication as they convey important information, such as a person’s mental state or disposition toward the social interaction (Frith, 2009).

To date, many investigations have been made into processing of emotional faces among people with schizophrenia, and most scholars have focused on recognition and memory of facial expressions (see, e.g., Shah et al., 2018; Turetsky et al., 2007). These researchers have found that individuals with schizophrenia show impairments in facial emotion recognition that are stable across different stages of the disorder and are already present at the time of the first episode. It has been shown that positive facial expressions (e.g., happiness) are recognized much faster than emotionally neutral and negative facial expressions (e.g., sadness, anger, or disgust), which has been labeled the positive classification advantage (PCA; see, e.g., Leppänen & Hietanen, 2004; Liu et al., 2013). Indeed, when participants are tasked with making a simple two-choice classification so researchers can test for a PCA, a distinct mechanism from recognition and memory of facial expressions has been reflected in most previous studies (e.g., Leppänen & Hietanen, 2004; Liu et al., 2013). Face classification is based on visual information that is similar to all facial action patterns, irrespective of the faces that are producing them, and the expression classification processes for faces include the extraction of attributes of expressions (Ganel & Goshen-Gottstein, 2002). To our knowledge, no study has been conducted using schematic faces that are presented both the right way up and inverted, to explore the PCA in a group of people with schizophrenia.

It is widely accepted that processing faces relies on both local and configural perceptual processes (e.g., Ganel & Goshen-Gottstein, 2002; Leder & Carbon, 2006; Maurer et al., 2002). Local processing mostly refers to distinct circumscribed characteristics of the face, such as the mouth or the nose. General spatial relations of the face (e.g., the eyes are above the nose) are usually described as configural information or first-order relations, whereas second-order relations refer to specific spatial relations (e.g., distance between eyes and nose) and involve a higher discriminative value (Leder & Carbon, 2006). Evidence has accumulated that the configural analysis underlying face recognition also applies to facial emotion recognition, and it has been shown that face inversion, which is relevant to the configural processing of faces, impairs recognition of sad, fearful, angry, and disgusted expressions, but not recognition of happy expressions (Calder et al., 2000; McKelvie, 1995). Recently, evidence has emerged that the PCA disappears for inverted faces, indicating that the configural computation could be an important source of the PCA and that this computation method is applied by default when categorizing facial emotions (Song et al., 2017).

It has been shown that dysfunction in recognizing faces among people with schizophrenia is attributable to specific deficits in facial configural processing (Soria Bauser et al., 2012). In contrast, in one study conducted with people who had been diagnosed with schizophrenia, the findings show that although the participants were impaired in facial emotion discrimination for faces presented the right way up, there was no difference between this group and a control group for inverted faces, indicating that the dysfunction of configural processing was not a source of the deficit in processing facial expressions among people with schizophrenia (Chambon et al., 2006).

Our experiment was designed to directly test the configural processing of emotional faces of people with schizophrenia by studying the inversion effect of faces in an expression categorization task. It has been shown that schematic faces also elicit face-sensitive robust negative deflection in a wave amplitude that occurs in the occipitotemporal brain area (N170) of event-related potentials (Sagiv & Bentin, 2001) around 170 ms after presentation, with a significantly larger amplitude being recorded for faces relative to other object categories. This suggests that schematic faces are valuable and reliable for studying brain activity relevant to facial expressions (Wright et al., 2002).

Recent evidence suggests that the processing of task-irrelevant schematic facial expressions is impaired in people with schizophrenia (Yin et al., 2018). Thus, in this study we used schematic emotional faces to minimize the variance associated with facial photographs of real people. We presented a set of schematic emotional faces to participants with the faces both the right way up and inverted. If there is a dysfunction of categorization of emotional faces, people with schizophrenia should exhibit an attenuated PCA. If a deficit in the PCA is, indeed, because of configural processing, then face inversion should not disturb the PCA in individuals with schizophrenia.

Method

Participants

The participants were 20 people being treated for schizophrenia (10 women and 10 men; M_age = 34.6, SD = 12.2) and 24 age-matched control participants (12 women and 12 men; M_age = 33.9, SD = 12.5). Participants in the former group had been diagnosed with schizophrenia according to the symptoms set out in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; American Psychiatric Association), and were without a history of severe medical or severe neurological disorders. Control group participants had no history of any major psychiatric disorders or major physical illnesses, and were not taking any medications that affect the nervous system.

Two people in the group with schizophrenia had to be excluded from the analysis because they did not finish the experiment. Hence, our final sample comprised 18 people with schizophrenia and 24 control participants.

Stimuli

To avoid the low-level processing of facial features, we designed each emotional category to consist of 20 different schematic face models created by manipulating the distance between facial features and the shape of those features (Liu et al., 2013; Song et al., 2017). The schematic stimuli are shown in Figure 1. Three blocks of 120 trials each (faces presented the right way up: 20 neutral, 20 happy, 20 sad; inverted faces: 20 neutral, 20 happy, 20 sad) were included in this experiment, with a short break between each block. All stimuli were randomly presented at the center of a video monitor and viewed from a distance of 100 cm with a visual angle of approximately 7.27° × 6.06°.

Procedure

This study was approved by the institutional review board of Taiyuan Brain Hospital. All participants received payment for taking part in the research and gave their informed consent prior to beginning the experiment.

The participants were seated in a dimly lit and sound-attenuated booth, and were asked to categorize the expression of each face by pressing correspondingly labeled buttons on a computer keyboard as quickly and accurately as possible. The labels for the response buttons were counterbalanced across the participants. Each stimulus was presented for 300 ms with an intertrial interval ranging randomly between 600 ms and 800 ms after response onset. The participants each completed one practice sequence of 36 stimuli that was not used in the main experiment.

Data Analysis

The positive, negative, and general psychiatric symptoms of participants were assessed using the Positive and Negative Syndrome Scale (Kay et al., 1987), and their social functioning was assessed with the Personal and Social Performance Scale (Morosini et al., 2000). The demographic and descriptive characteristics of participants are shown in Table 1.

The performance data, including reaction times (RTs) and accuracy rates, were analyzed using a three-way analysis of variance with expression (happy, neutral, sad) and orientation (right way up, inverted) as within-subject factors and group (control, people with schizophrenia) as the between-subjects factor. Degrees of freedom were corrected whenever necessary using the Greenhouse–Geisser epsilon correction factor.

Results

Accuracy Rates

A three-way analysis of variance was performed to assess the percentage of correct responses. Across the two experimental conditions, the group with schizophrenia showed a lower accuracy rate (88.5%) than did the control group (97.5%), F(1, 40) = 9.10, p < .002, η_p² = .191. The main effect of expression was significant, F(2, 80) = 16.20, p < .001, η_p² = .399, showing that neutral faces with neutral expressions were categorized more accurately (96.3%) than both happy (92.1%, p < .01) and sad (90.3%, p < .01) faces, but there was no significant difference between the latter two face conditions (p = .184). The main effect of orientation was also significant, F(1, 40) = 12.10, p < .01, η_p² = .253, showing greater accuracy for the right-way-up (94.2%) than the inverted (91.4%) condition, and this effect was qualified by the two-way interaction of expression × orientation, F(2, 80) = 5.60, p < .03, η_p² = .134, showing that the effects of facial expressions were similar for right-way-up and inverted conditions, but the inversion effect was evident for expressional faces (ps < .01 for happy and sad faces), whereas this effect was not evident for neutral faces (p = .998). We found a significant two-way interaction of expression × group, F(2, 80) = 6.10, p < .01, η_p² = .135, showing that although there were similar group effects for three expression conditions (p < .01), the pattern of expression effects was indeed different in the two groups. Participants in the control group classified the happy faces with an accuracy rate (95.9%) similar to that for the sad faces (96.5%, p = .86), whereas in the group of people with schizophrenia the accuracy rate for categorizing sad faces (83.2%) was significantly lower than that for classifying happy faces (87.9%, p < .01).

In addition, we found a significant three-way interaction of expression × orientation × group, F(2, 80) = 3.81, p < .05, η_p² = .099, and we analyzed this interaction by assessing separately the two-way interactions of expression × orientation for people with schizophrenia and for the control group. In the control group, as previously described, both the main effects of expression (p < .001) and inversion (p < .001) were significant, but the two-way interaction did not reach significance (p = .133). However, in the group of people with schizophrenia the two-way interaction did reach a significant level (p < .05), showing that the expression effect was significant for the inverted condition (p < .02), but not for the right-way-up condition (p = .18), and that the inversion effect was evident for sad faces only (p = .09, p < .05, and p = .36 for happy, sad, and neutral faces, respectively).

Reaction Time

Overall, people with schizophrenia exhibited a slower response speed (933 ms) than the control group participants did (721 ms), F(1, 40) = 25.80, p < .001, η_p² = .353. Across the two groups, face inversion modulated facial classification by expression, F(1, 40) = 13.60, p < .001, η_p² = .255, showing that the response was faster for faces the right way up (778 ms) than faces that were inverted (862 ms). The main effect of expression was significant, F(2, 80) = 14.10, p < .001, η_p² = .378, showing that neutral faces were categorized by expression more quickly (735 ms) than either happy (846 ms, p < .01) or sad (882 ms, p < .001) faces. However, there were no significant differences between the latter two face conditions (p = .189), although sad faces were categorized slightly slower than happy faces were, displaying the PCA.

There was a significant three-way interaction of expression × orientation × group, F(2, 80) = 3.90, p < .05, η_p² = .098, and we analyzed this interaction by assessing separately the two-way interactions of expression × orientation and of orientation × group for happy, sad, and neutral faces, respectively, for people with schizophrenia and for the control group. In the control group, the main effect of expression was significant, F(2, 44) = 40.80, p< .001, η_p² = .576, revealing that neutral faces were categorized by expression more quickly (615 ms) than either happy (748 ms, p < .001) or sad (805 ms, p < .001) faces, and happy faces were categorized faster than sad faces were (p < .01), which provides evidence of the PCA. This effect was also modulated by facial inversion, F(2, 44) = 7.80, p < .01, η_p² = .268, showing that although the control group participants were quickest at classifying the neutral faces, regardless of facial orientation, the PCA was evident for the right-way-up condition (p < .01) but not for the inverted condition (p = .165), and also that facial inversion delayed the response speed for happy (p < .001) and sad (p < .01) faces but did not modulate the RT for neutral faces (p = .477). In the group with schizophrenia, only the main effect of orientation was significant, F(1, 17) = 4.60, p < .05, η_p² = .201, showing a faster response time for faces that were the right way up (872 ms) than for those that were inverted (970 ms). The other effects and interactions did not reach significance (p > .10).

Analysis of the two-way interaction of orientation × group shows that for both sad and happy faces, the inversion effect was similar in the two groups (F < 1.00). However, for neutral faces, the inversion effect was evident in the group with schizophrenia (p < .05) but not in the control group (p > .10), indicating that the people with schizophrenia performed the categorization more slowly when the neutral faces were presented in an inverted (950 ms) rather than right-way-up (774 ms) orientation.

Finally, to investigate the perceptual mechanism of the PCA, we performed a Pearson correlation analysis to assess the relationship between the PCA and the RTs of the control group, and this showed a significant positive correlation between the RT for sad faces and the size of the PCA, r = .66, p < .01 (two-tailed), but not between the RT for happy faces and the size of the PCA, r = .06, p > .10.

Discussion

To our knowledge there have been few studies conducted to directly compare facial classification by expressions between people being treated for schizophrenia and a healthy age-matched control group. From our analysis of RTs, we found that the control group participants classified happy faces faster than they did sad faces, displaying the PCA, and that this PCA disappeared when the faces were inverted. In contrast, the participants with schizophrenia categorized emotional faces more slowly, with less accuracy, and did not show an evident PCA except for an overall delayed response for faces that were inverted rather than the right way up. Although face inversion delayed the categorization of neutral faces in the group with schizophrenia, inversion effects for happy and sad faces did not differ between the two groups.

In line with previous study results (Liu et al., 2013; Song et al., 2017), we found there was a response advantage for the control group in the categorization of happy versus sad faces; however, this PCA disappeared for inverted faces. It has been posited that face inversion impairs the structural features of faces and, thus, disturbs configural processing (Searcy & Bartlett, 1996). Therefore, the faster categorization of happy faces could be accounted for by a possible difference in configural analysis when processing happy and sad faces (see, e.g., Song et al., 2017). However, we did not find a different facial inversion effect between happy and sad faces. In contrast, Leppänen and Hietanen (2004) revealed there is a stronger influence of manipulating configurations for happy than for sad face recognition. The focus in previous studies has mostly been on facial emotion recognition (Chambon et al., 2006; Soria Bauser et al., 2012), whereas in this study we directly investigated the role of configural processing in expression classification. Because the configural processing was indeed task-irrelevant in this study, the fact that the PCA was not present in inversion conditions suggested that the configural analysis of facial expressions could account for the PCA, which individuals apply by default when classifying facial expressions.

We found that neutral faces were categorized more quickly than emotional faces were, which replicates the findings in our previous studies (Liang et al., 2019; Song et al., 2017), but is contrary to other researchers’ previous findings that happy faces are identified faster than emotionally neutral faces are (e.g., Hugdahl et al., 1993). This discrepancy could be because of the different stimuli sets used in the studies; for example, Hugdahl et al. (1993) used real face photographs, whereas we used schematic faces in this study. However, Leppänen and Hietanen (2004) used similar face stimuli to those in the present study and found the recognition speed did not differ between happy and emotionally neutral faces. Because face inversion did not modulate the RT for neutral faces, it is possible that when categorizing schematic neutral faces, respondents may rely on featural processing (e.g., the line representing the mouth in our schematic stimuli) more than on configural processing, resulting in faster categorization for neutral versus emotional faces. In our study people with schizophrenia categorized emotional faces more slowly than those in the control group did, were less accurate in emotion classification, and did not show an evident PCA, implying there is a dysfunction of processing emotional faces in people with schizophrenia.

Indeed, the converging evidence, as reviewed by Kohler et al. (2010), suggests a robust impairment in emotion recognition of faces among people with schizophrenia. Recent electrophysiological studies also show impaired processing of emotional faces in people being treated for schizophrenia, as revealed by a significant decrease in expressional mismatch-negativity (EMMN; Yin et al., 2018). Although we are the first to have investigated face classification by emotion in people with schizophrenia, we believe the absence of the PCA is indeed based on slower responses to happy versus sad faces. That is, greater attentional resources are required for categorizing a face displaying happiness in the group with schizophrenia, indicating a deficit for processing happy faces compared to control participants. Supporting this view, one recent study showed that among people with schizophrenia, EMMN reflecting the automatic processing of emotions elicited by sad faces was evident but EMMN was absent for happy faces, implying greater impairment when processing unexpected positive versus negative emotional faces (Yin et al., 2018).

We were interested to find that the inversion effect for emotional faces did not differ between control group participants and people with schizophrenia, indicating that all participants processed the configural information of the facial expression, at least in the simple classification task we presented. Chambon et al. (2006) used a similar facial emotion recognition task to test the inversion effect of facial expressions and found that participants with schizophrenia tended to adopt the same criterion pattern as a control group did for emotional faces that were either inverted or the right way up. However, there is converging evidence showing that people with schizophrenia exhibit dysfunction in recognizing faces, and that this occurs because of their specific deficits in facial configural processes (see, e.g., Soria Bauser et al., 2012). Therefore, the absence of the PCA in this study cannot be explained by a deficit in configural information processing, and further investigations are needed.

In sum, we have directly investigated whether there is a dysfunction of categorizing emotional faces in people with schizophrenia. Our participants with schizophrenia did not show an evident PCA, but inversion effects for happy and sad faces did not differ between control group members and those with schizophrenia. These data further support the presence of a dysfunction in categorization of expressional information among those with schizophrenia, and reveal important associations between schizophrenia and emotional categorization. Future studies could further clarify the specificity of emotional categorization, which is an important issue with significant implications in clinical settings, in regard to achieving better understanding and treatment of affective disturbances in those being treated for schizophrenia.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Author.
https://doi.org/10.1176/appi.books.9780890425596

Calder, A. J., Young, A. W., Keane, J., & Dean, M. (2000). Configural information in facial expression perception. Journal of Experimental Psychology: Human Perception and Performance, 26(2), 527–551.
https://doi.org/10.1037/0096-1523.26.2.527

Chambon, V., Baudouin, J. Y., & Franck, N. (2006). The role of configural information in facial emotion recognition in schizophrenia. Neuropsychologia, 44(12), 2437–2444.
https://doi.org/10.1016/j.neuropsychologia.2006.04.008

Frith, C. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society of London: Series B, Biological Sciences, 364(1535), 3453–3458.
https://doi.org/10.1098/rstb.2009.0142

Ganel, T., & Goshen-Gottstein, Y. (2002). Perceptual integrality of sex and identity of faces: Further evidence for the single-route hypothesis. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 854–867.
https://doi.org/10.1037/0096-1523.28.4.854

Hugdahl, K., Iversen, P. M., & Johnsen, B. H. (1993). Laterality for facial expressions: Does the sex of the subjects interact with the sex of the stimulus face? Cortex, 29(2), 325–331.
https://doi.org/10.1016/s0010-9452(13)80185-2

Kay S. R., Fiszbein A., & Opler L. A. (1987). The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13(2), 261–276.
https://doi.org/10.1093/schbul/13.2.261

Kohler, C. G., Walker, J. B., Martin, E. A., Healey, K. M., & Moberg, P. J. (2010). Facial emotion perception in schizophrenia: A meta-analytic review. Schizophrenia Bulletin, 36(5), 1009–1019.
https://doi.org/10.1093/schbul/sbn192

Leder, H., & Carbon, C.- C. (2006). Face-specific configural processing of relational information. British Journal of Psychology, 97(1), 19–29.
https://doi.org/10.1348/000712605X54794

Leppänen, J. M., & Hietanen, J. K. (2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research, 69, 22–29.
https://doi.org/10.1007/s00426-003-0157-2

Liang, S., Chen, S., Zhao, L., & Miao, D. (2019). Categorization of emotional faces in schizophrenia patients: An ERP study. Neuroscience Letters, 713, Article 134493.
https://doi.org/10.1016/j.neulet.2019.134493

Liu, X., Liao, Y., Zhou, L., Sun, G., Li, M., & Zhao, L. (2013). Mapping the time course of the positive classification advantage: An ERP study. Cognitive, Affective, & Behavioral Neuroscience, 13, 491–500.
https://doi.org/10.3758/s13415-013-0158-6

Maurer, D., Le Grand, R., & Mondloch, C. J. (2002). The many faces of configural processing. Trends in Cognitive Sciences, 6(6), 255–260.
https://doi.org/10.1016/s1364-6613(02)01903-4

McCleery, A., Lee, J., Joshi, A., Wynn, J. K., Hellemann, G. S., & Green, M. F. (2015). Meta-analysis of face processing event-related potentials in schizophrenia. Biological Psychiatry, 77(2), 116–126.
https://doi.org/10.1016/j.biopsych.2014.04.015

McKelvie, S. J. (1995). Emotional expression in upside-down faces: Evidence for configurational and componential processing. British Journal of Social Psychology, 34(3), 325–334.
https://doi.org/10.1111/j.2044-8309.1995.tb01067.x

Morosini, P. L., Magliano, L., Brambilla, L., Ugolini, S., & Pioli, R. (2000). Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS) to assess routine social functioning. Acta Psychiatrica Scandinavica, 101(4), 323–329.
https://doi.org/10.1034/j.1600-0447.2000.101004323.x

Sagiv, N., & Bentin, S. (2001). Structural encoding of human and schematic faces: Holistic and part-based processes. Journal of Cognitive Neuroscience, 13(7), 937–951.
https://doi.org/10.1162/089892901753165854

Searcy, J. H., & Bartlett, J. C. (1996). Inversion and processing of component and spatial-relational information in faces. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 904–915.
https://doi.org/10.1037/0096-1523.22.4.904

Shah, D., Knott, V., Baddeley, A., Bowers, H., Wright, N., Labelle, A., … Collin, C. (2018). Impairments of emotional face processing in schizophrenia patients: Evidence from P100, N170 and P300 ERP components in a sample of auditory hallucinators. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology, 134, 120–134.
https://doi.org/10.1016/j.ijpsycho.2018.10.001

Song, J., Liu, M., Yao, S., Yan, Y., Ding, H., Yan, T., … Xu, G. (2017). Classification of emotional expressions is affected by inversion: Behavioral and electrophysiological evidence. Frontiers in Behavioral Neuroscience, 11, Article 21.
https://doi.org/10.3389/fnbeh.2017.00021

Soria Bauser, D., Thoma, P., Aizenberg, V., Brüne, M., Juckel, G., & Daum, I. (2012). Face and body perception in schizophrenia: A configural processing deficit? Psychiatry Research, 195(1–2), 9–17.
https://doi.org/10.1016/j.psychres.2011.07.017

Turetsky, B. I., Kohler, C. G., Indersmitten, T., Bhati, M. T., Charbonnier, D., & Gur, R. C. (2007). Facial emotion recognition in schizophrenia: When and why does it go awry? Schizophrenia Research, 94(1–3), 253–263.
https://doi.org/10.1016/j.schres.2007.05.001

Wright, C. I., Martis, B., Shin, L. M., Fischer, H., & Rauch, S. L. (2002). Enhanced amygdala responses to emotional versus neutral schematic facial expressions. NeuroReport, 13(6), 785–790.
https://doi.org/10.1097/00001756-200205070-00010

Yin, G., She, S., Zhao, L., & Zheng, Y. (2018). The dysfunction of processing emotional faces in schizophrenia revealed by expression-related visual mismatch negativity. NeuroReport, 29(10), 814–818.
https://doi.org/10.1097/wnr.0000000000001037

This work was supported by the National Natural Science Foundation of China (61976150)

the Shanxi Provincial Key Research and Development Program (201803D31038)

the Jinzhong Key Research and Development Projects of Science and Technology (Y192006)

the CERNET Next Generation Internet Technology Innovation Project (NGII20181206)

and the Special Project for Meteorological Forecast of Domestic and Foreign Crop Production (RH19100004).

Haifang Li, College of Computer Science and Technology, Taiyuan University of Technology, Tai Yuan, People’s Republic of China. Email: [email protected], or Lun Zhao, School of Educational Science, Liaocheng University, Liaocheng, People’s Republic of China. Email: [email protected]