As reform movements have accelerated in the Turkish higher education context over the last five years, an urgent need has emerged to assess preservice teachers in terms of the extent to which they are disposed to think critically. As a result of this need, realization has developed that there is a significant lack in empirical research and assessment of critical thinking in the Turkish higher education context. Previous researchers have used only qualitative methods and no authorized Turkish instrument had been identified to evaluate teacher education programs in terms of critical thinking disposition. In this regard, much attention has been directed to assessing the critical thinking disposition of preservice teachers in the Turkish higher education context. The search around the world for a means of ascertaining an informed assessment of preservice teachers’ critical thinking disposition led to the development of the California Critical Thinking Disposition Inventory (CCTDI; Facione, 1990). As this is the only instrument found by researchers to conceptualize critical thinking dispositions in such a way that it could be used as a means of evaluation in teacher education programs, the CCTDI has been translated into many languages including Arabic, Chinese (Mandarin), Dutch, Farsi, Finnish, French (Canadian), Hebrew, Italian, Japanese, Korean, Portuguese, Spanish (Mexico-Latin America), and Thai. Because an authorized instrument in the Turkish language measuring critical thinking dispositions and professional judgment in teacher education was not currently available, we set out to adapt the CCTDI from the English source language into Turkish, and explore the factorial validity of the suggested multidimensional measurement model. The CCTDI has a strong theoretical foundation and, as well as the urgency of the need to assess the critical thinking dispositions of preservice teachers in the Turkish higher education context, our decision to adapt this instrument rather than developing a new inventory was supported by many researchers who have suggested translating existing instruments rather than developing new ones (Chapman & Carter, 1979; Waltz, Strickland, & Lenz, 1991). For example, Waltz et al. have observed that the use of existing instruments increases the utilization of assessment tests and decreases the cost of outcome assessment.

According to a number of scholars (Dewey, 1910; Ennis, 1993; Facione, 1990; Facione, Giancarlo, & Facione, 1995), including the researchers for the cross-disciplinary Delphi study, which was supported by the Pre-College Philosophy Committee of the American Philosophical Association critical thinking (CT) is composed of a cognitive skills dimension and an affective dispositions dimension and involves being willing and able to use one’s cognitive powers of analysis, interpretation, inference, evaluation, explanation, and self-monitoring metacognition to make purposeful judgments about what to believe or what to do in a given context. In other words, in order for individuals to make purposeful judgments about what to believe or what to do in a given context, they not only need to have the cognitive skills described, but they also need to be positively disposed to use these skills (Dewey, 1910; Facione, 1990; Lewin, 1935). The aim of the CCTDI, which was the end product of the Delphi study, is to assess the disposition dimension of CT. According to the 46 critical thinking international experts who conducted the Delphi study, the CCTDI represents a high degree of fit between the current conceptualization and measurement development of critical thinking dispositions. However, the decision to use an existing instrument to measure a phenomenon in another cultural group for whom the instrument was not originally developed, and who speak a different language from that original group, requires that, before the instrument is used in this different cultural context, the psychometric properties should be assessed and evidence presented for factorial validity (Ægisdóttir, Gerstein, & Çinarbas, 2007; Nunnally & Bernstein, 1994; Sireci et al., 2006). Thus, our purposes in this study were: (a) to translate the CCTDI from the English source language into Turkish; (b) to assess the psychometric properties of the Turkish version of CCTDI; and (c) to assess the factorial validity via confirmatory factor analysis (CFA).

Method

Translation and Backtranslation Process

Prior to any translation exercise, we obtained all the necessary permissions to translate the CCTDI (Facione, 1990) from Insight Assessment, a division of California Academic Press, and the copyright holder of the instrument. The first author of this article was confirmed as the authorized translator for the Turkish version of the CCTDI.

The first step in undertaking the process of translation was to clarify the intention of each item in the original CCTDI. Establishing an extensive informed opinion regarding the intention of each item prior to an initial translation was important for maximizing the semantic, conceptual, and normative equivalencies and minimizing item bias across different language versions (Ægisdóttir, Gerstein, & Çinarbas, 2007; Behling & Law, 2000). For this reason, each of the items in the inventory was negotiated with Peter Facione, who is the author of the inventory. We obtained a detailed informed opinion regarding the intended meaning of each item in the original CCTDI. Following clarification of meaning, we undertook an initial translation and backtranslation process. For the current study, as suggested by Brislin (1970), we embedded the translation and backtranslation process into an interactive adaptation process, in order to maximize the translation equivalency. Each cycle involved three important steps of initial translation, followed by backtranslation, and, finally, comparison of the original and backtranslated versions for any modification and adaptation. After we had repeated this cycle three times, the committee agreed that the Turkish version of the instrument was identical in meaning with the original version in English.

Participants

Our sample was composed of 583 undergraduate students (51.3% female) from five different teacher education programs in Turkey. Freshmen (n = 231) made up 39.6% of the sample, there were 179 (30.7%) sophomores, 124 (21.3%) juniors, and 49 (8.4%) were senior students. The age range of the students was from 18 to 29 years, with a mean age of 19 years (SD = 1.72). For the purpose of this validation study, the Turkish sample represented the population for which the CCTDI was adapted. In our sample 75 (12.9%) participants were students in the English Language Teacher Education Program, 92 (15.8%) were students in the Computer Science Teacher Education Program, 114 (19.6%) were students in the Elementary Teacher Education Program, 186 (31.9%) were students in the Preschool Teacher Education Program, and 116 (19.9%) were students in the Sociology Teacher Education Program.

Measures

The CCTDI is an inventory of 75 items rated on a 6-point, forced-choice scale (1 = disagree totally, 2 = disagree, 3 = disagree partially, 4 = agree partially, 5 = agree, 6 = agree totally) and measures seven dimensions of critical thinking dispositions with seven subscales. Facione, Giancarlo, and Facione (1995) reported alphas for the subscales of the CCTDI as follows: truth-seeking (12 items, α = .72); open-mindedness (12 items, α = .73); analyticity (11 items, α = .72); systematicity (11 items, α = .74); critical thinking self-confidence (9 items, α = .78); inquisitiveness (10 items, α = .80); maturity of judgment (10 items, α = .75); and overall scale (75 items, α = .90). In the current research, we reexamined alphas and other psychometric properties for the agreed Turkish version of the CCTDI.

Procedure and Data Analysis

Following the translation and backtranslation process, the next step was to check the content validity of the Turkish version of the CCTDI. Five Turkish experts in educational psychology, linguistics, and critical pedagogy served as content validators for the translated Turkish version. Experience in their fields of study, history of publications, research on the phenomenon of critical thinking, and academic qualifications were the criteria we set for selection of experts (Davis, 1992; Drasgow & Probst, 2005). Each expert was provided with a set of four documents consisting of a cover letter, content domains, the CCTDI, and the content validity estimation scale. In the cover letter we informed content validators about the study, confidentiality of their answers, their role as content validators, the measurement models of the CCTDI, and provided details of the other attached documents. The second document, which included the content domains and element definitions from which all 75 items of the CCTDI were obtained, served as a standard by which content validators could compare each item against the definition. In order for the Turkish experts to see the complete inventory, we gave a version of the CCTDI that had been translated into Turkish to them. Experts used the content validity index (CVI) developed by Waltz et al. (1991) to rate each item of the CCTDI firstly on its relevance to content domains on a scale ranging from 1 = not relevant to 4 = very relevant, then on its clarity (scale ranging from 1 = not clear to 4 = very clear), on its simplicity (scale ranging from 1 = not simple to 4 = very simple), and its ambiguity (scale ranging from 1 = doubtful to 4 = meaning is clear). The criterion for accepting a subscale or a total instrument as valid to the specified content was the percentage of subscale items or overall scale items from each category receiving a score of three or four (Drasgow & Probst, 2005; Waltz et al., 1991) with a minimum CVI of 0.90 (Davis, 1992). After examining the CVIs, we computed the alpha coefficients for the seven subscales of the CCTDI to assess the internal consistency reliability prior to any CFA. It was necessary to compute coefficient alphas for scales of the CCTDI before CFA and after CFA with modified items to check if any item trimming led to an unacceptable decrement in alpha coefficients (Nunnally & Bernstein, 1994). Because this was the first time that researchers had undertaken the translation of the CCTDI from the English source language into Turkish, the cutoff point for internal consistency reliability, as recommended by George and Mallery (2003) was set at .70.

On the basis of the related literature (Chen, 2007; Hu & Bentler, 1999; Milfont & Fisher, 2010; Sass, 2011), we used the following indices and cutoff points to evaluate the model fit of the 7-factor model of the CCTDI: chi square (χ²), degrees of freedom (df), chi square to degrees of freedom ratio (χ²/df < 4.0), the root mean square error of approximation (RMSEA < 0.06 = good fit; values between 0.06 and 0.08 are classed as adequate fit; and values between 0.08 and 0.10 as mediocre fit), the standardized root mean square residual (SRMR < 0.06 = good fit; values between 0.06 and 0.08 are classed as adequate fit; and values between 0.08 and 0.10 as mediocre fit), and comparative fit index (CFI ≥ 0.90 = adequate fit; values greater than 0.95 are classed as good fit). We used SPSS version 18 together with IBM AMOS version 20 to run the required statistical analysis throughout the study.

Results

Psychometric Properties

CVIs ranged from 0.83 to 0.99 for the subscales of the Turkish CCTDI. In all but one subscale evidence for content validity existed. Although the raters agreed that the items were relevant to the content domains and definitions specified for each corresponding latent factor, among the raters there was less agreement than there was for the other subscales about the open mindedness scale in the Turkish version (see Table 1 for descriptive statistics).

We studied alpha coefficients for the Turkish CCTDI with 75 items prior to a CFA attempt, and alphas for the subscales ranged from 0.81 to 0.90 (see Table 1). These values satisfied the minimum expected criteria of 0.70 for the first attempt at translating and adapting the scales, but our goal was to reach 0.90 (George & Mallery, 2003). Therefore, the results suggested the need for further development of the CCTDI for use in the Turkish culture and language. It should also be noted that we recalculated the coefficient alphas with the latest versions of these subscales after subsequent CFA.

Factorial Validity of the Turkish Version of CCTDI

In order to conduct further analysis regarding the psychometric properties of the Turkish CCTDI, we examined the factorial validity of the instrument. Our basic premise here was to explore the extent to which the hypothesized 7-factor structure of the measurement model for the original English-language CCTDI was valid for the Turkish language version. When we considered the standardized estimates for the examination of factorial validity, the hypothesized 7-factor measurement model produced a poor fit for the Turkish CCTDI, χ²(df = 2679) = 10090.724, p < .001, χ²/df = 3.767, RMSEA = .069, SRMR = .096, CFI = .66. When we examined the regression slopes and the correlation matrix to establish the reason for this poor model fit, the modification index suggested that, because of their low estimation effects of parameter estimates, such as factor loadings and factor pattern coefficients, three factors should be excluded from the measurement model; namely open-mindedness, analyticity, and inquisitiveness. When we removed those three selected factors from the model, the adapted 4-factor model produced a better fit for the Turkish version of the CCTDI, but still an insufficient improvement; χ²(df = 813) = 3019.200, p < .001, χ²/df = 3.714, RMSEA = .068, SRMR = .083, CFI = .76.

Further consideration of the standardized estimates revealed that there were a number of items with standardized factor loadings smaller than .30, indicating that these items might not belong to the corresponding hypothesized latent factors. For this reason, with the intention of increasing the factorial validity and construct validity of the Turkish version of the CCTDI, we deleted items with factor loadings smaller than .30 from the subscales as follows: we deleted five items from the truth-seeking subscale (items 12, 19, 23, 50, and 62); five items from the systematicity subscale (items 4, 29, 37, 58, and 68); four items from the critical-thinking self-confidence subscale (items 10, 16, 18, and 56); and six items from the maturity of judgment subscale (items 3, 7, 11, 14, 53, and 71). After these deletions had been made, the modified hypothesized model displayed a significant but still insufficient improvement for the Turkish version of the CCTDI, χ²(df = 203) = 730.348, p < .001, χ²/df = 3.598, RMSEA = .067, SRMR = .060, CFI = .92. When we evaluated the modification indices carefully, it was evident that correlating three residuals with their pairs (see Figure 1) produced a good fit for the Turkish sample χ²(df = 200) = 626.811, p < .001, χ²/df = 3.134, RMSEA = .061, SRMR = .057, CFI = .95.

When we recomputed the alpha coefficients after CFA with the 4-factor model, the alphas for the subscales of the Turkish CCTDI were as follows: (truth seeking, seven items, α = 0.80; systematicity, six items, α = 0.75; CT self-confidence, five items, α = 0.83; maturity of judgment, four items, α = 0.77). The modification suggested a result of CFA-produced decrements in alphas for all scales. The largest decrements among the subscales of the Turkish CCTDI were recorded in the systematicity scale and the maturity of judgment scale. All of the alphas for the subscales were above the critical point of 0.70 and displayed evidence for internal consistency reliability.

Discussion

In this research, we carried out the first translation into Turkish of the CCTDI (Facione, 1990), in order to measure the extent to which individuals in the setting of the Turkish culture are disposed to think critically and also to investigate the factorial validity of the Turkish version of the CCTDI. Our evaluation of the results has produced the following suggestions for researchers who wish to use the CCTDI with any selected group of individuals to measure their critical thinking dispositions (Cheung & Rensvold, 1999; Milfont & Fisher, 2010). First, despite the fact that the initial 75-item 7-factor model of the CCTDI did not achieve a good fit to the observed data in our study, researchers may use this model by relying on the high values of Cronbach’s alpha for internal consistency reliability and the high values of CVIs for evidence of content validity. Hambleton (2005) suggested that in such cases where the results do not reveal the instrument to have a one to one fit to the observed data, researchers may use the modified model with a reduced number of latent factors only if each latent factor has been assessed by at least four interrelated items. Second, Hambleton also suggested that latent factors considered to be noninvariant be omitted and items with low parameter estimates be deleted. In line with these suggestions we used a modified 22-item 4-factor model to assess the disposition dimension of critical thinking in a Turkish higher educational context. We recommend that further studies similar to ours should be conducted with the inclusion of the administration of the English version of the CCTDI in the US with an English-speaking population and the measurement invariance of the CCTDI should be assessed across Turkish and English-speaking US cultural groups by means of comparing the 4-factor model we have suggested in this study with the hypothesized 7-factor model of the original English version of the CCTDI. In conclusion, assessing critical thinking disposition is no longer a local issue. With an increasing interest in international research on this topic, researchers must identify reliable and valid instruments to make cross-cultural comparisons of critical thinking disposition.

References

Ægisdóttir, S., Gerstein, L. H., & Çinarbas, D. C. (2007). Methodological issues in cross-cultural counseling research: Equivalence, bias, and translations. The Counseling Psychologist, 36, 188-219. http://doi.org/fdhn32

Behling, O., & Law, K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. Thousand Oaks, CA: Sage.

Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1, 185-216. http://doi.org/b99wbg

Chapman, D. W., & Carter, J. F. (1979). Translation procedures for the cross-cultural use of measurement instruments. Educational Evaluation and Policy Analysis, 1, 71-76.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multi-Disciplinary Journal, 14, 464-504. http://doi.org/dmtdv9

Cheung, G. W., & Rensvold, R. B. (1999). Testing factorial invariance across groups: A reconcept- ualization and proposed new method. Journal of Management, 25, 1-27. http://doi.org/b3zpbn

Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5, 194-197. http://doi.org/c935qq

Dewey, J. (1910). How we think. Boston, MA: D. C. Heath.

Drasgow, F., & Probst, T. M. (2005). The psychometrics of adaptation: Evaluating measurement equivalence across languages and cultures. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Erlbaum.

Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32, 179-186. http://doi.org/ch9khk

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Millbrae, CA: California Academic Press.

Facione, P. A., Giancarlo, C. A., & Facione, N. C. (1995). The disposition toward critical thinking. The Journal of General Education, 44, 1-25.

George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (4th ed.). Boston, MA: Pearson Education.

Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Erlbaum.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. http://doi.org/dbt

Lewin, K. (1935). A dynamic theory of personality: Selected papers. Translated by D. K. Adams & K. E. Zener. New York: McGraw-Hill.

Milfont, T. L., & Fisher, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3, 111-121.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29, 347-363. http://doi.org/ds4r63

Sireci, S. G., Yang, Y., Harter, J., & Ehrlich, E. J. (2006). Evaluating guidelines for test adaptations: A methodological analysis of translation quality. Journal of Cross-Cultural Psychology, 37, 557-567. http://doi.org/d9ncfb

Waltz, C. F., Strickland, O., & Lenz, E. (1991). Measurement in nursing research (2nd ed.). Philadelphia, PA: Davis.

Ægisdóttir, S., Gerstein, L. H., & Çinarbas, D. C. (2007). Methodological issues in cross-cultural counseling research: Equivalence, bias, and translations. The Counseling Psychologist, 36, 188-219. http://doi.org/fdhn32

Behling, O., & Law, K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. Thousand Oaks, CA: Sage.

Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1, 185-216. http://doi.org/b99wbg

Chapman, D. W., & Carter, J. F. (1979). Translation procedures for the cross-cultural use of measurement instruments. Educational Evaluation and Policy Analysis, 1, 71-76.

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multi-Disciplinary Journal, 14, 464-504. http://doi.org/dmtdv9

Cheung, G. W., & Rensvold, R. B. (1999). Testing factorial invariance across groups: A reconcept- ualization and proposed new method. Journal of Management, 25, 1-27. http://doi.org/b3zpbn

Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5, 194-197. http://doi.org/c935qq

Dewey, J. (1910). How we think. Boston, MA: D. C. Heath.

Drasgow, F., & Probst, T. M. (2005). The psychometrics of adaptation: Evaluating measurement equivalence across languages and cultures. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Erlbaum.

Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32, 179-186. http://doi.org/ch9khk

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Millbrae, CA: California Academic Press.

Facione, P. A., Giancarlo, C. A., & Facione, N. C. (1995). The disposition toward critical thinking. The Journal of General Education, 44, 1-25.

George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (4th ed.). Boston, MA: Pearson Education.

Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Erlbaum.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. http://doi.org/dbt

Lewin, K. (1935). A dynamic theory of personality: Selected papers. Translated by D. K. Adams & K. E. Zener. New York: McGraw-Hill.

Milfont, T. L., & Fisher, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3, 111-121.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29, 347-363. http://doi.org/ds4r63

Sireci, S. G., Yang, Y., Harter, J., & Ehrlich, E. J. (2006). Evaluating guidelines for test adaptations: A methodological analysis of translation quality. Journal of Cross-Cultural Psychology, 37, 557-567. http://doi.org/d9ncfb

Waltz, C. F., Strickland, O., & Lenz, E. (1991). Measurement in nursing research (2nd ed.). Philadelphia, PA: Davis.

Gökhan İskîfoĝlu, Department of Educational Sciences, Eastern Mediterranean University, Famagusta, North Cyprus via Mersin 10, Turkey. Email: [email protected]