Factors influencing students’ success: A generalized estimating equations study
Main Article Content
The goals of this study were to determine the factors influencing children’s success according to Generalized Estimating Equations (GEE) through the Family Report Form (FRF; constituted by the authors), Teacher’s Report Form (TRF; Achenbach, 1991b) and Child Behavior Check List (CBCL; Achenbach & Edelbrock, 1983), and to form a new and actual reference success form for Turkish children. The study sample included 4130 primary school children in Eskisehir, Turkey. School success was measured as the total scores that students achieved in the following classes: cultural, social, science, foreign language, computer, picture, music and job-occupation. The TRF, CBCL and FRF forms consist of the sum of subtests, which includes 9 factor scales, namely anxiety/depression, somatic complaints, social withdrawal, delinquent behavior, social problems, thought problems, attention deficits, aggressive behavior, and other. As the statistical analyses, GEE method, stepwise regression analysis and missing value analyses were used. The items that had the greatest negative effect on the students’ success were “has poor school work” and “can’t concentrate or can’t pay attention for long”. On the other hand, the item that had the greatest positive effect on students’ success was “feels he/she has to be perfect”. It may be recommended that students, their families, and teachers should be informed about factors influencing school success.
The Child Behavior Checklist (CBCL) was originally an American behavioral questionnaire, which has become widely known in recent years (Achenbach, 1991a; Achenbach & Edelbrock, 1983). It is one of the best-studied empirically derived checklists available to measure psychopathology (Achenbach, Howell, Quay, & Conners, 1991), with excellent psychometric properties and a large body of research to demonstrate its reliability and validity in both clinical and nonclinical populations (Achenbach, 1991a). The CBCL is well suited to answer the question of whether children being identified by different assessment methods share a common behavioral phenotype and what that phenotype may be (Mick, Biederman, Pandina, & Faraone, 2003).
The development of the CBCL marked the beginning of the development of a series of behavioral questionnaires, each meant for a different informant. This has resulted in the construction of the Teacher’s Report Form (TRF; Achenbach, 1991b), and the Family Report Form (FRF), which was designed by the authors like the Youth Self-Report (YSR) by Achenbach (1991c), and was modeled on the CBCL. These questionnaires all have the same hierarchical structure in order to evaluate problems related to children or students. By using the CBCL, TRF, and FRF, observations on the problem behaviors of a specific child can be obtained from different informants.
Many countries have formed their own norms to investigate psychosocial problems connected with factors influencing children’s success at primary education age (Achenbach, 1991a; Achenbach & Edelbrock, 1983; De Groot, Koot, & Verhulst, 1994; Detrick, Greenbaum, Friedman, Wetherington, & Knoff, 1997; Mick et al., 2003). Since no common use norms have yet been developed in Turkey, norms developed by other countries have been used (Akbas, Tutuncu, Tamar, Coskunol, & Kesikci, 2003; Erol, Kilic, & Ulusoy, 1998; Ulu & Filologlu, 2002). However, evaluation made through these norms does not truly reflect the facts since psychosocial, economic, environmental and cultural positions between societies are different. Thus, it is vital to form norms that determine the psychosocial, economic and cultural problems influencing students’ success in each country.
In order to form a model(s) to accurately estimate success status, the relationships among many items are studied and determined by means of Generalized Estimating Equations (GEE). The GEE method has been previously employed by Horton and Bebchuk (1999) when they determined the charac- teristics affecting mental health in 2519 children.
Since no study relating to the influence of psychosocial, economic and cultural problems on children’s success in Turkey has been conducted, the primary goals of this study were to determine the factors influencing children’s success according to the GEE method (Agresti, 1999; Stata 8.2, 2003) through the FRF, TRF and CBCL and to form a new and actual success reference form for Turkish children. Participants were 4130 primary education children between 10 and 15 years of age in a city of west Turkey, Eskisehir.
Method
Eskisehir and Primary Schools in Eskisehir
The city has a total population of 600,000, which includes two universities. Its socioeconomic level is judged to be average in Turkey. The socioeconomic and cultural status of those living in the centre of the city and their children in the central schools are better than that of those in rural areas or slum houses. The city has seven regions with different socioeconomic characteristics.
The number of primary schools in Eskisehir is 101 with a capacity of approximately 50,000 students.
Participants
For the purposes of the study, we aimed to contact the entire student population (n = 50,000) of all 101 schools. When this study was conducted between January 20th, 2003 and January 20th, 2004, a total of 35,000 students aged between 10 and 15 years were studying in the 4th, 5th, 6th, 7th and 8th classes of those schools. Children of the 1st, 2nd and 3rd classes were excluded from the study due to the assumption that they would not be able to answer the questions on the forms owing to their young ages. However, due to the logistical problems of reaching all 35,000 students in the 101 schools, the sample was limited through the stratified and random sampling method to 4130 primary education children and to include 9 schools and the classes in those schools alongside their class teachers, as well as the students’ parents.
Instruments
Forms for collecting data Students, families, and class teachers were asked to respond to parallel questions. In addition, the form prepared for the students included students’ names, sexes, ages, classes and school success, and the forms prepared for students’ families and teachers included just the students’ names and students’ school success according to lessons, so as not to cause confusion between the forms. We distributed the forms to all students and teachers in the selected classes, as well as giving the children their parents’ forms to be filled in by the students’ mothers and fathers at home. A model was constituted to determine factors influencing students’ success through the GEE method. This meant employing responses to the 118 form questions given through 3 parallel channels in the form of the CBCL, TRF and FRF.
Measure Forms By reporting in each of the 118 concrete behavioral problems whether this behavior is not true, somewhat or sometimes, or very much or often true, parents or teachers give a detailed picture of the behavioral problems of their child or student. Research has shown that these concrete behaviors were grouped around nine core factors: Social Withdrawal, Somatic Complaints, Anxiety/Depression, Social Problems, Thought Problems, Attention Deficits, Delinquent Behavior, Aggressive Behavior and other (Achenbach, 1991a; De Groot et al., 1994; Detrick et al., 1997).
As the variable determining the child’s school success, the sum of scores received from cultural, social, science, foreign language, computer, picture, music and job-occupation lessons was used.
We used 3 types of measure forms: the Teacher’s Report Form (TRF) completed by the teachers, the Child Behavior Checklist (CBCL) completed by the students themselves, and the Family Report Form (FRF) completed by the students’ families. The TRF and CBCL are two of three assessment instruments developed by Achenbach and Edelbrock (1983) and Achenbach (1991a) to measure adaptive functioning and problem behaviors of students. The TRF and FRF completed by the teachers and families respectively are modeled on the CBCL and similarly are appropriate for children aged between 10 and 15. These similarities between the forms enabled comparison between the three measures. The 118 problem items are rated 0 if the item is not true, 1 if the item is somewhat or sometimes true and 2 if the item is very true or often true according to the observed frequency in the last six months. As well as providing information regarding the presence and severity of the 118 problem items, the TRF, CBCL and FRF also generate 9 factor scales of problems that usually co-occur. These are: anxiety/depression, somatic complaints, social withdrawal, delinquent behavior, social problems, thought problems, attention deficits, aggressive behavior and other.
Validity and reliability of the scales Erol et al. (1998) conducted the validity and reliability analysis of CBCL and TRF for use in Turkey. Furthermore, we wanted to evaluate the answers given to three-channel parallel questions, including the students’ parents, students’ teachers and students’ own responses. In order to achieve this, we developed the FRF by adapting the corresponding questions in the CBCL. Reliability analysis of the 118 questions in the 3 forms, and the success variable was fulfilled on 453 students, parents and teachers. Since the alpha coefficient was found to be 0.95, these forms were used as the main data in the study.
Statistical Analyses
In this study, GEE were used during investigation of psychosocial, economic, environmental and cultural problems relating to success and failure. This was due to the fact that the items were interconnected with each other. We introduced GEE to account for correlation between observations in generalized linear regression models (Liang & Zeger, 1986; Zeger & Liang, 1986).
A GEE model is obtained from the orthogonality of the estimating equations for the regression and association parameters. A population-averaged GEE model based on Pearson residuals, focusing on the marginal distribution of the outcome, was used (Hardin & Hilbe, 2003).
Using the notation from Liang and Zeger (1986), we let yi = (yi,l,…,yi,n)T be the nix 1 vector of outcome values, and let Xi = (xi,l,…,xi,n)T be the ni x p matrix of covariate values for the ith subject i = 1,…, m. We assumed that the marginal density for yi,j may be written in exponential family notation as f(yi,j) = exp ë{yi,jƟi,j– a(Ɵi,j) + b(yi,j)}øû where Ɵi,j was h(ηi,j), and ηi,j was xi,j β. Under this formulation, the first two moments were given by E(yi,j) = a'(ɵi,j), Var (y i,j) = a'' (ɵi,j)/ø. We defined the quantities [assuming that we have an nxn working correlation matrix R(α)], as ∆i = diag(Ɵi,j/dηi,j),Ai = diag{a''(Ɵi,j)}, Si = yi-a'(Ɵi),Di = Ai∆iXi and Vi=Ai1/2 R(α) Ai1/2 such that the GEE became
We then had
where the term
was what we called the IRLS variance estimate (iteratively reweighted least squares). It was used to calculate the standard errors if the robust option was not specified. For the calculation of the robust variance estimator D = (DTi ,…,DTm), S = (ST1 ,…,STm), and Ṽ = nmxnm block diagonal matrix with Ṽ were used.
At a given iteration, the correlation parameters a and scale parameter ∅ can be estimated from the current Pearson residuals, defined by ȓi,j {yi,j − a'(ij)}/{a'' (ij)}1/2 = , where ij depended on the current value for β. We can estimate ∅ by
As the above general derivation is complicated, we used the derivation of the Gaussian family with the identity link (regression) to illustrate the generalization. After making appropriate substitutions, we saw a familiar updating equation. First, we rewrote the updating equation for β as βˆ = βˆj+1 = βˆj – Z1-1Z2, and then derived for Z1 and Z2: In the above equation, Z1 was XT X., Z2 was XT ŝj. So, that meant that we may write the updated formula as βj+1 = βj − (XTX) XTŝj , which was the same formula for IRLS in regression. In GEE analysis, independent correlation structure, an identity matrix, was used (Stata 8.2, 2003).
To develop the factor scales according to the questions, each item was placed within a factor scale. In this way, items that belong to each scale were grouped. Nine factor scales were constituted separately for students, teachers and family. The school success variable was received separately for FRF, CBCL and TRF. To determine variables affecting the success by GEE method, the students’ school success points were taken into consideration as the dependent variable, the other variables as covariates, distributional family as Gaussian, and link function as identity. The study correlation structure was also selected as the independent correlation structure.
Items with a probability above 0.05 were excluded from the model by stepwise regression analysis. As a result, the items below 0.05 − or, those remaining in the model − were put into GEE analysis. Horton and Bebchuk (1999), when determining the characteristics that affect mental health on 2519 children established GEE method as [xtgee service olderchild*mentalhealth olderchild*school academicproblem*mentalhealth academicproblem*school, link(logit) corr(unst) family(binomial) robust] and [xtgee serv i.old*mental i.old*school i.acadpro*mental i.acadpro*school, link(logit) corr(unst) family(binomial) robust].
Missing Value Analyses were carried out for the gaps in the study and median values were put into those missing places.
Informed Consent
Before the commencement of the study, we obtained consent from both the teachers themselves and their students and school principals. This study was approved by the local authorities, those of the Osmangazi University School of Medicine and the education authorities in the city of Eskisehir. Subjects were told that participation in the investigation was strictly voluntary and refusal would not affect their status as a student or a teacher.
Results
A total of about 5000 students studying at 9 primary schools were approached for inclusion in the study. Of these, 4130 (93.4%) (1974 boys and 2156 girls) responded to the survey. Eight hundred and seventy questionnaires were excluded from the study because either students or teachers were absent from school, questionnaires were not filled in, the mother or father did not fill in the questionnaires, or sent questionnaires were missing.
Table 1 shows the GEE of the 4130 students peculiar to society averages. It was found that there were 24 items that indicated Turkish children’s norms.
Table 1. The Results of the Generalized Estimating Equations of 4130 Students Anomalous to Society Averages
Wald x2 (24) = 3407.62 p < 0.001
According to the composed GEE model, for primary school students, it was found that the items “acts too young for his/her age” (p < 0.001), “has very little he/she enjoys”, “can’t concentrate or can’t pay attention for long” (p < 0.001), “confounds or seems to be in a fog” (p < 0.001), “cruel to animals” (p < 0.001), “feels himself/herself worthless or inferior” (p < 0.001), “gets in many fights” (p = 0.001), “bites fingernails” (p = 0.002), “is nervous, high-strung or tense” (p = 0.003), “nightmares” (p < 0.001), “has poor school work” (p < 0.001), “runs away from home” (p < 0.001), “is too shy or timid” (p < 0.001), “talks too much” (p < 0.001), “is underactive, slow moving or lacks energy” (p < 0.001), and “wets the bed” (p = 0.001), affected success negatively; however, “argues too much” (p < 0.001), “complains of loneliness” (p = 0.004), “demands a lot of attention” (p = 0.011), “fears he/she might think or do something bad” (p = 0.001), “feels he/she has to be perfect” (p < 0.001), “is stubborn, sullen or irritable” (p < 0.001), and “threatens people” (p < 0.001) affected success positively.
The items that had the greatest effect on the students’ success negatively were “has poor school work” (coefficient: -2.85), and “can’t concentrate or can’t pay attention for long” (coefficient: -0.91). They were both in the factor scale ‘attention’. On the other hand, the item that had the greatest positive effect on the students’ success was “feels he/she has to be perfect” (coefficient: 0.604). This item was found to be in the factor scale ‘anxiety/depression”.
Discussion
Characteristics influencing psychosocial, economic, environmental and cultural conditions were investigated in the current study to determine the factors influencing school success among primary school students. According to the forms CBCL, FRF and TRF, it was found that the most critical reasons causing a decline in students’ school success out of 9 factor scales were the deficit problems such as “acts too young for his/her age”, “can’t concentrate, can’t pay attention for long”, and “poor school work” and anxiety/depression such as “feels worthless or inferior” and “too shy or timid”. This shows that attention and anxiety/depression are important factors in determining students’ school success and also that to increase success, psychological counseling centers or school counselors should be used more frequently.
According to the CBCL, FRF and TRF, the most important items found to effect an increase in students’ success were anxiety/depression problems including “argues a lot”, and “feels he/she has to be perfect”. This illustrates the fact that a student achieves success as a result of forcing him or herself psychologically and biologically because he/she would like to be perfect. However, through doing this, their experiencing anxiety and depression is easily attributable to psychological and biological distress. In addition, this may show that anxiety levels increase due to the fact that adolescent students are perfectionists, and as a result, anxiety/depression is observable. The fact that students want to be successful escalates anxiety and depression levels. According to the GEE analysis method, it was determined that there were 24 items which could form a new and actual reference form for Turkish children. Except for “clings to adults or too dependent in social problems”, the remaining 23 items are from seven factor scales: ‘attention’, ‘social problems’, ‘anxiety/depression’, ‘delinquent behavior’, ‘aggressive’, ‘social withdrawal’ and ‘other problems’. ‘Somatic complaints’ and ‘thought problems’ were excluded as they did not contribute significantly to whether a student was successful or unsuccessful. Further studies are needed to address factors affecting success. We recommend that students, their families, and teachers be informed about those factors influencing school success and also that each school’s management employs a psychologist or counselor who may prevent or deal with adolescent’s psychological problems at an early age.
References
Achenbach, T. M. (1991a). Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M. (1991b). Manual for the Teacher’s Report Form and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M. (1991c). Manual for the Youth Self-Report and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M., Howell, C. T., Quay, H. C., & Conners, C. K. (1991). National survey of competencies and problems among 4- to 16-year-olds: Parents’ reports for normative and clinical samples. Monographs of the Society for Research in Child Development, 56, 1-131.
Agresti, A. (1999). Modelling ordered categorical data: Recent advances and future challenges. Statistics in Medicine, 18, 2191-2207.
Akbas, S., Tutuncu, E., Tamar, M., Coskunol, H., & Kesikci, H. (2003). Academic success of Type I and Type II alcoholics children. Journal of Dependence, 4, 91-95.
De Groot, A., Koot, H. M., & Verhulst, F. C. (1994). Cross-cultural generalizability of the Child Behavior Checklist cross-informant factors. Psychological Assessment, 6, 225-230.
Detrick, R. F., Greenbaum, P. E., Friedman, R. M., Wetherington, C. M., & Knoff, H. M. (1997). Testing the structure of the Child Behavior Checklist/4-18 using confirmatory factor analysis. Educational and Psychology Measurement, 57, 306-313.
Erol, N., Kilic, C., & Ulusoy, M. (1998). Turkiye Ruh Sagligi Profili: Cocuk ve Genclerde Ruh Sagligi: Yeterlilik Alanlari Davranis ve Duygusal Sorunlarin Dagilimi. 1993. Ankara: Eksen Tanitim Limited.
Hardin, J. W., & Hilbe, J. M. (2003). Generalized estimating equations. London: Chapman & Hall/ Boca Raton, FL: CRC.
Horton, N. J., & Bebchuk, J. D. (1999). Goodness-of-fit for GEE: An example with mental health service utilization. Statistics in Medicine, 18, 213-222.
Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13-22.
Mick, E., Biederman, J., Pandina, G., & Faraone, S. V. (2003). A preliminary meta-analysis of the Child Behavior Checklist in pediatric bipolar disorder. Biological Psychiatry, 53, 1021-1027.
Stata Statistical Software (2003). Stata 8.2. College Station, Cross-Sectional Time-Series.
Ulu, G., & Filologlu, H. (2002). The relationship between Turkish children’s perceptions of marital conflict and their internalizing and externalizing problems. International Journal of Psychology, 37, 369-378.
Zeger, S. L., & Liang, K. Y., (986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics, 42, 121-130.
Achenbach, T. M. (1991a). Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M. (1991b). Manual for the Teacher’s Report Form and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M. (1991c). Manual for the Youth Self-Report and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington, VT: University of Vermont, Department of Psychiatry.
Achenbach, T. M., Howell, C. T., Quay, H. C., & Conners, C. K. (1991). National survey of competencies and problems among 4- to 16-year-olds: Parents’ reports for normative and clinical samples. Monographs of the Society for Research in Child Development, 56, 1-131.
Agresti, A. (1999). Modelling ordered categorical data: Recent advances and future challenges. Statistics in Medicine, 18, 2191-2207.
Akbas, S., Tutuncu, E., Tamar, M., Coskunol, H., & Kesikci, H. (2003). Academic success of Type I and Type II alcoholics children. Journal of Dependence, 4, 91-95.
De Groot, A., Koot, H. M., & Verhulst, F. C. (1994). Cross-cultural generalizability of the Child Behavior Checklist cross-informant factors. Psychological Assessment, 6, 225-230.
Detrick, R. F., Greenbaum, P. E., Friedman, R. M., Wetherington, C. M., & Knoff, H. M. (1997). Testing the structure of the Child Behavior Checklist/4-18 using confirmatory factor analysis. Educational and Psychology Measurement, 57, 306-313.
Erol, N., Kilic, C., & Ulusoy, M. (1998). Turkiye Ruh Sagligi Profili: Cocuk ve Genclerde Ruh Sagligi: Yeterlilik Alanlari Davranis ve Duygusal Sorunlarin Dagilimi. 1993. Ankara: Eksen Tanitim Limited.
Hardin, J. W., & Hilbe, J. M. (2003). Generalized estimating equations. London: Chapman & Hall/ Boca Raton, FL: CRC.
Horton, N. J., & Bebchuk, J. D. (1999). Goodness-of-fit for GEE: An example with mental health service utilization. Statistics in Medicine, 18, 213-222.
Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13-22.
Mick, E., Biederman, J., Pandina, G., & Faraone, S. V. (2003). A preliminary meta-analysis of the Child Behavior Checklist in pediatric bipolar disorder. Biological Psychiatry, 53, 1021-1027.
Stata Statistical Software (2003). Stata 8.2. College Station, Cross-Sectional Time-Series.
Ulu, G., & Filologlu, H. (2002). The relationship between Turkish children’s perceptions of marital conflict and their internalizing and externalizing problems. International Journal of Psychology, 37, 369-378.
Zeger, S. L., & Liang, K. Y., (986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics, 42, 121-130.
Table 1. The Results of the Generalized Estimating Equations of 4130 Students Anomalous to Society Averages
Wald x2 (24) = 3407.62 p < 0.001
Appreciation is due to reviewers including
Shane R. Jimerson
PhD
Department of Counseling Clinical &
Social Psychology
University of California Santa Barbara
2208 Phelps Hall
Santa Barbara
CA 93106
USA
Dr. Unal Ayranci, Osmangazi University, Medico-Social Center, Meselik 26480Z, Eskisehir, Turkey. Phone: +90 542 660 0203; Email: [email protected]