Problem Areas in Multilevel Modelling of School and Classroom Effects...

'Children in schools' is one of the most cited textbook examples for multilevel analysis (e.g., Ditton, 1998; Raudenbush & Bryk, 2002; Snijders & Bosker, 1999). And there is, in fact, already a large body of literature modelling school effects (e.g., Gustafsson et al., 2018; Mohammadpour et al., 2015) or classroom effects on student achievement (e.g., Kruger et al., 2017; Lipowsky et al., 2009). However, studies dedicated to a three-level modelling approach (school-class-student) are scarce (e.g., Creemers & Kyriakidēs, 2008; Vanlaar et al., 2015). The effects of both school and class level factors (and possible interactions) has hardly been investigated so far (Ditton & Müller, 2015). Therefore, the present study examines a comprehensive three-level model taking into account contextual factors at the student, the class, and the school level. Predictors were selected after extensive literature reviews. From a methodological perspective, this approach revealed two major problem areas:

• Multicollinearity: Few studies deal with multicollinearity issues in two-level models (Shieh & Fouladi, 2003; Yu et al., 2015), but I have found no concrete study that analysis the effect of multicollinearity in a three-level context. Therefore, two-level model recommendations were adapted for the three-level model. Factors at the second and third level were highly correlated. The higher level of multilevel models can have more influence than that on the lower level (Yu et al., 2015). To deal with this multicollinearity in the data only a model re-specification was possible. Therefore higher order factors were built, which integrated categorical and numeric information in scale construction. I combined latent dimensions into higher order factors. The person parameters obtained in a multidimensional Rasch model were used to build higher order factors through confirmatory factor analysis (CFA). Although this approach helped dealing with multicollinearity, it disregards much information pertaining to the third level 'school' and the second level 'class'.
• Linking the three levels: Data for student achievement was available from the educational standards survey (8th grade in mathematics). In contrast, data at school and classroom level was collected independently. Therefore, linking schools and teachers was straightforward, but there is no possibility of linking between teachers and pupils due to data protection regulations in Austria. The pupils can be assigned to the schools (level 3) but not to the teachers (level 2). Therefore, a "random match" approach was chosen to match levels 1 (pupils) to levels 2 and 3 (i.e., classes and schools). The pupils are assigned to the schools and each pupil is matched with each teacher of the school. This results in an "enlargement" of the data set because each pupil occurs several times. To correct this oversizing, a weighting factor is calculated for each observation, based on the relative frequency of teachers in the school. This approach ensures linkage, but some of the actual variance is lost through weighting.

In the lecture, the problem areas themselves and their approaches to solutions should be presented. For this purpose, they will be critically discussed taking into account the results in the multi-level analysis.