Reliability of Students’ Teaching Practice Scores Using Generalizability Theory
DOI:
https://doi.org/10.26437/ajar.v10i1.666Keywords:
Evaluation. generalisability theory. students. teaching practice. training institutions.Abstract
Purpose: The aim was to demonstrate the potency of using Generalisability Theory (GT) over Classical Test Theory (CTT) in reliability estimation. The objectives are to find the major source of error in the Intern Teaching Evaluation Form (ITEF) scores and to identify the optimum number of occasions of rating of teaching practice that would give the most reliable scores.
Design/ Methodology/ Approach: A random effects one-facet fully crossed design in which intern (p) was fully crossed with occasion (o) was adopted for the study. In all, 9,082 bachelor’s degree teaching practice triplicate scores for three academic years from 2015/2016 to 2017/2018 were analysed in this study. A univariate generalisability analysis with EduG was performed to analyse data.
Findings: The finding for relative interpretation, the scores were strongly reliable. For absolute interpretation, the scores were moderate to strongly dependable. The major source of error in the ITEF scores was the p x o interaction combined with unidentified sources.
Research Limitation/Implications: A firm conclusion on one major source of error being a single facet in the ITEF scores could not be made.
Practical Implication: The level of reliability and the optimum scoring design of the ITEF established by this study will aid in selecting the ITEF by teacher training institutions in Ghana for evaluation of teaching practice to train pre-service teachers for effective curriculum implementation to achieve national educational aims.
Social Implication: Reliable evaluation data can inform policy decisions related to teacher certification, professional development, and educational standards. Policymakers can use this data to develop evidence-based policies that support teacher quality and student achievement.
Originality/ Value: It has unearthed the psychometric properties and the ideal scoring design of the ITEF which were hitherto unknown.
References
Atilgan, H. (2013). Sample Size for Estimation of G and Phi Coefficients in Generalizability Theory. Eurasian Journal of Educational Research (EJER)(51).
Brennan, R. L. (2001). Generalizability Theory. Springer Science & Business Media.
Brennan, R. L., & Kane, M. T. (1977). An index of dependability for mastery tests. Journal of Educational Measurement, 277-289.
Burns, K. J. (1998). Classical reliability: using generalizability theory to assess dependability. Research in nursing & health, 21(1), 83-90.
Cardinet, J., Johnson, S., & Pini, G. (2011). Applying generalizability theory using EduG. Routledge.
Cronbach, L. J. (1972). The dependability of behavioral measurements. Theory of generalizability for scores and profiles, 1-33.
Huijgen, T., van de Grift, W., Van Boxtel, C., & Holthuis, P. (2017). Teaching historical contextualization: the construction of a reliable observation instrument. European Journal of Psychology of Education, 32(2), 159-181.
Khan, K. K., & Mohsin Reza, M. (2022). Social Research: Definitions, Types, Nature, and Characteristics. In Principles of Social Research Methodology (pp. 29-41). Springer.
Li, G. (2022). How many students and items are optimal for teaching level evaluation of college teachers? Evidence from generalizability theory and Lagrange multiplier. Sustainability, 15(1), 2.
Marcoulides, G. A. (2000). Generalizability theory. In Handbook of applied multivariate statistics and mathematical modeling (pp. 527-551). Elsevier.
Oduro-Okyireh, G. (2020). Dependability of Students ‘Internship Mentors ‘Results Using Generalizability Theory at University of Education, Winneba.
Patrick, H., French, B. F., & Mantzicopoulos, P. (2020). The reliability of framework for teaching scores in kindergarten. Journal of Psychoeducational Assessment, 38(7), 831-845.
Quansah, F., & Ankoma-Sey, V. R. (2020). Evaluation of pre-service education programme in terms of educational assessment. Int. J. Res. Teach. Educ, 11, 56-69.
Ramadhan, S., Nasran, S. A., Utomo, H. B., Musyadad, F., & Ishak, S. (2019). The implementation of generalisability theory on physics teachers’ competency assessment instruments development. International Journal of Scientific and Technology Research, 8(7), 333-337.
Shavelson, R. J., & Webb, N. M. (1991). Generalizability Theory: A Primer. SAGE.
Webb, N. M., & Shavelson, R. J. (2005). Generalizability theory: overview. Encyclopedia of statistics in behavioral science.
Webb, N. M., Shavelson, R. J., & Haertel, E. H. (2006). 4 reliability coefficients and generalizability theory. Handbook of statistics, 26, 81-124.
Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness.
Wiley, E. W., Webb, N. M., & Shavelson, R. J. (2013). The generalizability of test scores. In APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology. (pp. 43-60). American Psychological Association.
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2024 AFRICAN JOURNAL OF APPLIED RESEARCH
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
By submitting and publishing your articles in the African Journal of Applied Research, you agree to transfer the copyright of the Article from the authors to the Journal ( African Journal of Applied Research).
