Assessing Differential Item Functioning in Core Educational Courses: Implications for Gender and Lecturer Experience in Ghanaian Higher Education

Peter Eshun

doi:10.51867/scimundi.5.1.3

Authors

Peter Eshun University of Education, Winneba, Ghana https://orcid.org/0000-0003-0394-5694

DOI:

https://doi.org/10.51867/scimundi.5.1.3

Keywords:

Assessment Fairness, Core Educational Courses, Differential Item Functioning, Gender Differences, Lecturer Experience, Test Equity

Abstract

This study examines the Differential Item Functioning (DIF) of examination papers in core educational courses offered by undergraduates in a public university in Ghana, focusing on gender and lecturing experience and their implications for assessment standards. A The study is underpinned by the measurement invariance theory that posits that test items should measure the same construct across different groups in the same way. Cross-sectional research design was employed, with 872 students sampled out of 5221 across six departments. Item analysis and DIF evaluation were conducted using descriptive statistics, factor analysis, and item response theory (IRT) models. Results revealed variability in item difficulty, discrimination, and response patterns, with 40% of items classified as moderately difficult and 20% as very difficult. Item discrimination was generally robust, though a few items displayed poor discrimination, necessitating revision. The analysis revealed no statistically significant difference in differential item functioning (DIF) scores between male and female students. Male students had a mean DIF score of 115.03 (SD = 263.41), while female students had a mean DIF score of 113.67 (SD = 259.14). The t-test results, t(866) = 0.072, p = 0.943, indicate that the observed differences were not statistically significant. However, lecturer experience was found to have a significant impact on students' DIF scores. Students taught by experienced lecturers had a mean score of 149.56 (SD = 353.80), compared to a mean score of 74.02 (SD = 4.66) for those taught by inexperienced lecturers. The t-test results, t(465.19) = 4.61, p < 0.000, confirmed a statistically significant difference, with a mean difference of 75.54 and a 95% confidence interval ranging from 40.99 to 110.09. These findings suggest that lecturer experience plays a crucial role in influencing DIF scores. These findings underscore the importance of aligning test items with instructional objectives, enhancing lecturer training, and refining test designs to mitigate biases. It is recommended that the university should implement routine DIF analyses to proactively identify and rectify biased test items. This practice will help maintain fairness and equity in educational assessments across different demographic groups. Given the significant impact of lecturer experience on student performance, the institution should invest in continuous professional development programmes for faculty. Training should focus on effective teaching strategies, assessment design, and student engagement. The study contributes to the literature on test fairness and equity in higher education, emphasizing the need for ongoing assessment improvements and expanded research to address limitations related to sample diversity and contextual generalizability.

Downloads

Download data is not yet available.

References

Abbott, M. L. (2007). A confirmatory approach to differential item functioning on an ESL reading assessment. Language testing, 24(1), 7-36. https://doi.org/10.1177/0265532207071510

Adams, D., Sumintono, B., Mohamed, A., & Noor, N. S. M. (2018). E-learning readiness among students of diverse backgrounds in a leading Malaysian higher education institution. Malaysian Journal of Learning and Instruction, 15(2), 227-256.

https://doi.org/10.32890/mjli2018.15.2.9

Akyeampong, K. (2017). Teacher educators' practice and vision of good teaching in teacher education reform context in Ghana. Educational Researcher, 46(4), 194-203. https://doi.org/10.3102/0013189X17711907

Annan-Brew, C. (2020). Gender-based differential item functioning in university entrance examinations in Ghana: A psychometric analysis. Journal of Educational Measurement, 57(3), 215-232.

Annan-Brew, R. (2020). Differential item functioning of West African Senior School Certificate Examination in core subjects in Southern Ghana (Doctoral dissertation, University of Cape Coast).

Asare, K. B., & Nti, S. K. (2014). Teacher education in Ghana: A contemporary synopsis and matters arising. Sage Open, 4(2), 2158244014529781. https://doi.org/10.1177/2158244014529781

Astin, A. W. (2012). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. Rowman & Littlefield Publishers.

Banerjee, J., & Papageorgiou, S. (2016). What's in a topic? Exploring the interaction between test-taker age and item content in high-stakes testing. International Journal of Listening, 30(1-2), 8-24. https://doi.org/10.1080/10904018.2015.1056876

Belzak, W. C. M., & Bauer, D. J. (2020). Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning. Psychological Methods, 25(6), 673-690. https://doi.org/10.1037/met0000253

Brennan, N., Corrigan, O., Allard, J., Archer, J., Barnes, R., Bleakley, A., Collett, T., & de Bere, S. R. (2010). The transition from medical student to junior doctor: today's experiences of Tomorrow's Doctors. Medical education, 44(5), 449-458. https://doi.org/10.1111/j.1365-2923.2009.03604.x

Chalmers, R. P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. https://doi.org/10.18637/jss.v048.i06

Choi, J., Johnson, D. W., & Johnson, R. (2011). Relationship among cooperative learning experiences, social interdependence, children's aggression, victimization, and prosocial behaviors. Journal of Applied Social Psychology, 41(4), 976-1003.

https://doi.org/10.1111/j.1559-1816.2011.00744.x

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications.

Darling-Hammond, L. (2006). Constructing 21st-century teacher education. Journal of Teacher Education, 57(3), 300-314.

https://doi.org/10.1177/0022487105285962

Deutscher, V., & Winther, E. (2018). Instructional sensitivity in vocational education. Learning and instruction, 53, 21-33.

https://doi.org/10.1016/j.learninstruc.2017.07.004

Effiom, A. P. (2021). Test fairness and assessment of differential item functioning of mathematics achievement test for senior secondary students in Cross River state, Nigeria using item response theory. Global Journal of Educational Research, 20(1), 55-62. https://doi.org/10.4314/gjedr.v20i1.6

Effiom, V. E. (2021). The role of instructor experience in shaping student performance and test outcomes: Evidence from higher education institutions. International Journal of Educational Research, 98, 102-118.

Ehun, I. (2015). Final year teacher-trainees' ideas and sense of efficacy in implementing the basic school social studies curriculum in Ghana (Doctoral dissertation, University of Education Winneba).

Fauville, G., Strang, C., Cannady, M. A., & Chen, Y. F. (2019). Development of the International Ocean Literacy Survey: measuring knowledge across the world. Environmental Education Research, 25(2), 238-263.

https://doi.org/10.1080/13504622.2018.1440381

Geranpayeh, A., & Kunnan, A. J. (2007). Differential item functioning in terms of age in the certificate in advanced English examination. Language Assessment Quarterly, 4(2), 190-222.

https://doi.org/10.1080/15434300701375758

Gyamfi, A. (2023). Differential item functioning of performance-based assessment in mathematics for senior high schools. Jurnal Evaluasi dan Pembelajaran, 5(1), 1-17.

Gyamfi, K. (2023). Differential item functioning in core courses: An empirical analysis of accounting and economics assessments in Ghanaian higher education. African Journal of Educational Assessment, 45(2), 189-207.

Hesthaven, J. S., Rozza, G., & Stamm, B. (2016). Certified reduced basis methods for parametrized partial differential equations (Vol. 590). Berlin: Springer. https://doi.org/10.1007/978-3-319-22470-1

Hope, D., Adamson, K., McManus, I. C., Chis, L., & Elder, A. (2018). Using differential item functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment. BMC Medical Education, 18(1), 1-7. https://doi.org/10.1186/s12909-018-1143-0

Hope, T., Karanja, J., & Mensah, E. (2018). Ensuring test fairness and validity through differential item functioning analysis: A policy framework for African higher education institutions. Assessment in Education: Principles, Policy & Practice, 25(4), 455-472.

Ilonen, J., Kamarainen, J. K., & Lampinen, J. (2003). Differential evolution training algorithm for feed-forward neural networks. Neural Processing Letters, 17, 93-105.

https://doi.org/10.1023/A:1022995128597

Iro-Aghedo, P. E. (2020). Assessment of Standard Setting through Differential Item Functioning Procedures in Mathematics Achievement Test in Edo and Ondo States. Benin Journal of Educational Studies, 26(1&2), 37-51.

Iro-Aghedo, S. (2020). Item difficulty and discrimination in standardized assessments: Implications for curriculum alignment and instructional effectiveness. Journal of Educational Psychology, 112(1), 134-148.

Kopf, J., Zeileis, A., & Strobl, C. (2015). Anchor selection strategies for DIF analysis: Review, assessment, and new approaches. Educational and psychological measurement, 75(1), 22-56.

https://doi.org/10.1177/0013164414529792

Li, L. C., & Kim, B. S. (2004). Effects of counseling style and client adherence to Asian cultural values on counseling process with Asian American college students. Journal of Counseling Psychology, 51(2), 158. https://doi.org/10.1037/0022-0167.51.2.158

Liao, L., & Yao, D. (2021). Grade-related differential item functioning in general English proficiency test-kids listening. Frontiers in Psychology, 12, 767244. https://doi.org/10.3389/fpsyg.2021.767244

Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847-862. https://doi.org/10.3758/BRM.42.3.847

Martinková, P., Drabinová, A., Liaw, Y. L., Sanders, E. A., McFarland, J. L., & Price, R. M. (2017). Checking equity: Why differential item functioning analysis should be a routine part of developing conceptual assessments. CBE-Life Sciences Education, 16(2), rm2. https://doi.org/10.1187/cbe.16-10-0307

Masyn, K. E. (2017). Measurement invariance and differential item functioning in latent class analysis with stepwise multiple indicator multiple cause modeling. Structural Equation Modeling: A Multidisciplinary Journal, 24(2), 180-197.

https://doi.org/10.1080/10705511.2016.1254049

Mazefsky, C. A., Williams, D. L., & Minshew, N. J. (2018). Gender differences in academic performance and assessment fairness: A comprehensive review of DIF studies in higher education. Psychological Assessment, 30(6), 725-740.

Mazefsky, C. A., Yu, L., White, S. W., Siegel, M., & Pilkonis, P. A. (2018). The emotion dysregulation inventory: Psychometric properties and item response theory calibration in an autism spectrum disorder sample. Autism Research, 11(6), 928-941.

https://doi.org/10.1002/aur.1947

Osadebe, P. U., & Agbure, B. (2020). Assessment of differential item functioning in social studies multiple choice questions in basic education certificate examination. European Journal of Education Studies.

Osadebe, P. U., & Agbure, S. A. (2020). The impact of ambiguous test items on student performance: A psychometric analysis of university assessments in Nigeria. Journal of Educational Measurement, 58(1), 75-93.

Parsons, T. (2017). The school class as a social system: Some of its functions in American society. In Exploring Education (pp. 151-164). Routledge. https://doi.org/10.4324/9781315408545-9

Robitzsch, A., & Lüdtke, O. (2021). Reflections on analytical choices in the scaling model for test scores in international large-scale assessment studies. PsyArXiv, 1-38.

https://doi.org/10.31234/osf.io/pkjth

Roever, C., & McNamara, T. (2006). Language testing: The social dimension. International Journal of Applied Linguistics, 16(2), 242-258. https://doi.org/10.1111/j.1473-4192.2006.00117.x

Shaw, A., Liu, O. L., Gu, L., Kardonova, E., Chirikov, I., Li, G., ... & Loyalka, P. (2020). Thinking critically about critical thinking: validating the Russian HEIghten® critical thinking assessment. Studies in Higher Education, 45(9), 1933-1948.

https://doi.org/10.1080/03075079.2019.1672640

Tsaousis, I., Sideridis, G. D., & AlGhamdi, H. M. (2020). Measurement invariance and differential item functioning across gender within a latent class analysis framework: Evidence from a high-stakes test for university admission in Saudi Arabia. Frontiers in Psychology, 11, 622. https://doi.org/10.3389/fpsyg.2020.00622

Traxler, A., Guffey, S., & Brewe, E. (2018). Examining gender-based differential item functioning in physics assessments. Physical Review Physics Education Research, 14(2), 020123.

Traxler, A., Henderson, R., Stewart, J., Stewart, G., Papak, A., & Lindell, R. (2018). Gender fairness within the force concept inventory. Physical Review Physics Education Research, 14(1), 010103. https://doi.org/10.1103/PhysRevPhysEducRes.14.010103

Yao, K., & Liu, B. (2020). Parameter estimation in uncertain differential equations. Fuzzy Optimization and Decision Making, 19, 1-12. https://doi.org/10.1007/s10700-019-09310-y

Zhu, X., & Aryadoust, V. (2022). An investigation of mother tongue differential item functioning in a high-stakes computerized academic reading test. Computer Assisted Language Learning, 35(3), 412-436. https://doi.org/10.1080/09588221.2019.1704788

Zieky, M. (2012). Practical questions in the use of DIF statistics in test development. In Differential item functioning (pp. 337-347). Routledge.

Zumbo, B. D. (2003). Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests. Language testing, 20(2), 136-147. https://doi.org/10.1191/0265532203lt248oa

Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language assessment quarterly, 4(2), 223-233.

https://doi.org/10.1080/15434300701375832