DEVELOPMENT AND VALIDATION OF A DIFFERENTIATED PBL ASSESSMENT RUBRIC FOR INDONESIAN ELT

Abstract

Indonesian English Language Education department is increasingly filled with diverse learners requiring an assessment strategy, but no empirically-based rubric can be found to evaluate the Project-Based Learning (PBL) outcomes in Indonesian EFL context. This study addresses this gap by creating and validating the Differentiated Assessment Rubric for PBL (DAR-PBL), which incorporates flexible weighting of criteria and allows students to choose focus of assessment, in order to leverage learner diversity. This study was conducted in four phases (rubric development, content validity assessment with three expert judges, face and construct validity using six lecturers and 20 students as participants, and inter-rater reliability testing with 20 authentic PBL work samples scored by six raters) following DeVellis’s (2017) validation framework. Results Quantitative analyses included Aiken’s V for content validity, mean comparisons for face and construct validity; Gwet’s consensus reliability coefficient (AC1) was used to calculate the intraclass correlation coefficients (ICC) with Fleiss’ Kappa. Qualitative thematic analysis of the open-ended responses provided supplementary support. Results showed excellent content validity because five of six criteria resulted in Aiken’s V > 0.75 (English Language Use :V=1.00). Face validity refers to the 4.0 mark (M=4.50 lecturers, M=4.35 students), and construct validity were also robust (M=4.61lecturers / M=4.42 students). Inter-rater reliability was good (ICC=0.87, Fleiss’ Kappa=0.68) and test-retest reliability excellent (mean r=0.91). The DAR-PBL represents a valid working tool that can help Indonesian ELT lecturers assess PBL in a fair manner, with implications for differentiating assessment practices across EFL contexts.