Local Early Grade Reading Assessment to Raise Reading Standards in Lower Primary Classes in Rwanda

Group Submission Type: Formal Panel Session

Proposal

The need for regularly collected population-level assessment data on student performance in early grade reading is driven by the role that such data can play in improving teacher instruction. It also provides a dipstick of system level reading performance. This metric not only relates to student preparation for later grades and success in school but to potential dropout rates and helps identify at risk students.

The panel explores various aspects of the introduction of a comprehensive assessment regime in Rwanda in line with the Rwandan government policy on assessment. Working in collaboration with the Rwandan Education Board (REB), USAID Soma Umenye has been innovating with specific ways of measuring population-level early grade reading performance in the home language (Kinyarwanda) while creating a cycle of actionable data for teachers to use in real-time to identify struggling learners and put in place guided remediation activities. The aim of this panel session will be to present this innovation, called Local Early Grade Reading Assessment or LEGRA.

Prior to the piloting of LEGRA, Rwanda - in line with most African countries - undertook irregular sample-based reading assessments. These assessments used a range of benchmarks to ascertain oral reading fluency (ORF). As a result, it was impossible to draw out any sense of longitudinal progressive system-level improvement. Given this, LEGRA was predicated by REB, working with USAID Soma Umenye, putting in place ORF benchmarks for grades 1 to 3 with cutscores developed using the Modified Angoff method. The benchmarks are broadly in line with international norms for languages with many compounded words.

Once the benchmarks were in place, USAID Soma Umenye explored possible models for providing early grade teachers with a tool for assessing all their students home language skills. The work that RTI had been conducting for USAID on Local Education Monitoring Approach (LEMA), and specifically Group Administered Literacy Assessment (GALA) offered the most suitable starting point. Based on quality assurance processes in the industry, it talked to a representative sampling approach to assessing early grade reading. We quickly realized that if teachers could be trained to administer these GALA-style tests in their own classroom with their own students that they could test all early grade students.

The contents for the group administered tests were developed and the approach and tests presented to senior REB assessment officials in a pre-pilot in 10 schools August 2019. These tests (and GALA) do not measure ORF and comprehension. REB indicated that it wanted these added. USAID Soma Umenye worked with REB to design a way of preparing teachers to conduct both the group administered and one-on-one ORF and comprehension tests. This was piloted in late September 2019 in 5 districts (out of 30 national districts), with a total of over 170,000 students being assessed.

The pilot included 4 subtests: two group administered letter and word recognition and dictation subtests and two one-on-one ORF and reading comprehension subtests. These together provide a rounded picture of a student’s literacy capacity. Prior to the pilot, 2,065 teachers from 341 schools were trained in administering the assessment. They were trained over a single day on the technicalities of undertaking each test and it was impressed on them that, as the main user of the data, there was no gain in cheating to overstate their students’ performance.

The LEGRA process in each school involved a pre and post assessment meeting. In these, the early grade teachers met with their headteacher. In the first meeting, the teachers predicted how their students would perform and estimated the proportion of their students who would reach the expected level of achievement for their grade. In the second meeting, immediately after the testing, the teachers compared the actual performance with their predictions and planned how they would use the results to target remedial support to those students who require it.

Besides reporting on the LEGRA pilot and results and its rollout, which was delayed by the COVID-19 lockdown, but we expect to implement before CIES 2021, the panel session will explore the data turn-around time. The teacher has the full data set and graphics for her class within a day of completing the assessment. This real-time full population data has much more value to a teacher than months-old sample data which with teachers are usually presented. We will also discuss the system-level remediation innovation that REB has been trialing with USAID Soma Umenye. These Kinyarwanda Reading Camps (KRCs) were piloted in December 2019.

Apart from using the data generated by LEGRA to inform remediation at school level, it is uploaded onto tablets in each school. The program generates clear graphics and reports which analyze those data as well as providing advice to the school on how to respond to the specific grade level data. These data generate consolidated graphics and interpretation at sector and district level and then are uploaded onto a national dashboard. At each level, the data is used to inform a series of community meetings or inama. At these inama, the data is presented in graphic form and the implications of the data discussed with stakeholders. At school level, the stakeholders are the parents, early grade teachers, community leaders and school owners; at sector level, the sector education officers, primary school headteachers and community leaders; at district level, the district education officials, mayor and vice mayor, along with the district political, religious and community leadership. At national level, the inama becomes a learning laboratory with senior education officials and civil society organizations chaired by the Minister.

The various presentations in this panel unpack the origins, design and piloting of LEGRA, and the institutionalization of student remediation through the Kinyarwanda Reading Camps. They posit LEGRA and the KRCs within the context of REB’s Comprehensive Assessment regime and can provide a model for how this may be expanded to other contexts.

Sub Unit

• Global Literacy SIG

Individual Presentations

• Raising Standards in Lower Primary Education in Rwanda - Protogene Ndahayo, Chemonics International

• Delivering large scale local early grade reading assessment in Rwanda - Martin Prew, University of Witwatersrand

• Local early grade reading assessment in Rwanda embedded within the system - Stephen Derek Blunden, Chemonics