CXC controversy: Review cites poor assessments, communication

The controversy over lower-than-expected grades on Caribbean examinations can be attributed largely to inaccurate teacher assessments, data failures and communications errors, according to an independent review of 2020 CSEC and CAPE exam results.

While none of those issues were new to this year’s Caribbean Examinations Council (CXC) tests, the unique conditions brought about by the COVID-19 pandemic accentuated existing flaws in the process.

The panel stood by the overall results of the CXC exams, which as a whole were in line with scores in past years. However, the report includes 23 recommendations to remedy the “crisis of expectations in 2020” and to improve the testing process in the future.

Of the conclusions reached by the review panel, consisting of academics and education officials from several CXC countries, the one that has gained the most attention regionally is the finding that “school-based assessments” or SBAs (i.e. grades that teachers predicted their students would attain) tended to award students higher marks than they achieved on the actual exams.

According to the report, “a statistical analysis of data for previous years found a weak correlation between the final grade for the three papers and the teachers’ predicted grade in general”.

Further, while in the past CXC examiners reviewed a random sample of SBAs for moderation, this year all SBAs were subject to moderation. As in years past, examiners found that SBAs they marked tended to have lower grades than the ones awarded by teachers, for reasons ranging from “lack of thoroughness” to outright “mark inflation”.

Tables attached to the report show that from 2017-2020, anywhere from 30%-43% of teachers’ predicted grades on CSECs were significantly different from grades given by moderators. In 2020, the amount was 36%.

The inconsistency on CAPE exams was greater. From 2017-2020, anywhere from 42%-66% of teachers’ predicted grades on CAPEs were significantly different from grades given by moderators. In 2020, the amount was 66%, slightly greater than the 63% incidence of low correlation in 2019.

According to the report:

“The Chief Examiners attributed the higher marks that teachers tended to award for the SBA to a number of factors, including:

i. the lack of thoroughness and vigilance by teachers while marking SBAs, leading to the award of full marks in some instances for areas that students did not even attempt;

ii. projects submitted by a number of candidates were inconsistent with the expectations of the scoring rubrics and the requirements of the syllabus;

iii. overuse of informal sources from the internet which detracted from the quality of SBA assignments;

iv. the misguided belief that the CXC moderators would be inclined to reduce teachers’ marks and that some element of mark inflation would protect their students from falling below their expected grade.”

Students’ total CXC scores were determined by combining the SBA results (lower than in past years) with the results of multiple-choice exams — which were higher than in past years.

“While there was an overall lowering of SBA scores resulting from the expanded process, the improved performance of students in the Paper I Examination resulted in the overall grade distribution of students in 2020 being comparable to previous years,” according to the report.

One possible factor cited by the report for the improved multiple-choice exam results is “greater focus on preparation of students for this format”. Another possible factor is that questions on the CSEC and CAPE exams were found circulating in the public domain.

“This issue must be addressed urgently” by creating new items for the exams, according to reviewers.

In addition to the downward adjustment of SBAs, some students who sat exams received a preliminary result of ‘no grade’. Reviewers attribute that to “school-related issues” such as incorrect registration of students, non-submission of scores, missing documents, corrupt files, etc., as well as issues on the CXC end related to file processing.

Overall, “The Team concluded that there were challenges with communication about the examinations in 2020 and some improvement with communication between CXC and schools would have helped diminish the concerns expressed by teachers, students and parents.”

About 122,000 students took CSEC exams this year, and nearly 40,000 students took CAPE exams. Far ahead of the deadline to submit requests for review, CXC had fielded nearly 5,000 submissions.

The report’s recommendations include:

Immediate term

Extend deadline to submit requests for review beyond 23 Oct. (extended to 5 Nov.)
Reduce cost for requests for review (cut by 50% to US$15)
Include explanationatory remarks in reviews
Expedite requests for review
Do not lower grades for exams that are reviewed — only increase or keep scores the same
Improve system for query requests, separate from requests for review
Hold ‘reflection workshop’ to improve communication with local registrars
“Execute a damage control campaign” in the media

Mid-to-Long term

Provide training for teachers on SBA scoring
Provide training for teachers on SBA prep for students
Provide training for teachers on preparing and uploading SBAs
Create education campaigns for students on SBAs
Improve technology for handling large files
Audit the standardisation process for exam moderation
Provide training for examiners
Regularly create new questions for exams and institute stricter security measures on question bank
Hold campaign to strengthen relationship between CXC and teachers
Improve and increase sampling of exams throughout territories
Audit role of local registrars in terms of communication, problem solving and training
Create individual portals on CXC website for stakeholders such as students, teachers, registrars, testing centres and parents
Establish research unit to mine data being collected by CXC to improve policy
Undertake contingency planning
Strengthen biennial stakeholder satisfaction surveys

The CXC grades and report have generated controversy across the Caribbean, sparking demonstrations, government protests, potential lawsuits and possible sanctions against teachers.

Here is a sample of news headlines from CXC member countries:

Barbados

British Virgin Islands