CEO Mafu Rakometsi: Pronouncement on 2022 National Examination Results

Opening Remarks by the CEO of Umalusi, Dr MS Rakometsi, at the Pronouncement on the 2022 National Examination Results, held on 16 January 2023, Tlhame Boardroom, Umalusi House, Pretoria

Thank you Mr Lepota - Mr Biki Lepota is the Senior Manager for Public Relations and Communications at Umalusi.

Acknowledgements

• Colleagues, good morning to you all.
• I would like to start by introducing to you the following members of our panel and other dignitaries:
• Professor Yunus Ballim is the Chairperson of Umalusi Council,
• I acknowledge members of Umalusi Executive Committee of Council (in absentia),
• Deputy Directors-General of the DHET: Ms Thembisa Futshane and Mr Sam Zungu
• Mr Keith Maseko - CEO of SACAI
• Ms Zodwa Modimakwane: Executive Manager: for the Quality Assurance and Monitoring Branch,
• Mr Emmanuel Sibanda: Executive Manager: for the Qualifications and Research Branch,
• Mr Hendrik van der Walt - CFO
• Ms Mary-Louise Madalane: Senior Manager for Quality Assurance of Assessment Unit (School Qualifications),
• Dr Mary-Antoinette Dliwayo: Senior Manager for Quality Assurance of Assessment Unit (Post-School Qualifications),
• All other Umalusi Senior Managers present;
• The hardworking staff of Umalusi,
• Representatives of different assessment bodies,
• Journalists representing different media houses,

Ladies and Gentlemen,

I would like to take this opportunity to warmly welcome you all to this Umalusi media briefing on the results of national examinations that were administered by different assessment bodies between October and December 2022.

We make this pronouncement at the time when Umalusi has just begun the journey of its next 20 years in pursuit of equity in the provision of quality education for all in the general and further education and training sector.

I would like to start by expressing my deep appreciation to all the media houses that are here today to ensure that the message of this briefing reaches the people of South Africa.

Let me start off by clarifying the much talked about delays in the release of results. I am sure that you are all aware that Covid-19 pandemic hit us in 2020. As a consequence, matters of timetabling (whether you talk about teaching, examinations or standardisation of results) have been affected since then.

The schools started late in 2022 as a result of the knock-on effect on the disruptions of 2020 and 2021. The schools had to do a lot of catching up as the class of 2022 has suffered the full impact of the pandemic. The catching up programme needs extra time for the schools. As a result, the examinations and the processes that flow from the commencement of exams started later than usual. Hence the late release of the results.

Regarding today's approval date, this is something that was agreed upon between Umalusi and all assessment bodies in the first half of 2022.

As regards applications for further or higher education study, you are aware that the system opens in June and closes in September of the preceding year. Higher education institutions make provisional admission decisions based on the results that learners obtained while in Grade 11. Final decisions are taken as soon as the results are submitted.

Ninety-four (94) days ago on 14 October 2022 we informed the South African public that the public and private assessment bodies were ready to conduct, administer and manage the 2022 end of the year national examinations. We gave a full account with reasoning in terms of how Umalusi arrived at the conclusion that the system was ready.

Scope and size of the 2022 examinations

We are here today to make a pronouncement on the outcome of Umalusi's quality assurance processes for the 2022 national examination results of both the public and private assessment bodies in respect of the:

a) National Senior Certificate (NSC) - assessed by the Department of Basic Education (DBE); the Independent Examinations Board (IEB), and the South African Comprehensive Assessment Institute (SACAI);

b) National Certificate Vocational (NCV: L2 - L4) - assessed by the Department of Higher Education and Training (DHET);

c) NATED Report 190/191 (N2 - N3) - assessed by the DHET; and

d) General Education and Training Certificate: Adult Basic Education and Training (GETC: ABET) - assessed by the DHET, IEB and SACAI.

Overall, the November 2022 examinations were written by 1 222 652 candidates. This is the grand total for the system in its entirety: all candidates registered for the four qualifications across four assessment bodies.

Of this total, the NSC accounts for 940 366 candidates and is followed by the NC(V) with 166 184 candidates. The balance is accounted for by candidates who wrote the NATED (N2-N3) and GETC examinations. The approval statement to be read by Prof Ballim will provide the exact details by qualification and assessment body.

The 2022 NSC cohort is the ninth to sit for the examinations based on the Curriculum and Assessment Policy Statement (CAPS) curriculum. The year 2022 was dissimilar to the previous two years (2020 and 2021) because it was the year in which the examinations were written under no COVID-19 related restrictions.

While we acknowledge that the Class of 2022 may have suffered the worst part of COVID-19 in that they

were confronted with a disrupted academic year when in Grades 10 and 11, in 2020 and 2021 respectively, Umalusi is satisfied with the kinds of interventions that the Department of Basic Education (DBE) and all other assessment bodies put in place to reduce the negative impact of lost time emanating from the disruptions of the Covid-19 pandemic.

As a reminder, on 23 September 2022, the month during which Umalusi celebrated its Twentieth Anniversary, we launched an important research report titled 'International Benchmarking of the National Senior Certificate'. This study, which was commissioned to Ecctis, (formerly known as the UK NARIC), benchmarked the NSC against five international qualifications:

a) International Baccalaureate Diploma Programme, b) Kenyan Certificate of Secondary Education,

c) New South Wales Higher School Certificate,

d) the Zimbabwean Forms 5-6 advanced level, and

e) Cambridge Assessment International Education AS and A Levels.

The overall conclusion of the study is that the NSC provides its holders with an internationally comparable level of education. The full report is accessible on the Umalusi website (www.umalusi.org.za(link is external)).

How Umalusi discharged its mandate

As South Africa's Quality Council (QC) for General and Further Education and Training (GFET), Umalusi derives its mandate from two pieces of legislation, namely:

a) the General and Further Education and Training Quality Assurance (GENFETQA) Act (58 of 2001), as amended in 2008, and

b) the National Qualifications Framework Act (67 of 2008), as amended.

Section 17A of the same Act (GENFETQA) confers a range of powers and functions on Umalusi in relation to quality assurance. Amongst the functions included under this section are that:

(3) The Council must perform the external moderation of assessment of all assessment bodies and education institutions; and

(4) The Council may adjust raw marks during the standardisation process.

Umalusi exercises these powers and functions to both the public and private assessment bodies. The sections of the GENFETQA Act that I have just quoted signal that Umalusi is entrusted with a very important task of assuring the quality of the entire value chain of the national examination process.

It is the vision of Umalusi to be a trusted authority in fostering high education standards in general and further education and training (GFET) sector of South Africa. The operative word here is "trust". Therefore, at the heart of Umalusi's quality assurance processes is the need to inculcate "trust" in the people of South Africa and those beyond the borders.

One of the critically important tasks performed by Umalusi before the start of the examinations is the audit of the state of readiness of the public and private assessment bodies to manage and conduct national examinations. This was done and Umalusi reported publicly on 14 October 2022 that the system was ready to conduct, administer and manage the 2022 end of the year national examinations.

Quality assurance of assessment processes

Concerning quality assurance of assessment processes, Umalusi:

a) externally moderated and approved (i) question papers, (ii) School-Based Assessment (SBA) tasks, (iii) Site Based Assessment tasks (SBA) Common Assessment Task (CAT), (iv) Internal Continuous Assessment (ICAS) portfolios, and (v) Internal Continuous Assessment (ICASS),

b) monitored a sample of examination centres during the writing of the examinations,

c) participated in marking guidelines standardisation meetings in a selection of subjects,

d) monitored selected marking centres during the marking process, and

e) verified quality of marking in selected subjects.

Dealing with possible hindrances to the smooth running of the examinations

In November 2022 Umalusi expressed its concerns about the effect of loadshedding, community protests and reported cases of cheating on the smooth running and the integrity of the examinations.

With regard to load shedding, Umalusi is pleased that all assessment bodies heeded the advice to make alternative arrangements to mitigate the possibility of load-shedding particularly on the days when subjects such as Computer Applications Technology (CAT) and Information Technology (IT) were written.

In relation to community protests, Umalusi received reports that in instances where candidates either started the examinations later than the stipulated time or missed them completely on the scheduled date due to no faults of their own alternative arrangements were made for the affected candidates to write the examination like every other candidate.

The "special examinations" were based on externally moderated back-up question papers the standard of which compares with that of the original papers.

Umalusi appreciates the efforts of all the four assessment bodies for ensuring that candidates were shielded from all these hindrances and thereby ensuring that the integrity and credibility of the 2022 national examinations is not compromised.

Constant engagements with assessment bodies

In the performance of its legislated areas of work, Umalusi leaves no assessment body behind. Umalusi engages constantly with assessment bodies. To illustrate the point, during the year 2021 Umalusi embarked on a process to review and strengthen the approaches and principles used to standardise examination results. The process was conducted through work-streams and letters were sent to assessment bodies inviting them to submit proposals to Umalusi in terms of how to strengthen the approaches and principles.

In consideration of the proposals received from assessment bodies Umalusi Council approved, on the recommendation of the Assessment Standards Committee (ASC), the slight amendments introduced to the current principles. The intention with the amendments has been to bring greater degree of clarity and simplification from the point of view of language. Upon their approval, the principles were submitted to all assessment bodies in May 2022.

About the reported cases of examination irregularities

Another equally important matter about which Umalusi remains seriously concerned are reported cases of irregularities during the writing of the examinations. According to the reports received from different assessment bodies, different forms of cheating were uncovered in some centres during the examinations. These include candidates found in possession of crib notes and/or cell phones, sharing of answers via WhatsApp groups, imposters were found in the examination rooms, some answer scripts had different handwritings, etc.

Based on the magnitude of the offences, different sanctions have been meted against the candidates involved such as the nullification of the results, barring the candidates from writing the examination for either 1 or 2 examination cycles. Professor Ballim will present more details on these matters in his report.

Umalusi gives commendation to all the assessment bodies for having uncovered the malpractices and dealing with the culprits. This is one important way of safeguarding the integrity and credibility of all the qualifications on the Umalusi sub-framework of qualifications.

Dealing with problematic examination questions

Umalusi remains concerned about some of the questions that were found problematic in some subjects during this examination cycle. Let me briefly share with you examples of cases where questions were excluded from the marking process because of their problematic nature.

Examples of cases where questions were excluded a) Independent Examinations Board (IEB)

i. NSC - History Paper 2: Question 3.4.1 (2 marks) referred candidates to find a historical concept in Paragraph 2 instead of Paragraph 1.

ii. GETC- English: Questions 5(a) and 5(b) worth 4 marks required candidates to identify the underlined words and write them in full where no words were underlined.

b) Department of Basic Education (DBE)

i. NSC - Mathematical Literacy Paper 2: Question

4.2.6 (6 marks) erroneously refers candidates to the year 2019 instead of 2020.

ii. NSC - Sepedi Home Language Paper 2:

Question 2.5 (2 marks) refers to an incorrect word that was used. The wrong word "kgakantšho" instead of "kgakantšhano" was used.

iii. These problematic questions resulted in marking concessions being applied for by the relevant assessment bodies for them to be excluded either entirely or partially from the question papers. Some of the requests for marking concessions were granted whereas others were rejected.

Regarding the problematic Question 5.1 allocated 7 marks in Mathematics Paper 2 Professor Ballim will present findings of the report of a three-person independent panel appointed by Umalusi 'to investigate how the error/s in the Mathematics Paper 2 may have crept in and went undetected until the final question paper was administered by the DBE on 7 November 2022.

What I can report now is that prior to the receipt of the report, Umalusi had, based on its obligation to mitigate the possible impact of the error on leaner performance, advised the DBE to exclude the faulty question in its entirety from the marking. Therefore, no marks were allocated to Question 5.1 (7 marks), thereby reducing the total of 150 marks by 7 marks to 143 marks for all candidates. All candidates were marked out of a total of 143 marks and the marks achieved by candidates were converted to a mark out of 150 marks.

Such errors cannot be justified and are therefore regrettable. Umalusi wishes to apologise profusely for these errors in question papers. The South African public must find comfort in the fact that Umalusi, as a Quality Council, has tried and tested methods to remedy the situation where such unfortunate errors are detected late in the question paper. Our tried and tested approaches ensure that no learner is disadvantaged by the error in the question paper. The mitigating measurers are implemented either during marking or during the standardisation meetings.

"The greatest mistake is to imagine that human beings will never commit mistakes." Ladies and gentlemen, mistakes will always happen where human beings are involved.

The test of the resilience of the system is found in a system that is able to self-correct and normalise the situation in the unlikely event of errors being detected late in the value chain.

Umalusi will be hosting a webinar on (1) question paper development processes, (2) the use of the missing script formula, (3) examinations concessions and (4) the scaling up formula used when a question is discarded because it was wrong. The webinar will be in either February or March 2023. The members of the media will be invited to this webinar.

Regarding all matters concerning how these examinations were run, Umalusi is finalising detailed reports to highlight areas of non-compliance and directives for compliance and improvement for the future for all assessment bodies.

About standardisation of examination results

Standardisation is a quality assurance process that is based on principles and is used worldwide to mitigate the impact on learner performance caused by factors other than the learners' subject knowledge, abilities and aptitude.

Any large-scale examination process invariably has many and varied sources of variability. Most of these are unplanned, unintended, and undesirable. They vary from mistakes in a question paper to subtle issues of level of difficulty of a question paper and possible multiple and valid interpretations of questions. In the main, standardisation has two objectives:

a) First, to ensure that a cohort of learners is not advantaged or disadvantaged by factors other than their knowledge of the subject, abilities and their aptitude.

b) Second, to achieve comparability and consistency of learner performance across the years and the various subjects within the given year.

The approach that Umalusi uses to standardise marks is norm referencing. In simple terms, this means that whenever there is deviation from history (norm) in terms of performance, the whole distribution, not individual marks, is adjusted either upward or downward toward the norm.

As I have mentioned already, the direction of the adjustment (downward or upward) is based on the evidence presented. Subjects are standardised individually and in a non-iterative manner.

Let me take the opportunity to assure all South Africans that the Assessment Standards Committee (ASC) of Umalusi Council arrives at standardisation decisions after going through quantitative and qualitative reports to find evidence to serve as the basis for its decisions. Thus, the decisions of the committee are not based solely on statistical information. On the contrary, all forms of evidential information are considered.

Umalusi is deeply indebted to the ASC because it carries responsibility for ensuring that Umalusi truly honours its vision of being a trusted authority in fostering high education standards in GFET.

The ASC is constituted by highly qualified academics who are willing to sacrifice their valuable time during the festive season to ensure the best outcome for our learners.

As I conclude, I want to inform members of the public that Umalusi is seriously concerned about candidates who register to write examinations at unaccredited private centres. It is equally concerning that there are some unaccredited independent schools and private colleges that admit learners fully aware that they are operating unlawfully.

In this regard, we urge the public to access a presentation titled "What to look for in choosing an independent school or private college" on the Umalusi website (www.umalusi.org.za(link is external)). This information is very useful because it assists you to verify if an independent schools or a private college is accredited before you can register with it.

Finally, I want to express Umalusi' deep appreciation to every single player in both the public and private assessment systems for putting great effort into ensuring that the Class of 2022 reaches this stage.

Ladies and gentlemen, I thank you. Please join me in welcoming the Chairperson of the 6th Umalusi Council, Professor Yunus Ballim, to present the details on the findings of Umalusi's quality assurance in respect of the results of examinations conducted and managed by the IEB, SACAI, DBE and DHET in November 2022.

Thank you.

Public link

Please use the above public link if you want to share this noodl on another website.