A survey of selected schools on the 2012 grade 9 Mathematics ANA results: Implications for teaching and learning:
Dr VG Govender (Advisory Committee for Mathematics) (ACM)
Introduction and background
Many countries have introduced large-scale educational assessment programmes as part of their public policy. These large-scale assessments are used not only to measure student achievement but also to hold schools accountable for the educational outcomes of students (Crundwell, 2005; Earl, 1999). In Britain and Chile, countries with highly centralised educational curriculum commonly developed at the national level, have large-scale testing programmes designed either to monitor or certify student achievement in relation to national standards (National Assessment Agency, 2008, Organization for Economic Co-operation and Development , 2004; Qualifications and Curriculum Authority, 2008)
The South African curriculum is also highly centralised and all policy matters with regard to curriculum are taken at a National level.The South African Government has identified the improvement of the quality of basic education as the top priority of the country and this has to be delivered by the Department of Basic Education (DBE). In this regard, the Annual National Assessment (ANA) is a critical measure for monitoring progress in learner achievement (DBE, 2012).
ANA, a large-scale national assessment, was introduced by the DBE in South African schools in 2011. This was conducted in Language and Mathematics and involved grades 1-6. In 2012, this was extended to grade 9. The average percentage for Mathematics grade 9 in 2012 was 13% (DBE, 2012:3). This caused a huge outcry in the country.
The Sunday Times in South Africa (9 December 2012) described the grade 9 and other mathematics tests under the heading “Errors blamed for maths test 'disaster'” in which some of the questions in the test were criticised as being “mathematical unsound” . A summary of some of the points of the Sunday Times report is shown below:
The Association for Mathematics Education in South Africa (AMESA) was asked to given input on the standard of the grade 9 Mathematics paper. The following key points were captured from the AMESA report:
The Sunday Times and AMESA reports focus on different but important issues. Questions need to be asked about what has led to this unacceptable learner performance in a large-scale assessment in a key school subject, Mathematics.
Statement of the problem
In both the Sunday Times and AMESA reports, mention is made of teaching and learning issues although, in the Sunday Times case, it does not grab the headlines. The AMESA report speaks about didactical obstacles such as the quality of teaching which children receive. In this regard, it well known that teaching and learning are the key drivers of assessment. In a large-scale assessment as ANA, although the focus is on the final results, the writer believes that the dismal average of 13% for grade 9 Mathematics ANA is symptomatic of state of mathematics teaching in the senior phase (grades 7 – 9) in South Africa. Over the years, the DBE and other stakeholders have paid more attention to the further education and training (FET) band, especially grade 12, and neglecting other phases. It is possible that this neglect has contributed this poor state of teaching and learning of mathematics in grade 9. This study attempts to provide some insight into this matter with a survey of selected schools using various criteria which are spelt out in the research questions and sub-questions in the next section.
Statement of the problem
In the light of the discussion in the previous two sections, the following research question was proposed for this study.
How did selected schools do in their grade 9 ANA Mathematics assessments and how do these results compare to the final grade 9 Mathematics results?
The following sub-questions were formulated to answer the research question:
The writer contacted key persons in the provinces and asked them to provide information on the grade 9 ANA. Information on how to select the schools was given. Most of them were successful in obtaining information from schools in their provinces. Although the sample was not representative, it consisted of both advantaged and disadvantaged schools. The sample may be considered as a convenience sample as the schools readily gave the information selection. The sample consisted of 17 schools, all having Grade 9 learners. Table 1 shows a provincial breakdown of the sample. For ethical reasons, the letters A, B, C, … are used to denote provinces.
|Province||Number of schools|
|Table 1: Provincial breakdown of sample|
Only eight provinces submitted data for this survey.
Research Methodology and Design
In this study, the data was collected from various schools located in different regions or provinces in South Africa. Both quantitative and qualitative data was collected. The quantitative data included the percentage pass rate for ANA grade 9 and the final examination pass rate, the number of grade 9 mathematics teachers and the experience of these teachers. The rest of data was qualitative data and involved issues such as the location of the school, learner backgrounds, resources used by teachers and other factors which may impact on learning.
The data collected was interpreted in the context of the research question, thus locating the study in the interpretative domain. Since various factors (including learner background and the location of the school) impact on the teaching and learning of mathematics, the socio- cultural dimension was also applicable to this study.
Data for this report was collected using questionnaires. These questionnaires sought the following details from the participants.
Location of the schools
Of the 17 schools in the sample, 5 were classified as ex-model C schools, 6 as rural township schools and 6 as urban township schools. These details are shown in table 2, which also includes the ANA pass rate, the final exam pass rate and the difference. The final exam pass rate for mathematics includes the continuous assessment component of 75%.
|A||B||C||D||D - C|
|Table 2: Provincial breakdown, school location, ANA pass rate and final exam pass rate|
|Province||Location of school||ANA pass rate for mathematics||(%) Final exam pass rate for mathematics(%)||Difference|
|A1||Urban (ex-model C)||80,4%||95,1%||14,7%|
|B||Urban (ex-model C)||61,3%||67,8%||7,5%|
|D1||Urban (ex-model C)||56%||99% (56% without cont. assessment)||43%|
|F||Urban (ex-model C)||38,8%||47,8%||9%|
|G||Urban (ex-model C)||54,2%||84,3%||32,1%|
Table 2 clearly shows that ANA performance in the ex-model C schools far exceeded those of rural and urban township schools. Amongst the ex-model C schools, the performance ranged from 38,8% to 80,4% while the remaining schools performed from 0% to 23,4%. However, performance in the final examinations showed a marked improvement amongst all schools. One school from province A went up from 0,3% in the ANA to 97,4% in the final examinations; a similar improvement was shown in a province D school which went up from 0% to 90,4%. Although the schools in province C schools improved, this improvement was not as significant as the other schools in the sample. It is important to note that the final examination pass rate included the continuous assessment component of 75%. One school from province D showed the pass rate with, and without this component. Had this continuous assessment component not been considered, the final pass rate for grade 9 mathematics would have dropped from 99% to 56%, a decrease of 43%. Coincidently, the mark without continuous assessment 56% is exactly the same as the ANA pass mark.
Qualifications,the development of teachers and resources
The number of teachers teaching grade 9 mathematics in these schools ranged from one to three. These teachers were fairly experienced, with years of service ranging from 3 years to 21 years. The teachers all appeared to be qualified with qualifications such as the STD (Secondary teacher’s diploma), ACE (Advanced certificate in education), B.Sc (Bachelor of Science) and the B.Ed (Bachelor of Education). The majority of the rural teachers had the STD qualification while the ex-model C schools’ teachers had degrees. The remaining teachers had a combination of the various qualifications.
Teachers at ex-model C schools attended staff development programmes, funded by the schools, to keep abreast of new developments in the subject. Those in the other schools relied on the education department for workshops and training but these were seldom organised. In some cases, teachers did not have money to pay for transport to attend departmental programmes. Further, these schools did not have funding to send their teachers to privately arranged staff development programmes.
The majority of the teachers used workbooks and textbooks. In addition, the ex-model C schools used teacher developed notes and power-point slides. Teachers at two of the other schools claimed to have used charts and worksheets, in addition to the workbooks and textbooks.
Learner background in mathematics
The ex-model C schools tended to draw learners from selected feeder schools. These schools appear to be very strong in mathematics. There is a close relationship between these schools and their feeder schools, with issues such as high school requirements in mathematics and other subjects being discussed at meetings. Thus, the feeder schools know what to expect from their high schools. There is a smooth transition from the primary school to the high school. The ex-model C high schools also have measures in place to assist grade 8 learners with “gaps” in their mathematical knowledge.
The other schools complained about their learners having a poor background in mathematics and this translated to a poor attitude towards the mathematics.Teaching and learning appeared to be a challenge as learners tended to see everything new and not see the link between sections. Classes were usually very large and learners did not do their homework. When they did their homework, it was usually copied from peers.Learners were also very poor at problem solving and were not familiar with using technology such as calculators and computers.The language barrier also affected their performance.
Other issues from the survey
Schools complained that the ANA was not a high-stakes assessment and as a result, learners did not take it seriously. They also complained about the timing of the paper that is, being written well before the completion of the syllabus. The fact that ANA did not “count” in their final mark and the paper was written before the syllabus was completed, negatively affected their results.
Ex-model C schools explained that the ANA provided good preparation for the final examinations as it subjected grade 9 learners to the discipline of sitting for two hours and writing an external mathematics examination, something they did not do in the past. This in turn provided a solid foundation for mathematics in the FET. One school noted that “ learners improve amazingly in the FET phase when they become used to exams forming the major part of assessment”.
The data from this research provided some valuable and rich data to work with as indicated in the previous section. It is now opportune to discuss these results further.
The ANA results showed a great disparity between the ex-model C schools and other schools in the sample. While there was an element of consistency between the ANA results and the final exam results for ex-Model C schools, this consistency was lacking in the other schools. Two of these schools showed more than a 90% difference when comparing the ANA results with the final exam results. Some of these schools had better final results than the ex-model C schools.
While the actual impact of continuous assessment is not known on the final results, it is clear from the data of one province D school that continuous assessment tended to “inflate” the marks of grade 9 learners. Further, not all provinces set common Grade 9 end of the year papers so it is not possible to comment on the authenticity of the final exam results.
The teaching staff at ex-model C schools had degrees while those at other schools had mainly diplomas and advanced certificates. Further, ex-model C schools had regular staff development programmes while it was lacking for the other schools. These schools depended on the DBE for such programmes but these were seldom organised. These teachers had the added disadvantage of having to travel some distances to attend these programmes, something they could not afford. However, all the schools had the necessary basic resources such as textbooks and work-books for teaching.
Although all schools complained about the quality of learners coming through from the primary schools, the problem appeared to be not serious with ex-model C schools as they tended to have very close and workable relationships with their feeder schools. These schools were able to identify “gaps” in their learners’ mathematical knowledge in grade 8 and they attempt to bridge these “gaps” with additional support. Thus, when they move to grade 9, most of these knowledge “gaps” have been addressed. The other schools did not have this luxury. Their learners tended to have a poor attitude to mathematics, had difficulty in completing their homework and also had a language barrier. They were not familiar with using calculators and computers and this further retarded their progress.
Although there were complaints about the scheduling of ANA (being written in September) and that it did not “count” towards learner’s final marks, there were positive comments about Grade 9 learners sitting for a two hour formal assessment (“exam”). In this regard, the ANA gave learners the opportunity of sitting through a rigorous “exam”, something they had not experienced previously.This experience held them in good stead when they proceeded to the FET where examinations comprise 75% of the final promotion mark.
An analysis of the data and discussion in the previous two sections of this research provides the following findings, which are viewed in the context of the research questions and sub- questions.
From the findings in the previous section of this study, the following recommendations are made:
This study has shown that there are various factors which play a role in learner’s performance in mathematics. Although the 2012 grade 9 ANA average for Mathematics of 13% was a sad indictment on the teaching and learning of Mathematics, the findings and recommendations of this study suggest that there are solutions at hand in addressing this dismal situation.
The Association for Mathematics Education in South Africa (AMESA) (2012).AMESA Grade 9 analysis. Available from amesa.org.za (accessed 23 April 2013)
Cohen, L., Manion, L. and Morrison, K. (2000).Research methods in education. 5th ed. London: RoutledgeFalmer
Crundwell, R. M. (2005). Alternative strategies for large scale student assessment in Canada: Is value-added assessment one possible answer. Canadian Journal of Educational Administration and Policy, 41 1-21. Retrieved October 11, 2007 from http://www.umanitoba.ca/publications/cjeap/pdf%20files/crundwell.pdf
Department of Basic Education (DBE) (2012): Report on the annual national assessments; Grades 1 to 6 & 9. Pretoria: Department of Basic Education.
Earl, L. M. (1999). Assessment and accountability in education: Improvement or surveillance. Education Canada, 39(3), 4-6, 47.
Klinger, D.A., Deluca, C and Miller,T. (2008). The evolving culture of large-scale assessments in Canadian education. Canadian Journal of Educational Administration and Policy (76)
National Assessment Agency (2008). Retrieved April 8, 2008 from: http://www.naa.org.uk/ Organization for Economic Co-operation and Development. (2004). Chile: Reviews of National Policies for Education. Centre for Co-operation with non-members. Paris
Sunday times (9 December 2012): Errors blamed for maths test 'disaster' by Prega Govender