2012 ANA results: implications for teaching and learning

A survey of selected schools on the 2012 grade 9 Mathematics ANA results: Implications for teaching and learning:

by

Dr VG Govender (Advisory Committee for Mathematics) (ACM)

 

Introduction and background

Many countries have introduced large-scale educational assessment programmes as part of their public policy. These large-scale assessments are used not only to measure student achievement but also to hold schools accountable for the educational outcomes of students (Crundwell, 2005; Earl, 1999). In Britain and Chile, countries with highly centralised educational curriculum commonly developed at the national level, have large-scale testing programmes designed either to monitor or certify student achievement in relation to national standards (National Assessment Agency, 2008, Organization for Economic Co-operation and Development , 2004; Qualifications and Curriculum Authority, 2008)

The South African curriculum is also highly centralised and all policy matters with regard to curriculum are taken at a National level.The South African Government has identified the improvement of the quality of basic education as the top priority of the country and this has to be delivered by the Department of Basic Education (DBE). In this regard, the Annual National Assessment (ANA) is a critical measure for monitoring progress in learner achievement (DBE, 2012).

ANA, a large-scale national assessment, was introduced by the DBE in South African schools in 2011. This was conducted in Language and Mathematics and involved grades 1-6. In 2012, this was extended to grade 9. The average percentage for Mathematics grade 9 in 2012 was 13% (DBE, 2012:3). This caused a huge outcry in the country.

The Sunday Times in South Africa (9 December 2012) described the grade 9 and other mathematics tests under the heading “Errors blamed for maths test 'disaster'” in which some of the questions in the test were criticised as being “mathematical unsound” . A summary of some of the points of the Sunday Times report is shown below:

• An organisation called FEMSISA stated that grade 9 paper was “mathematically unsound” with mistakes found in 76 out of 140 marks.
• The Department of Basic Education denied that the papers were “mathematically unsound” and saying that the papers were fair and the questions were clear and unambiguous.
• A Human Sciences Research Council official said that South African performance in Grade 9 mathematics is not as bad as made out by the results and stated the need to raise questions about the fairness of the assessment instrument for grade 9 learners.
• A University researcher remarked that the results should be viewed with caution as one was not certain as to the methodology used to obtain these results.
• Another University academic commented that poor teaching of mathematics in preceding grades was the cause of poor performance in grades 6 and 9.

The Association for Mathematics Education in South Africa (AMESA) was asked to given input on the standard of the grade 9 Mathematics paper. The following key points were captured from the AMESA report:

• The questions were formulated clearly and unambiguously. Second language learners would have been able to understand most terms/concepts in the paper as these are standard and thus the language usage in the paper was appropriate.
• There was adequate time allowed for the question paper and that learners had enough time to complete the question paper.
• Using the criteria of content and cognitive level coverage, language and time and AMESA’s analyses of these, it was stated that the paper was set at an appropriate, acceptable standard and must be regarded as fair.
• The poor performance of learners could be associated to the nature of the content (that of being too difficult for learners); psycho-genetic factors (where learners are not yet “ready” for the given mathematical content) and didactical obstacles (the quality of teaching that children receive) (AMESA, 2012)

 

The Sunday Times and AMESA reports focus on different but important issues. Questions need to be asked about what has led to this unacceptable learner performance in a large-scale assessment in a key school subject, Mathematics.

Statement of the problem

In both the Sunday Times and AMESA reports, mention is made of teaching and learning issues although, in the Sunday Times case, it does not grab the headlines. The AMESA report speaks about didactical obstacles such as the quality of teaching which children receive. In this regard, it well known that teaching and learning are the key drivers of assessment. In a large-scale assessment as ANA, although the focus is on the final results, the writer believes that the dismal average of 13% for grade 9 Mathematics ANA is symptomatic of state of mathematics teaching in the senior phase (grades 7 – 9) in South Africa. Over the years, the DBE and other stakeholders have paid more attention to the further education and training (FET) band, especially grade 12, and neglecting other phases. It is possible that this neglect has contributed this poor state of teaching and learning of mathematics in grade 9. This study attempts to provide some insight into this matter with a survey of selected schools using various criteria which are spelt out in the research questions and sub-questions in the next section.

Statement of the problem

In the light of the discussion in the previous two sections, the following research question was proposed for this study.

How did selected schools do in their grade 9 ANA Mathematics assessments and how do these results compare to the final grade 9 Mathematics results?

The following sub-questions were formulated to answer the research question:

• What were the ANA results for Mathematics grade 9 in selected schools?
• What were the final grade 9 Mathematics results in selected schools?
• What are some of factors which may have played a role in the learner performance in grade 9 mathematics?

Sample

The writer contacted key persons in the provinces and asked them to provide information on the grade 9 ANA. Information on how to select the schools was given. Most of them were successful in obtaining information from schools in their provinces. Although the sample was not representative, it consisted of both advantaged and disadvantaged schools. The sample may be considered as a convenience sample as the schools readily gave the information selection. The sample consisted of 17 schools, all having Grade 9 learners. Table 1 shows a provincial breakdown of the sample. For ethical reasons, the letters A, B, C, … are used to denote provinces.

 

Only eight provinces submitted data for this survey.

 

Research Methodology and Design

In this study, the data was collected from various schools located in different regions or provinces in South Africa. Both quantitative and qualitative data was collected. The quantitative data included the percentage pass rate for ANA grade 9 and the final examination pass rate, the number of grade 9 mathematics teachers and the experience of these teachers. The rest of data was qualitative data and involved issues such as the location of the school, learner backgrounds, resources used by teachers and other factors which may impact on learning.

The data collected was interpreted in the context of the research question, thus locating the study in the interpretative domain. Since various factors (including learner background and the location of the school) impact on the teaching and learning of mathematics, the socio- cultural dimension was also applicable to this study.

 

Data Collection

Data for this report was collected using questionnaires. These questionnaires sought the following details from the participants.

• Location of the school
• The ANA pass rate for grade 9 mathematics
• The final mathematics pass rate
• The number of teachers teaching grade 9 mathematics
• The qualifications of the teachers
• Resources used by the teachers
• The background of the learners
• Any other suitable comment

 

Results

Location of the schools

Of the 17 schools in the sample, 5 were classified as ex-model C schools, 6 as rural township schools and 6 as urban township schools. These details are shown in table 2, which also includes the ANA pass rate, the final exam pass rate and the difference. The final exam pass rate for mathematics includes the continuous assessment component of 75%.

Table 2 clearly shows that ANA performance in the ex-model C schools far exceeded those of rural and urban township schools. Amongst the ex-model C schools, the performance ranged from 38,8% to 80,4% while the remaining schools performed from 0% to 23,4%. However, performance in the final examinations showed a marked improvement amongst all schools. One school from province A went up from 0,3% in the ANA to 97,4% in the final examinations; a similar improvement was shown in a province D school which went up from 0% to 90,4%. Although the schools in province C schools improved, this improvement was not as significant as the other schools in the sample. It is important to note that the final examination pass rate included the continuous assessment component of 75%. One school from province D showed the pass rate with, and without this component. Had this continuous assessment component not been considered, the final pass rate for grade 9 mathematics would have dropped from 99% to 56%, a decrease of 43%. Coincidently, the mark without continuous assessment 56% is exactly the same as the ANA pass mark.

 

Qualifications,the development of teachers and resources

The number of teachers teaching grade 9 mathematics in these schools ranged from one to three. These teachers were fairly experienced, with years of service ranging from 3 years to 21 years. The teachers all appeared to be qualified with qualifications such as the STD (Secondary teacher’s diploma), ACE (Advanced certificate in education), B.Sc (Bachelor of Science) and the B.Ed (Bachelor of Education). The majority of the rural teachers had the STD qualification while the ex-model C schools’ teachers had degrees. The remaining teachers had a combination of the various qualifications.

Teachers at ex-model C schools attended staff development programmes, funded by the schools, to keep abreast of new developments in the subject. Those in the other schools relied on the education department for workshops and training but these were seldom organised. In some cases, teachers did not have money to pay for transport to attend departmental programmes. Further, these schools did not have funding to send their teachers to privately arranged staff development programmes.

The majority of the teachers used workbooks and textbooks. In addition, the ex-model C schools used teacher developed notes and power-point slides. Teachers at two of the other schools claimed to have used charts and worksheets, in addition to the workbooks and textbooks.

 

Learner background in mathematics

The ex-model C schools tended to draw learners from selected feeder schools. These schools appear to be very strong in mathematics. There is a close relationship between these schools and their feeder schools, with issues such as high school requirements in mathematics and other subjects being discussed at meetings. Thus, the feeder schools know what to expect from their high schools. There is a smooth transition from the primary school to the high school. The ex-model C high schools also have measures in place to assist grade 8 learners with “gaps” in their mathematical knowledge.

The other schools complained about their learners having a poor background in mathematics and this translated to a poor attitude towards the mathematics.Teaching and learning appeared to be a challenge as learners tended to see everything new and not see the link between sections. Classes were usually very large and learners did not do their homework. When they did their homework, it was usually copied from peers.Learners were also very poor at problem solving and were not familiar with using technology such as calculators and computers.The language barrier also affected their performance.

 

Other issues from the survey

Schools complained that the ANA was not a high-stakes assessment and as a result, learners did not take it seriously. They also complained about the timing of the paper that is, being written well before the completion of the syllabus. The fact that ANA did not “count” in their final mark and the paper was written before the syllabus was completed, negatively affected their results.

Ex-model C schools explained that the ANA provided good preparation for the final examinations as it subjected grade 9 learners to the discipline of sitting for two hours and writing an external mathematics examination, something they did not do in the past. This in turn provided a solid foundation for mathematics in the FET. One school noted that “ learners improve amazingly in the FET phase when they become used to exams forming the major part of assessment”.

 

Discussion

The data from this research provided some valuable and rich data to work with as indicated in the previous section. It is now opportune to discuss these results further.

The ANA results showed a great disparity between the ex-model C schools and other schools in the sample. While there was an element of consistency between the ANA results and the final exam results for ex-Model C schools, this consistency was lacking in the other schools. Two of these schools showed more than a 90% difference when comparing the ANA results with the final exam results. Some of these schools had better final results than the ex-model C schools.

While the actual impact of continuous assessment is not known on the final results, it is clear from the data of one province D school that continuous assessment tended to “inflate” the marks of grade 9 learners. Further, not all provinces set common Grade 9 end of the year papers so it is not possible to comment on the authenticity of the final exam results.

The teaching staff at ex-model C schools had degrees while those at other schools had mainly diplomas and advanced certificates. Further, ex-model C schools had regular staff development programmes while it was lacking for the other schools. These schools depended on the DBE for such programmes but these were seldom organised. These teachers had the added disadvantage of having to travel some distances to attend these programmes, something they could not afford. However, all the schools had the necessary basic resources such as textbooks and work-books for teaching.

Although all schools complained about the quality of learners coming through from the primary schools, the problem appeared to be not serious with ex-model C schools as they tended to have very close and workable relationships with their feeder schools. These schools were able to identify “gaps” in their learners’ mathematical knowledge in grade 8 and they attempt to bridge these “gaps” with additional support. Thus, when they move to grade 9, most of these knowledge “gaps” have been addressed. The other schools did not have this luxury. Their learners tended to have a poor attitude to mathematics, had difficulty in completing their homework and also had a language barrier. They were not familiar with using calculators and computers and this further retarded their progress.

Although there were complaints about the scheduling of ANA (being written in September) and that it did not “count” towards learner’s final marks, there were positive comments about Grade 9 learners sitting for a two hour formal assessment (“exam”). In this regard, the ANA gave learners the opportunity of sitting through a rigorous “exam”, something they had not experienced previously.This experience held them in good stead when they proceeded to the FET where examinations comprise 75% of the final promotion mark.

 

Findings

An analysis of the data and discussion in the previous two sections of this research provides the following findings, which are viewed in the context of the research questions and sub- questions.

• The results in ANA grade 9 mathematics, was mostly poor in all schools. Only one school (an ex-model C school) had in excess of 80% pass rate. Overall C schools results in ANA surpassed all the other schools. Seven schools in the sample had a less than 5% pass rate in the ANA.the ex-model
• The final results in all schools, was far better than the results in ANA with two schools increasing in excess of 90%. The improved final results could be attributed to the impact of continuous assessment (75%) on the final promotion mark.
• There was some consistency in the ANA results and the final results for ex-model C schools. This was lacking in other schools. With the exception of province C , all other rural and urban township schools showed double digit increases in their final exam results, when compared to ANA.
• One of the key factors in the performance of learners was the qualifications of teachers. Teachers at the ex-model C schools had degrees while those at the other schools had mostly diplomas and/or advanced certificates. It is clearly evident that those with diplomas and advanced certificates were in need of additional support from the DBE. Unfortunately, this support was not usually forthcoming and teachers had to fend for themselves. At the same time teachers at ex-model C schools usually attended workshops and support programmes which were funded by the schools. It is prudent to remark here that the teachers at ex-model C schools are doubly advantaged. Firstly, they are well qualified and secondly they are funded with various workshops and support programmes. Those at the other schools are doubly disadvantaged. They are poorly qualified and are not given the necessary support. The scenario spelt out here can have a major impact on learner performance.
• Another important factor is the allocation of resources. All schools had the necessary textbooks and workbooks. It is likely that teachers from the urban and rural township schools were not able to use the workbooks and textbooks judiciously. In this regard teachers probably followed the textbook word for word without checking for relevance and adapting to their learners’ own needs. The fact that there was little or no effort to supplement these resources in class probably caused a lack of interest in their learners. The ex-model C schools supplemented these resources by using teacher developed notes and power-point slides of lessons. These additional features in their lesson delivery probably enhanced the lesson quality and contributed to more learners understanding their work. Learners have different learning styles and teachers should try to incorporate these styles in their lessons.
• The same ANA paper was administered to all learners in the country. However, learners have different experiences at their schools. At ex-model C schools, the gaps in learners mathematical knowledge in grade 8 was addressed through a series of support programmes to help bridge the gap. This was not so in the other schools. Here, there were usually large classes and children learned their work in isolation without any regard for how lessons are linked. This is probably due to a poor primary school background in mathematics. These learners tended not to do their homework, thus depriving themselves of much needed practice. These learners were further compromised by not being able to use basic technology such as calculators and computers. The scheduling of the ANA, in September, was a cause for concern. At that time, teachers are just completing their third terms work and have not yet done the fourth terms work. Teachers are placed in a dilemma as to whether to focus on ANA (which does not count towards promotion) or to attempt to complete the syllabus. In this regard, teachers would rather focus on completing their work in time for the final examination and pay lesson attention to ANA.
• A positive spin-off from ANA is that learners get to experience what it is to write an externally set, intensive mathematics paper for two hours. This experience teaches time management and pacing skills which hold them in good stead in grades 10 -12, where examinations account for 75% of the final promotion mark.

 

Recommendations

From the findings in the previous section of this study, the following recommendations are made:

• The poor performance in grade 9 ANA (Mathematics) suggests that there are challenges with teaching and learning in Senior Phase mathematics. This is a clear indicator that the DBE’s and the country’s obsession with grade 12 results is unfounded. Rather, there should be a concerted effort to improve teaching and learning in all grades and all underperforming schools should be given additional support. In this regard, support for early grades is likely to have a positive impact on learner performance in these and later grades.
• There should be efforts to improve the qualifications of mathematics teachers. This study has shown that a STD qualification is no longer sufficient for mathematics teachers in the senior phase. The department should ensure that these teachers are upgraded to a more appropriate qualification such as the B.ED.
• Teachers at all levels should make lesson planning and preparation a key part of their teaching strategy. It is all well to follow the material in a text-book but to do so without adapting the content to suit one’s learners is likely to make the mathematics “inaccessible” to learners. In this regard, teacher developed notes would be useful. In the current training of pre-service teachers, lesson plans and preparation for their method subjects is a mandatory part of their curriculum. This practice should be encouraged for fully-fledged teachers.
• The ANA for grade 9 is the same for all learners, irrespective of location of school and cultural background. These socio-cultural influences tend to affect learner performance as seen in this study. The DBE should attempt to level the playing fields for all learners by ensuring that disadvantaged schools are prioritised for support such as such as teacher workshops, additional resources and more exposure to the language of teaching and learning (English).
• There should be an attempt by the DBE to make sure that the ANA does count toward the promotion of learners. This could be done in a similar way to the trial examinations for grade 12 learners.
• The experience in writing a two-hour paper in an examination type setting has the potential to assist in learner development. Learners learn how to prepare for an examination-type assessment task and to pace themselves during the “examination”. This exposure in grade 9 could have a positive influence on the way they approach their examinations in the FET. One school in this study reported an “amazing” improvement in their learners approach to grade 10 mathematics, attributing this improvement to their grade 9 ANA experience.

 

Conclusion

This study has shown that there are various factors which play a role in learner’s performance in mathematics. Although the 2012 grade 9 ANA average for Mathematics of 13% was a sad indictment on the teaching and learning of Mathematics, the findings and recommendations of this study suggest that there are solutions at hand in addressing this dismal situation.

 

References

The Association for Mathematics Education in South Africa (AMESA) (2012).AMESA Grade 9 analysis. Available from amesa.org.za (accessed 23 April 2013)

Cohen, L., Manion, L. and Morrison, K. (2000).Research methods in education. 5th ed. London: RoutledgeFalmer

Crundwell, R. M. (2005). Alternative strategies for large scale student assessment in Canada: Is value-added assessment one possible answer. Canadian Journal of Educational Administration and Policy, 41 1-21. Retrieved October 11, 2007 from http://www.umanitoba.ca/publications/cjeap/pdf%20files/crundwell.pdf

Department of Basic Education (DBE) (2012): Report on the annual national assessments; Grades 1 to 6 & 9. Pretoria: Department of Basic Education.

Earl, L. M. (1999). Assessment and accountability in education: Improvement or surveillance. Education Canada, 39(3), 4-6, 47.

Klinger, D.A., Deluca, C and Miller,T. (2008). The evolving culture of large-scale assessments in Canadian education. Canadian Journal of Educational Administration and Policy (76)

National Assessment Agency (2008). Retrieved April 8, 2008 from: http://www.naa.org.uk/ Organization for Economic Co-operation and Development. (2004). Chile: Reviews of National Policies for Education. Centre for Co-operation with non-members. Paris

Sunday times (9 December 2012): Errors blamed for maths test 'disaster' by Prega Govender