An Action Research Project by Tom Nadin (Science)
The objective of this project is to research and implement e-learning strategies as a means of improving the progress and outcomes of key marginal students in GCSE (Core) Science.
As a school, we operate a key stage four model in which historically, the majority of students complete the GCSE Science course in year ten and GCSE Additional Science in year eleven. We also have a group of students completing the Separate Sciences course. Using this model, the majority of students are entered for all GCSE units and accreditation at the end of year ten. The exception to this is students who we do not feel to be ready to sit their exams at this point. Each year we also have a number of students who sit their GCSE examinations at the end of year ten but who are re-entered for these exams in year eleven. Almost always this is either at the school’s request because a student’s overall GCSE Science grade is below that expected, or because students or parents request a re-sit in order to improve their overall grade, in order to access college courses.
In the academic year 2014-15 the school entered 122 students for their GCSE Science exams at the end of year ten. Having consulted with teachers, parents and teachers, the decision was made to withdraw 18 students from these exams with a view to entering them at the end of year eleven. Although overall, the GCSE Science results for this cohort were pleasing the further decision was made to re-enter 29 students to re-sit their GCSE Science exams at the end of year eleven.
This presented the Science faculty with a particular set of challenges. 47 students were due to sit their GCSE Science exams alongside their GCSE Additional Science exams in the summer of 2016. This would place significant extra demands on these students, in terms of the number of science exams they would sit and especially in terms of the bulk of knowledge they would need to retain in order to achieve successful grades. By definition this group of students overwhelmingly consisted of students who either had not achieved, or were not on track to achieve their targeted grade at the end of year ten. This was for a variety of reasons but often underpinned by a failure to successfully access the curriculum and support offered to them in year ten, or by difficulties in retaining and applying the bulk of knowledge required. In other words, this group consisted largely of the very students least likely to be able to successfully overcome the challenges facing them.
As a faculty we were not in a position financially or logistically to offer significant additional in class support to these students so needed to think creatively about out of class solutions which would help these students to access and retain the knowledge required to succeed in their exams. Online E-learning packages such as SAM learning (to which the school subscribes), seemed to offer an avenue through which support could be provided and monitored effectively.
SAM learning is an online package consisting of student activities and online tests. Many of these are self-marking and can provide immediate feedback to both students and teachers. Activities can be set by teachers but can also be accessed independently by students. The school has subscribed to SAM learning for several years prior to the period of this investigation, and although it has been used by staff and students, for example for homework tasks and independent revision, it was yet to be used systematically by the Science faculty.
The functionality and accessibility of SAM learning seemed to provide a means through which interventions could be put in place for our targeted group of Y11 students. In addition to this, I was aware anecdotally of examples of SAM learning being used effectively to support revision and exam preparation at KS4, both in other departments in our school and in other local institutions. Although there appeared to have been little research done specifically in to the use of SAM learning as an intervention tool, there did seem to be robust evidence to support the view that consistent and regular use of SAM learning could lead to an improvement in student outcomes overall.
One of the most comprehensive studies into this area was conducted by the Fisher Family Trust (FFT). They investigated the effect of SAM learning on the progress and outcomes of 258 599 UK students between 2009 and 2011. The findings of this study seemed to suggest a strong link between the use of SAM learning and an improvement in student outcomes. For example the study found that on average 10 plus hours use of SAM learning led to students achieving 12.3 capped points scores higher than expected and that, although less significant, as little as between 2 and 10 hours study on SAM learning could lead to a measurable improvement in student outcomes as shown in figure one.
Figure 1. The actual and estimated attainment of students, with regards to their usage of E-learning.
This study also suggested that the group of students whose outcomes were improved most significantly by the use of SAM learning were those with the lowest achievement at KS2. This is summarised in figure 2. This was particularly interesting as many of the students in our intervention group were relatively low achievers at KS2.
Figure 2. Value added performance with usage of E-learning in relation to prior attainment.
Most interestingly the results of the FFT study seemed to indicate that as little as two ten minute sessions per week could have a measureable impact on outcomes in Science and that more than ten hours spent on SAM learning would on average, improve outcomes by a third of a grade. These findings are summarised in figure 3.
Figure 3. Value added performance in core subjects with usage of E-learning.
Other research appeared to support the view that SAM learning could act as a valuable intervention tool, especially for students with a back ground of lower or under-achievement. For example an American study (Jorgensen, 2010), stressed the potential effectiveness of SAM learning in providing accessible interactive and scaffolded learning. She also stressed the program’s potentially positive impact on disengaged learners in academic and content-rich subjects such as Science.
The research seemed to suggest that SAM learning was worth exploring for use as an intervention tool with our targeted group of students in Y11.
Having made the decision to use SAM Learning as an intervention tool, it was essential to devise a programme through which it could be effectively introduced and delivered to students and monitored by members of staff. I also felt that the interventions were likely to be most effective if they integrated a range of resources and activities, drawing on existing best practice, rather than using SAM learning as a stand-alone intervention. As such, the faculty and I decided on the following actions;
- Each student in the intervention group would be allocated a member of teaching staff as a mentor. This member of staff would ensure that the student had access to appropriate resources and would monitor and support their use. They would also be the first point of contact with home.
- Each student would be provided with a paper revision guide and GCSE Science workbook. The staff mentor would discuss this with students and monitor their use.
- Each student was provided with a content specific, personalised learning checklist (PLC). This had been modified so that once students had identified specific areas of need, they could reference the appropriate activities both in the revision guide/workbook and in SAM learning. Figure 4 provides an example of such a PLC
Figure 4. PLC, showing specific links to revision guide and SAM learning activities
These PLCs were designed to make students’ use of SAM learning (and other revision resources), more effective by allowing them to target their efforts on the areas of greatest need.
- It proved relatively easy to set up a group in SAM learning which contained all of the students in the intervention group. This enabled me to set the relevant tasks for the students concerned. I made the decision to set all of the Core Science tasks up front, giving students the opportunity to complete them at their own pace. I felt that this would allow them to use their PLCs to identify and then work on key areas of the subject content.
- The interventions were tracked at the faculty level though mentors regularly updating a central spreadsheet indicating when actions had taken place. Figure 5 shows an excerpt from this tracking grid.
Figure 5. An excerpt from the faculty interventions tracking grid.
- Mentors regularly met with the intervention students in order to discuss their exam preparation and their use of resources. Having set the SAM learning tasks, I was able to monitor their usage online.
Initial analysis of the GCSE Science headline figures in 2016 suggested that the results were pleasing. Overall, students had exceeded national expectations in terms of outcomes and progress. This is summarised in figure 6.
Figure 6. Summary of achievement in GCSE Science 2016.
These results also suggested a modest, but significant improvement in results from those achieved and/or predicted by/for these students in the summer of 2015. Taking into account the predicted grades of the students who were not entered for GCSE Science in Y10, the percentage of students achieving A* to C grades had increased from 52% to 57% and the percentage of students making at least three levels of progress has increased from 51% to 57%. Both of these gains were particularly significant as they pushed faculty outcomes above national expectations.
It was clear that students had made limited, yet significant gains. It also appeared that the interventions we had put in place had had some impact. Of the 47 students in the intervention group, 16 (34%) had improved by one or more grade from year ten to year eleven in GCSE Science. Interestingly, of the 18 students that we did not enter for GCSE examination in Y10, only 4 (22%), improved their grade, whereas 12 of the 29 students (41%), who did take GCSE Science in Y10, but re-sat in Y11, improved their grade.
It also appeared that the use of SAM learning had had some impact on student outcomes, both for the targeted intervention group of students and perhaps unexpectedly, across the whole year group. SAM learning usage reports in June 2016 suggested that use of SAM learning across the year group in Science had increased by over 200% on the previous year, and that students in our school were on average marking greater use of SAM learning than the national average. Although it is impossible to link this usage with outcomes across the faculty, the research suggests that it is likely that it did have a positive impact. It seems likely that the raised profile of SAM learning and the distribution of resources such as the amended PLCs to students outside the target group led to increased use of SAM learning across the year group.
Specific analysis of the outcomes of students in the targeted intervention group also proved to be very interesting, especially when compared to usage of SAM learning. This is summarised in Figure 7.
* Based on national transition matrices.
Figure 7. SAM Learning usage and change in GCSE Science grade from year ten to year eleven.
Indicates where a student did not sit GCSE Science in Y10. The grade shown here is the teacher assessed grade.
Although there does not appear to be a direct correlation between use of SAM learning and an improvement in outcomes (and detailed statistical analysis would be needed to show this), there do appear to be some patterns.
- 14 of the 16 students who had improved their grades had spent at least some time on SAM learning.
- Of the 24 students who had spent at least 30 minutes on SAM learning 12 (50%), had made an improvement in their grade.
- Of the 23 students who had spent less than half an hour, or no time on SAM learning 4 (17%), had made an improvement in their grade.
- 2 of the 3 students with the highest usage of SAM learning made no improvement to their grade.
- It appears that girls were more likely than boys to use SAM learning, with girls accessing SAM learning for an average of 2.40 hours and boys for an average of 1.45 hours. Twelve boys did not access SAM learning at all compared to 7 girls who did not. Interestingly this difference seemed to correspond with a slight but not significant difference in improvement of outcomes with 7 out of 24 boys (29%) and 9 out of 23 (39%) girls improving their grades.
It does appear that the interventions used with this cohort of students had a limited, yet significant (in terms of improved outcomes for the faculty), effect on students’ grades between year ten and year eleven. This view is supported by analysis showing that 34% of students in the intervention group improved by at least one grade. This improvement was more marked in girls (39%), than it was in boys (29%). It was also notable that more students improved their grade having first taken the exams in year ten (39%), then re-sat in year eleven, than those who were withdrawn in year ten and sat for the first time in year eleven (22%).
It is impossible to demonstrate a causal relationship between these improvements in grades and the use of SAM learning. No attempt was made to control other variables which may have had an impact on student outcomes. All students had access to a range of intervention resources, for example revision guides and work books as well as SAM learning. In many cases students who had high usage of SAM learning also regularly accessed and used other resources. Indeed, those students who were most willing to access and use SAM learning were usually those who were best motivated in general and most willing to seek support from their mentors and indeed to access other resources. However, there does seem to be a tentative relationship between use of SAM learning and an improvement in grades. Fourteen of the sixteen students who made an improvement in their grades had spent some time working on SAM learning with 50% of students who accessed SAM learning for more than half an hour seeing an improvement in their grades. A gender difference in SAM learning was also apparent, with girls being much more likely to use SAM learning and likely to spend longer using it in total, than boys.
It seems at least possible, although it is by no means proved by this piece of research, that the use of SAM learning has had a positive impact on outcomes for students retaking GCSE Science in our school. However, further more detailed research, with an attempt to control other potentially contributing variables, would be needed to demonstrate a positive correlation, let alone a causal relationship.
This piece of action research has provided me with the opportunity to learn some valuable lessons in to how to provide effective interventions. I will be able to apply this learning and improve the package we offer to the current year eleven. It has also raised some interesting questions that could form the basis for further research.
Learning, to be applied to the current year eleven:
- SAM learning is a valuable intervention tool. It allows students to access and assess learning independently. We will be using it again with the current year eleven
- SAM learning is especially effective when combined with a self-diagnosis system, such as the PLCs as this allows students to direct their effort to the areas where it is most needed. Again, we will make sure that these are available for use with the current year eleven.
- This research supports the view that SAM learning has a positive impact on student outcomes, although it by no means proves it. We will make sure that current students are aware of this and of the potential gains to be made by the regular use of SAM learning.
- Girls appear to be more likely to access SAM learning than boys. In the current Y11, boys will need more monitoring and support than girls.
- Out mentoring was not always effective in leading students (especially boys), to consistently use SAM learning. We will need to consider more effective ways in which mentoring and support can be offered.
Possible areas for further research:
- Is there a causal relationship between the use of SAM learning and improved outcomes in GCSE Science, especially for previously underachieving students?
- What is the relative effectiveness of SAM learning as an intervention tool compared with more traditional paper based resources, for example revision guides and work books?
- Are girls more likely to access SAM learning than boys? Why might this be?
References and further reading
Fisher Family Trust (2012), Impact of E.Learning.
Jorgensen M. (2010), An intervention that works – SAM Learning.