Top Nav Breadcrumb

Making the change from MYP moderation to on-screen examination

By Gareth Jones.

Gareth JonesWhat changes when a school moves from moderation to on-screen examinations and what becomes better/easier for that school?

Six months ago, I sat with a group of international educators from across Europe in a school café at the International School of Milan. Our students were presenting finely polished presentations that had been drafted, proofread, re-drafted, practised, self- and peer-evaluated. They demonstrated the finest elements of self and peer formative assessment. The 10th annual Global Issues Network conference showcased the international minded, active and compassionate learners of international schools with both IB and non-IB backgrounds. However, the conversation in the café was about a different form of assessment.

The educators around the table discussed the IB Middle Years Programme (MYP) on-screen examinations. The discussion was based around the need for such an assessment, questions were asked and, from behind the coffee cups, the IB’s decision was being scrutinised. “It’s too traditional and starts to prescribe content?” and “One of the reasons we do the MYP is that it is not driven by assessment” were arguments that were being stated. I too had previously doubted the intentions of an externally set examination. An area of the MYP that I had fallen in love with was the freedom to plan valid assessment that was relevant to today’s world. Examination room 2I had written about the limitations of standardised and national tests and their role in both testing for national citizens and ranking populations for national economies. However, these did not fit as issues that would influence the IB, with its values ingrained in international mindedness and understanding.

When deciding to apply to be part of the examining process I considered my own reservations, I too had grappled with some of the arguments stated over coffee in Milan. I understood the tensions that the IB has always been under to ensure it meets the requirements of universities, and that the IB Diploma Programme (DP) has continued to grow whilst the MYP has found growth more challenging. It was not these areas that influenced me in my decision to become involved in the examining process, but the issues of reliability of the moderation process did.

It is true that the validity of the assessment produced under the old system of moderation is strong. The new assessment can be tailored to individuals settings, engrained in context and be matched to the students development and needs, it allows for a holistic assessment of the student, assessing areas such as knowledge, investigation, communication and application. This can be done in many settings and by many schools, but it will never be reliable between all schools doing the programme. A student assessed in Lusaka will not have the same experience as a student assessed in London. A student’s level of achievement in Barcelona will not necessarily be reflective of an equally deserving student in Bangkok. The moderation process helps to limit these differences, but overall the system is flawed, due to both the sheer size of the population being moderated and the differences between the assessment being carried out.

Under the on-screen examinations process, the practice of holistic assessment will still happen in MYP schools.

Educators will still have the freedom to create and develop assessment which meets the needs of today’s world. What the on-screen examinations now provides is the additional opportunity for MYP schools and their students to demonstrate these skills in a reliable way. This culminating assessment provides an opportunity for students to apply the skills learnt through the MYP objectives. Unlike traditional examinations based on just knowledge and memorisation, students will need to demonstrate investigative, analytical, reflective and communicative skills. Perhaps most importantly in the interdisciplinary on-screen examinations, they will synthesize their knowledge and skills from different subject areas to produce new understanding and perceptive solutions to problems grounded in a global context. In this year’s pilot on-screen examinations, students grappled with the challenging issue of Universal Primary Education, an issue that has so far managed to get the better of some of our best and brightest at the United Nations.

… on-screen examinations challenged students to synthesize information from a range of disciplines and from feedback they enjoyed the challenge.

Synthesis is a skill which the lifelong learners who we hope will affect change will need to master to be successful in the world.

When writing the on-screen examinations the team starts by producing unit planners around the global context for the session. Concepts are chosen, statements of inquiry are produced and scrutinised. The team looks at what areas of the approaches to learning framework would be useful for the unit and the unit is built. The examination is designed so that it addresses the MYP objectives and is grounded in both concepts and context. The examination forms a valid method of assessment. As such, the examination will not drive schools to teach a content-heavy course, but will enable schools to engage with the conceptual framework and push the boundaries in all criteria to allow students to be prepared for the on-screen examinations.

Exam close up 3

Not only is the examination a valid form of assessment grounded in MYP philosophy, the results are more reliable. This is due to the examining process being far more rigorous than the moderating process that preceded it. Furthermore as the questions in the examination are the same for each student it does not have the same variability as moderated tasks. The first element of the examining process involves the senior examiners finding a common standard with the chief examiner at standardisation. Once this is achieved practice, qualification and seed scripts are created for each group of questions. Examiners need to practice, qualify and then one in every ten questions is a seed that the examiner needs to mark in tolerance. Examiners who cannot mark to the tolerance of the exam do not qualify. If at any point after qualification the examiner is not in tolerance feedback is given and if this is not adhered to the examiner is stopped from marking.

This thorough method of marking is quick and efficient with the on-screen setup and digital marking software (RM assessor). It allows for a high level of reliability in the marking and enables schools to feel confident in the marking process.

Gareth Jones on location at a school field trip 11 September 2015

Gareth Jones on location at a school field trip 11 September 2015

To establish student grades a grade award meeting of senior examiners occurs. At grade award whole scripts are viewed, descriptors are consulted, statistically recommended boundaries are suggested and a common grade boundaries using both qualitative and quantitative information are agreed. This allows for a true reflection of MYP achievement to be seen, the examination process was written with objectives in mind and then grade award returns to this point.

Although less important than the continued validity and increased reliability benefits to the student, practically the examination will also be better for schools and the environment. Schools will no longer need to produce moderation folders for the on-screen examined subjects, results will be quicker for student certification and the on-screen setup means less paper and post.

Overall, it can be a hard decision to change from a known to an unknown, but nothing ever improved by standing still. Just as the students at the Global Issues Network conference formatively assessed their own work to monitor and modify it to the polished final product the IB still reflects on its own practice and modifies for the better, the on-screen examinations are a product of this. This reflective practice is important in the classroom and the context driven on-screen examinations will reward this, just as it will reward the students of schools who use the global context and teach conceptually. To reinforce this and to answer a few of the questions from around the coffee table in Milan, I will leave you with the words of an educator from a school involved in the pilot “If teachers do begin to “teach to the test” then at least they are teaching to a multi-modal test that demonstrates good pedagogy. This is a good example of how a test can break the mould of traditional testing structures”. I am sure that the authors of the on-screen examinations will strive to continue to break this mould and develop further as the MYP continues to break new ground.

Gareth Jones is an examiner for MYP interdisciplinary, Halcyon London International School, UK.

Watch a video showing student and teacher responses to the MYP eAssessment pilot 2015.