Reviewing assessment principles and practices

The two primary functions of the IB Information System (IBIS) are to deliver our assessments for the Middle Years (MYP), Diploma (DP) and Career-related (CP) Programmes in May and November each year, and to calculate grades for candidates each July and January. Over time, IBIS has been edited to include the additional assessment functions of eMarking and eCoursework uploads and, most recently, the changes necessary for dynamic sampling of internal assessment (IA) materials from schools.

IBIS copes with the demands of each examination session and each additional function or update, but we receive feedback from schools and examiners about the way it works and associated frustrations with the interface, the need for a search function, and issues with passwords.

We know we need to replace IBIS.

There are compelling technical reasons for developing a new Assessment Management System for use by IB World Schools, examiners and IB staff. There are also redundant legacy features in IBIS such as the postage of scripts and IA materials directly to examiners, and the moderation of examiner marking and a number of complex business rules.

Developing a new Assessment Management system gives us the unique opportunity to carry out a deep analysis of our assessment principles and practices and to ask why we do what we do in assessment and why we do it the way we do it.

Our goal is to develop an unrivalled assessment experience – for candidates, for schools, and for examiners.

To achieve our goal, senior staff in the Assessment Division have identified 63 areas of interest that require deep analysis and the opportunity to explore new ways of working. Four Business Analysts are currently working with school coordinators, principal and senior examiners, school surveys and colleagues from across the IB’s Schools Division, Academic Division and Assessment Division to understand the ‘pain points’ of the current system and to investigate the principles behind fundamental assessment practices.

For each of the 63 areas of interest the analysis team identifies a series of problem statements, describes how the system currently works, identifies the underlying principles, investigates any manual processes undertaken, and highlights the “user voice.” For each of the 63 areas, we have also identified the accountable person amongst IB staff, and it falls to them to decide which if any of the proposed new approaches to recommend to adopt.

So far we have carried out detailed analysis on:

• Internal assessment
• School consequences of missed deadlines
• Administrative burden
• Usability and user experience
• Candidate registration
• Mother tongue entitlement
• Examiner ratings
• Examiner training
• Marking strategies
• EURs
• Fees rationalisation

Each area of analysis is presented in a series of workshops involving the most appropriate staff from across the IB, and each workshop concludes with a recommended course of action or possible courses of action requiring a decision by the accountable person.

It is not possible to go into the detail of all of these areas in this brief article, but we can explore the topic of internal assessments as an example.

Challenges with internal assessment for schools revolve around the complexity of the upload process, the amount of additional information that needs to be added to the work, the size and volume of files that need to be uploaded and the amount of time preparing and carrying out the upload. For those moderating the work, it is often difficult to understand why a certain mark has been awarded in the absence of any commentary by the teacher and sometimes there is confusion over whether an examiner marks the work again or looks to see if they can agree with the teacher’s mark.

A new process might include these steps:

1. The candidate uploads their IA work
2. The upload system checks for plagiarism and collusion
3. The teacher marks the IA materials in a system that automatically returns marks to the IB
4. The system selects samples for moderation
5. Examiners moderate the teacher’s marks as appropriate
6. If more samples are required, the system already has them

There will be logistical and cost implications to this proposed process but it reduces the burden on schools, reduces administration on teachers and the coordinator, and it provides all work for sampling.

The investigation into assessment principles and practices will continue until the end of December 2016, and we will keep you updated with progress in future issues of the IB in Practice newsletter.