We spoke to Chris Rust, Professor Emeritus of Oxford Brookes University and author of ‘Assessment Literacy: The Foundation for Improving Student Learning’ and numerous other publications on assessment and pedagogy. Chris was BILT’s first visiting professor and has facilitated a number of workshops for BILT. He was the keynote speaker in BILT’s launch symposium in June 2017 on Assessment and Feedback.
What are the most common problems you tend to observe with current assessment practices?
I think the most common problem is a lack of alignment, or a fudging of alignment, between the learning outcomes and the task set. And then a further fudging when it comes to the assessment criteria (which may bear little or no connection to the outcomes), the fact that it as all then finally reduced to one virtually meaningless number (mark), and the subsequent opacity of the feedback given. There may be four or five excellent outcomes but then the task chosen to assess them may be an essay, or a report, or exam, or whatever (regardless of whether that will actually assess whether the outcome/s have been met or not) and the assessment criteria then tend to focus on the medium of the task rather than the individual outcomes – structure, fluency, grammar, spelling, referencing, etc. Now while those all may be important, they almost certainly do not explicitly feature in the learning outcomes. And then finally, the worse sin of all, the assessment decisions are aggregated.
What benefits do students experience through a programme level approach to assessment?
Well the programme specifications and subsequent programme level outcomes, should be the vital things the student needs to achieve to merit the qualification. So focussing on them should benefit both the teaching staff and the student. The problem with unitised or modular programmes is that outcomes can be atomised at the lower level to the point that they don’t add up to the espoused programme outcomes, or reach the greater depth and complexity of programme outcomes. A programme level approach should also benefit students by explicitly encouraging the integration of learning from the different units or modules.
How can universities help students to understand these benefits?
By being explicit at all times – in programme and module documentation, when assessment tasks are set and discussed – and also be ensuring that assessment tasks are valid and, wherever possible, authentic.
What are the most valuable resources/articles you use?
I have summarised a lot of the useful research in a freely available paper: ‘What do we know about assessment?’ I would also recommend the Australian website Assessment Futures (found here).
What one piece of advice would you give to help improve students’ assessment literacy?
You must involve students in the activity of assessment – marking work and having to think like assessors – whether it is through marking exercises, giving self and/or peer feedback, or actually allocating actual marks.
You advocate ‘quick and dirty feedback’- what does this mean?
I only advocate this when detailed, individualised feedback may not be logistically possible, or perhaps necessary. In the case of, say, weekly lab reports it is much more useful to take them in and sample them and then send an all class e-mail with generic feedback than for students to receive detailed individualised feedback on a report they did three weeks ago, and since then they have done another two. I would also class on-line possibly multiple-choice quizzes in this category. They may not be able to assess at the higher end of Bloom’s taxonomy (discuss!) but they can give instant feedback to the student on how much they have understood this week’s topic, and depending on the software can also possibly give hints and tips when the answer is wrong.
What inspired you to first start looking at assessment practice and advocating change?
When I did my MEd at Bristol, I had a session from David Satterly and was introduced to his book ‘Assessment in Schools’ which highlights many of the problems in assessment practice which sadly still exist today over 30 years later. And out of all of them, I am especially incensed by the misuse of numbers in assessment, and the fact that university assessment systems get away with doing things a first year statistics student would fail for.
Are there any models you would recommend following to redesign programme assessment?
Yes. I particularly like the idea of requiring programmes to identify cornerstone and capstone modules, which are where the programme outcomes are explicitly assessed. I also think that Brunel’s system of allowing the separation of what they call study blocks from assessment blocks is especially ingenious and clearly allows for all sorts of creativity by the programme team.
Can you think of any case studies from other institutions that would inspire staff to change their programme assessment?
Further to what I said above, I think the Brunel model is certainly worth the effort needed to understand it because of the potential it opens up.
What is your view on 100 point marking scales and would you advocate use of any different forms of marking scales?
If I had my way I would ban the use of numbers in the assessment process completely – they are worse than unhelpful, and I have written on this at length! See for example: Rust, C. (2011) “The unscholarly use of numbers in our assessment practices; what will make us change?” International Journal for the Scholarship of Teaching and Learning, Vol. 5, No. 1, January 2011 (available here). I would advocate much simpler grading – pass/fail, or perhaps pass/merit/distinction, or at most a four-point scale (perhaps based on Biggs’ SOLO taxonomy) – specifically for each learning outcome.
What one film/book/resource would you like to share with the academic community?
In addition to those already mentioned, maybe the video A Private Universe. (available here). It is quite old now but still totally relevant regarding issues of teaching and the failure of many of our assessment practices.
If you could change one thing about HE in the UK what would it be?
Banning the use of numbers in assessment.
Who was your favourite teacher at school/university and why?
That’s hard – I went to a boys’ grammar school – much easier to list the bad teachers, and why. Not sure about favourite but I can only remember two good teachers at school – Mr Allen for English and Mr Thomas for maths – and they were good in that they explained things in easily accessible ways, with humanity and humour, had passion for their subjects and appeared to care about us learning.