April 26, 2011, [MD]
Introduction
I have had many opportunities to think about grading and assessment this term. I am currently running a course called Introduction to Computer-Supported Collaborative Learning, and have thought a lot about incentive mechanisms and badges with Monica Resendes, who is co-facilitating the course with me.
I am also preparing to design and teach a course in ICT for development at Scarborough Campus in the fall, and as part of my preparations, I attended a two-day course design institute for graduate students with the University of Toronto Center for Teaching Support & Innovation (CTSI), led by Megan Burnett. This institute was a great experience, giving me a lot of new ideas, and new perspectives on things.
During the same time, I was enrolled in an online class on Computer-Mediated Communications, and I brought many of the ideas from the CTSI institute into the class discussions, including this discussion of grading.
Evaluative grading
According to Megan, when you design a course, you should start with identifying your specific course goals and objectives, which can be content-specific or skill-specific. For example, an undergrad class might aim to both give you the ability to identify the main historical figures from the 20th century, and the skill to use proper MLA citation (as well as many other goals, of course).
Only when you have these goals, do you look at assessment, and ask: what kinds of assessments are appropriate to test if the student has acquired the content knowledge and skills that you hoped. The key point is that you should only test this, not all kinds of other things - ie. if your goal is to teach your students to write proper English, then you should correct the English in their essays, not otherwise.
Coercive grading
However, thinking about the grading structure of many courses I have taken, I realized that there is a very different way of using grades - not to evaluate, but to compel behaviour.
You could argue that all grades are too some extents designed to be “compelling” - humans are not like inert objects that can simply be measured, without it leaving any effect - clearly one of the functions of giving grades is to compel us to work harder (although in some cases, it can have the opposite effect). However, there are situations in which the grading has no evaluative aspect at all, and it functions merely as a carrot and stick to get students to do something - typically something the teacher believes is helpful to their learning.
If, for example, you believe that students who write at least two posts per week learn more than students who don’t, and you allot 15% of the grade to “participation” - ie. writing at least two posts per week, then you are not really “assessing” students ability to write two posts per week, you are simply using the grade coercively to “punish” those who do not do this.
Perhaps there are students who differ from the average, and actually would learn much better by not participating in the forum at all? In a traditional undergraduate class, you can encourage them to come to class, but it is their choice, and if they choose not to, and do excellently on the essay and final exam, you’ve got two choices: admit that your assessment methods do not properly capture the learning you wish to assess, or be very happy about their success.
(Another issue is of course that since there is no pre-test, it is difficult to assess the "value added” of your specific course - perhaps they had this knowledge and skills before they joined your course. But at least your final assessment should make you confident that they have reached a level of knowledge and skill that you desire).
Online courses
****What’s interesting is that this “coercive grading” tends to be much more prevalent in online courses than in offline courses, partly perhaps because it is easier to do (with the wealth of statistics being captured, and made available to the teacher), and partly because the teacher is more afraid of student disengagement.
At the CTSI workshop, they told me that UofT policy strongly discouraged including class attendance as one factor in grading. Yet almost all online courses include “posting every week” as one of their explicit or unspoken factors of assessment.
When I brought this into the discussion in the online course, I received some very interesting feedback, implying that people felt very “defensive” about the term coercive grading, which seemed to have very negative connotations. For me, this concept of evaluative vs “coercive” or “compelling” grading is a really useful thought tool, and does not in itself imply any value judgment.
In thinking about it, the first question should be whether this is a valid distinction, or whether there are other categories that should also be included. Another question altogether is whether, or in which circumstances, coercive grading might support the learning process. However, this is a discussion that we could not even begin to have, without having clarified the terms first.
P2PU
This is also a very interesting discussion when thinking about P2PU, which has no coercive grading. We also have no evaluative grading right now, although we are developing an infrastructure for evaluation and recognition of learning achievements, focused much more on competency-based assessment. This really changes how we can implement different pedagogies, and the amount of leverage that a course organizer has over participants.
Talking to a professor at UofT about his or her course, the discussion is structured very much in terms of power: “Then we’ll have them do this, and then they’ll have to do this”. The students have paid good money for the course, and need a decent grade, so they have no choice but to comply. Whereas at P2PU, it is free to attend, there is not much external value (currently) in completing a course, and often you don’t know the people you are studying with, or the course organizer, so there is less of a social commitment.
The unrealistic ideal might be that students would be focused on their own learning goals (which they would have clearly outlined before the course), and would, with the support of the course organizer and other participants, pursue the learning that would be most useful for them to reach their learning goals, while at the same time supporting the learning community in the group. Of course, we know that this often does not happen, and sometimes we might wish we had the power to “compel” certain behaviour… But I think an environment without that power is a great place to experiment.
Stian Håklev April 26, 2011 Toronto, Canada comments powered by Disqus