Assessment Differently

Seth and I are running a teaching and learning workshop on Tuesday, which has the following description:

A case study on how self and peer assessment has been used. This will form the basis for a discussion about the problems and pitfalls of self assessment and how it can make your assessment load lighter. Will also include some information about blogs.

What follows are dot points for our session (aka quick and dirty media) and all that follows is part of an implicit process based learning model that Seth and I use. While this isn’t part of today’s topic, I implicitly model most of my assessment on the assumption that understanding and assessing how you get to where you got to is usually more useful for students than the thing left at the end (the object). I also have a simple rule of thumb. If any assessment task significantly increases my work load then it isn’t going to happen. This is not only because I’ve already got more than enough to do, but that it is easy to confuse more teacher time assessing with student centred learning, but this ain’t necessarily so.

SELF ASSESSMENT

Why

  • Have the view that we confuse student centred learning and good teaching with micro-assessment
  • It lessens the work load of the academic
  • In many cases students are the better judge of their efforts and achievements than staff
  • Student responsibility for assessment increases their investment and participation in learning
  • The ability to critically reflect and judge your own work is a key learning outcome in its own right
  • Self assessment helps to make concrete for the student what is being assessed, how, and why
  • Being able to assess your own (and others) practice and work is a basic professional skill – we aim to develop this during the degree rather than afterwards
  • Self (and peer) assessment encourages students to take responsibility for their own learning and this responsibility is put into practice by literally providing them with the responsibility for this mark
  • This also helps to make visible to the student what changes have happened for them through the course

How

  • the use of self assessment is graduated across year levels (this is not yet consistent in the media program)
  • in first year it is applied to participation only and is worth a minimum of 25% of a course
  • a participation protocol is used which is largely student defined, this produces a participation ‘diary’ which is evaluated weekly and forms the basis of an overall mark the last class all students are required to assess their participation against this protocol and to report to the class on what they did well, what they have learnt to do better, and what they could have done better. They then give themselves an overall mark for participation
  • in second year self assessment can also be extended from participation to include submitted work, however we maintain participation as a self assessed activity
  • what is crucial in this process is that the protocol must be explicit, and in our experience it has worked best when students have a voice in these protocols – even to the extent of helping define what assessment criteria ought to be used. Time must be spent in the curriculum to develop these – this is the content of the class or classes!
  • Students have control over these grades,they are not moderated by the teacher, though we do make them subject to peer questions

Outcomes

  • students are more articulate about their own learning
  • students become more aware of their achievements (or not) during a course and are then better able to judge their experience of the course
  • they develop reflective skills that can be applied to other tasks (not just participation)
  • it encourages the development of critique skills (which we also develop separately in class through exercises)
  • it encourages a focus on how they are learning and what actually makes a difference to them as learners, these are transferable across subjects and also into their careers
  • it has been universally very successful (with minor exceptions)
  • it is able to acknowledge different learning/cognitive styles and validate different sorts and types of learning
  • it can be used to provide a road map for students to identify their own strengths and weaknesses and to then work out which they want to work on

PEER ASSESSMENT

Why

Peer assessment is done on the basis of students having had experience with self assessment. We believe it is only after they have learnt how to evaluate themselves responsibility and with clear criteria, that they are able to then assess others. Peer assessment for us might also include peer critique. Peer assessment is done for many of the same reasons as self assessment:

  • students who work in groups usually are the best judges of who was done what and why (not the teacher)
  • (a consequence of this is that teachers end up just marking the finished product, not how it came to be – students have more insight into the process than the teachers)
  • it frees up assessment from the outcome of the collaboration towards the collaborative process itself
  • increasingly our graduates professional experience will require and assume collaborative skills or roles
  • as part of our process based approach we want to increase their ability to assess their work
  • peer assessment (and critique) can allow students to experience a range of quality of work which they otherwise would not see

How

Peer critique is done often using de Bono’s six hat criteria. This is done verbally, in class and to begin with is highly moderated as students develop a language of criticism. However, peer assessment is most productive in group/collaborative work, this is because it is the group who best knows who has contributed, and it is this that needs to be assessed to teach collaboration – not the finished objects (as these are generally mute when it comes to indicating the contributions of the participants).

We have not developed these very far (partly because I am no longer teaching in those parts of the program that align with group projects). However, in the past I have used wikis to allow collaborative research to be done (which I am repeating this year) and to then develop protocols for the assessment of individuals in the group activity.

In this case each group undertook an activity that helped them identify their individual strengths and weaknesses, and on this basis they identified their contributions to the project. A contract was made, with a project timeline which showed who was responsible for what aspects, and when. They received a bonus if meetings were held, that were minuted, and these were submitted as part of the project. In this case the group received a common mark for the project (which was submitted with a report indicating what processes had been undertaken to achieve the project), but the contracts and timeline could be used as evidence is it was believed that their result was compromised by someone’s lack of contribution.

This was a successful first step, and I would have liked to have developed this further so that we could then develop protocols with students for the assessment of individuals within a collaborative activity – something I believe is readily possible and of great benefit to the students.

What is most important in all of these assessment activities is to recognise that:

  • the development of the skills and literacies to do this often forms the ‘content’ of a class
  • students must feel that they do have responsibility and ownership (this is achieved by giving him this)
  • if the teacher is unsure or prevaricates, it will does not work
  • that the approach must be graduated, it consists of small steps, in first year students are helped to assess themselves, in second year they define this, and then are helped to assess each other in a group, and so on – if you try to do a later step without the earlier ones it tends to collapse
  • these can be thought of as a literacy (if you like) and just as we spent many years learning how to write, we need to spend several semesters developing these (if you want to use all of them)

Outcomes
The clearest outcome that I experienced doing this activity was that students were able to recognise their individual abilities and so what they could, and wanted to, contribute to a project. All students had had experience of group work where someone did not do anything, or that they had to really put in a lot of hours to get the results they desired. The reflective tasks allowed them to identify what they were able to do and so all had significant parts of each project to contribute to. This can be further strengthened by using a similar reflective exercise as the basis of developing a group so that ideas generators work with makers and finishers (to use some simple types).

In addition this process separated the finished work from the collaborative process, and does let both be assessed (if desired). Increasingly the role of education is not to provide content to our students but to provide them with skills that can be implemented in different contexts

BLOGS

That’s probably enough, but we use blogs as an integral part of all that has been described here. The individual blog is where a great deal of learning is documented and explored, and it has the benefit of becoming a record which students can then use as the basis for evidence for process. This is important, and is pretty much a portfolio (if you like), as in this process orientated approach it is imperative that the process (the learning journey) can be documented, reflected upon, and evidenced.

Sorry

We've had over ten years of a conservative government who thought saying sorry meant we were accusing people, people who might

New Plumi

Plumi, an open source video sharing CMS based on plone, has just had a major upgrade. Don’t really have other news. Trying to pack the house as we’re moving (I hate moving). Trying to get together courses for the new academic year which is beginning real soon now. Head down, bum up.

Video Comments for Drupal

Andreas has ported David Meade’s WP video comment module so that it is now a video comment plugin for Drupal. It lets commenters in Drupal add a url to a video entry and a url for a video thumbnail and they then can be played inline.