Draft: Authentic Self Assessment of Participation: A Protocol

DRAFT
Authentic Self Assessment of Participation: A Protocol
Adrian Miles
(adrian.miles@rmit.edu.au)

Abstract
Historically, and anecdotally, participation is generally teacher assessed, and commonly used as a cognate for attendance and contributing to class discussions. However, these are very poor indices of participation as they are not judging what is done outside of class time, nor acknowledging the existing abilities or capabilities of the individual student. This paper uses thick description to narrate a pedagogical model that supports the critical self reflection of participation and its associated assessment protocol. This protocol is an effort to assess participation as an authentic assessment item in undergraduate teaching. This process based protocol allows for students to self assess their participation within a subject in a manner which is able to acknowledge their actual participation. This improves their understanding of themselves in their role as learners, and also leads to a more accurate measurement of participation.

Keywords: reflective practice, participation, self assessment.

Introduction
All that follows is premised on a teaching methodology which understands that learning is a practice, and therefore like other practice based disciplines reflection is an exemplary and important model by which the processes, activities and deep structures of these practices can be investigated and understood (Schn 1983, 1987). The object of such reflective practice is not for our students to become content experts around education, but to become articulate about themselves as learning practitioners, on the basis that these skills, or literacies, are relevant and applicable across all of their learning and into their professional careers. This reflects the deep change in the knowledge environment and experience that students now have where we have moved from an economy of information and knowledge scarcity where the role of the university and the teacher was to be a source of rare and privileged expertise and guidance to one of excess where our role, and that of our students, is no longer the problem of how to gain access to expertise but how to successfully harvest, farm and triage the plethora of high and low quality resources so readily available. In such environments one of the aims of teaching is not to judge the merit, or otherwise, of student work but to teach the student ways for themselves to be able to judge and value what they produce, and why. As we increasingly move to a world where intellectual property is shared, readily available, and where we desire increased autonomy in our careers, it is essential that students stop treating teachers as the determinants of value and quality and develop a vocabulary and reflective understanding of their own work so that they are able to know when it is excellent, good enough, or lacking.

This essay adopts a method of what Id characterise as synchronic thick description. This is appropriated, with omissions, from Geertzs anthropological method (Geertz, 1973) which can be characterised as an effort to describe an activity with as much context and detail as possible on the basis that meaning is embedded within and constituted fundamentally by its context. In addition, and to paraphrase Geertz, if you want to understand what education is, you should not look at its theories or findings, you should look at what the practitioners of it do. This means I describe, with significant detail, the protocol I have developed for student self assessment of participation. This description will concentrate on my individual implementation and practice. I believe many, like myself, find themselves teachers by accident, by virtue of being academics, so I hope that a detailed description and discussion of my practice will offer many points of contact and intersection for others, the sort of thickness that a more formal or data driven education research essay misses.

Finally, a caveat. While this essay is specifically about assessment I do not regard myself as an educator in the academic sense. That is I teach, like most academics, but my disciplinary training and practice is not within education. Hence, what follows is the product of my own appropriation and misreading of a variety of rather generic ideas around teaching and learning, combined with my interest and application of a deeply reflective methodology to my own teaching practice. This informs my use of thick description as a method because I am interested in exploring and documenting practiceinitself rather than a report or other evidence based research.

A scenario
You are teaching a university course where some percentage, lets say ten, of the overall grade is allocated to participation. When it comes to allocating this participation mark you rely on attendance, and perhaps contribution, as the things to help determine what mark students receive for participation. You know this is not very accurate, or perhaps even meaningful, and you also know that most of your students think that participation probably equals attendance after all it is only worth ten percent of the overall mark, so near enough is probably good enough. Oh, and it is always useful to have a bit of room up your sleeve to reward those that deserve a lift perhaps because they always contributed during class and conversely trim those high fliers who have done well in spite of never actually attending.

New teacher, new subject, first class. Looks well organised, been given all the assessments, a reading list, outline of the lectures and tutorials. Lets see. A fifteen hundred word essay worth thirty percent, two and a half thousand words worth sixty percent, and participation as ten percent. Pretty standard. So, most of my marks are from the essays, and the questions are based on the readings? Cool. Ive always been an excellent essay writer, the readings are pretty straight forward, and the questions just need me to rework what the readings say. The lectures are being podcast? Ok, so can skip going to them, lets see how the tute goes. Ill make sure I ask a question, get noticed, but as long as I do the readings and write Ill get a fine mark here.

The problems with the scenario (why participation matters)
The scenario described above, which anecdotally seems to have some verisimilitude for many new staff and students, raises several problems. The first is that we generally recognise that participation, genuine participation, is fundamental to successful learning and that such participation extends beyond the simple measures of attendance, or even asking questions while in class. If this is the case, and it is difficult to argue why it is not (after all there is reading, thinking, reflecting, writing, making, and so on which all happens outside of the classroom), then the simple use of participation assessment as an index of attendance and conversation seems to miss much of what might constitute participation.

This raises the second point, which is simply that the scenario does risk reducing participation to something quite literal, and banal, and so fails to assess what it purports to be assessing. It also reinforces quite a false sense of what it means to participate on behalf of the student. It does this in two ways, the first by leaving participation as an unquestioned assumption within the learning experience, and on the other by reinforcing this quaint view of assessment through rewarding it through the provision of teacher directed marks.

Finally, if participation, however we eventually define or consider it, is important for students to do in order to successfully learn in class then giving it a weight of, in our scenario, ten percent would seem to neuter this simply because the contribution it can make to the students overall grade is so minimal. As a consequence the student, if they wish to perform well in the subject from the point of view of what counts academically (that is in terms of what is actually being assessed) must dedicate their time and attention to other tasks. This is simple common sense, and while we may then wonder why so little happens in our tutorials, or why students dont seem to want undertake additional reading or activities beyond the minimal necessities, it makes little sense to do so if it is not being rewarded. As a teacher what you choose to assess, and the overall value of this as an assessment task, is a statement by you about what matters in your teaching, and is certainly interpreted by your students as such. So if you insist that participation is important, but it constitutes only ten percent of their final grade then there would appear to be a contradiction between what you profess and what your assessment performs.

On the other hand our student in the above scenario understands only too well what is being asked of them. They are an excellent student, the sorts that most of us who became academics were probably like, and so they will most likely attend the tutorials (if only because they already and always have enjoyed learning) and contribute. However, they also know that what counts and matters in the subject are the two essays, that this is the key in relation to getting an excellent result. Furthermore, that to get this result they need to do the readings and of course the essays. In this scenario participation, the actual participation that this subject requires is reading and writing but the activity and process of this is not being assessed. The tutorials may contribute to the essay writing and understanding the readings, but the essays are being assessed in relation to an understanding of the readings and this does not have any necessary relationship to the amount of time, reading, or learning that may have taken place for this student.

In my discipline these can be controversial claims, so Ill unpack them a little. Remember, our student is a classic high distinction student. The sort of student that is often used to show that what I do must be OK since look at the quality of their work. The problem with this is that the student is already motivated to learn and, certainly in my experience, also able to contextualise knowledge in abstract ways. In other words their essays and other assessable work always exhibits the qualities of deep learning. This is present for this student before I have taught them, and will continue afterwards it is not a consequence of my teaching. So for such a student writing an excellent essay may still require considerable work, and of course they may have learnt a lot, but assessing the essay is not a measure of this since there is no mechanism to measure or determine what they knew before, after, or actually had to do in the process of writing the essay. Hence, if the student already has a very strong understanding of the content material they will probably do very well, without actually learning much more, and they will not have needed to undertake or learn any of those collateral activities that participation might encourage, promote and teach.

Now, such assessment is of course not wrong, but there is little scope here to measure, teach or model the specific activities that students actually may need to do to learn successfully, nor anything that measures what work has actually been done. More significantly, with assessment structured in such a way there is little incentive for students to undertake activities that will contribute to their learning beyond the pragmatics of completing the task. For these other activities to be reflected and rewarded via assessment, and for an opportunity to investigate and teach what these activities may actually be, the assessment of participation as an activity and process in itself needs to be developed.

It is in response to this problem that I have, over several years, developed a protocol for a richer and more relevant assessment of participation by students within subjects. The aim of this protocol is to try to make participation a more central and grounded element of the learning experience, to offer students a method by which to reflect upon and consider participation in a manner that supports the development of the student as a reflective and self aware learner, and to reward this participation in a meaningful way.

The protocol
The protocol that I have developed varies depending on the year level of the subject, and what previous experience students may have had with the assessment of participation. However, the basic premises are that a) students are the most accurate judges of their own participation, b) that participation is not equivalent to attendance, and c) if students are given sufficient ownership of the process they accept the responsibility, responsibly. In what follows I am making a series of implicit propositions about the value of participation, student based assessment, and reflective practice.

The process begins in the first lecture, where after the distribution of the various documents and a general overview of what the subject entails I move onto the discussion of assessment. In a first year subject I would ordinarily have participation valued at a minimum of 25% of the overall grade and this is indicated in the material that has been distributed. The discussion around participation provides much of the formal content of this first lecture. First of all, I talk about how in many university subjects teachers will tell you things like you only get out what you put in or that participation is really important, but that it might only make up 10% of the overall mark for a subject. I then wonder out aloud that if participation really is important, then what does it mean for the teacher, and the student, if it is worth 10%? I also wonder what might count as participation in such contexts and touch on the usual criteria of attendance and contribution. I do not invite responses to these questions as they are to model an imaginary internal conversation from the point of view of a teacher and a student to make concrete some common approaches to participation in university subjects.

From this point I talk about my own approach to participation. This concentrates on the claim that you really do gain much more by participating, and that participation is a substantial activity and so should receive a substantial assessment weight. Furthermore, I ask if participation is not just or only attendance, then what else is it? This operates as a prompt and all students are invited to draw a simple line graph for the semester. The x, or horizontal, axis is the duration of the semester while the y axis is how much they know about the subject area. They are to indicate how much they think they know about the topic now, and then how much they think they will know by the end of the semester, and to then draw a line connecting these two points. I then ask, in all seriousness, if anyone has a graph with a line that descends. No one ever does. I then ask if anyone has a line that more or less remains horizontal. Again, no one ever has. I then ask who has a line that ascends over the course of the semester. And, of course, everyone has. I describe how in doing this each of them has said that they intend to learn during this semester, and that this intention has nothing to do with me, that they have defined this for themselves and that this is an agreement, orientation, or attitude towards learning in this subject that they have bought of their own accord.

I then ask the key question for this exercise. What sorts of things do they think they need to do to make that learning line go up? They then list everything they can think of in their notebooks, and after a few minutes of feverish writing we compile a list using the board. Students call out things they have on their list (only one thing per student) and I write them all down, asking if anyone has anything else once students have stopped self nominating tasks. The list generated is invariably extensive, detailed and impressively thorough. Some items may generate some questions, for example ask questions is common so Ill simply wonder ask questions of who?. The inevitable response is the teacher so Ill then ask who else should, or could, they ask questions of so that they can see that they can ask questions of each other, other academic staff, even possibly their friends, family or external experts via email or blogs.

This list of activities and actions is defined as participation for this subject, and that participation is all of these things, and not simply turning up to class. At times this conversation might be extended to help make concrete the way in which attendance turns out to be quite a small element of participation, or discuss the ways in which only thinking about attendance misses all of these other activities. The implication of this, which is made explicit in the lecture, is that these are the things that will help you learn, but if we were then only assessed on attendance then we would not actually be assessing participation. It is then clear that there are many other ways to participate outside of attendance, and that even within this quite small thing of attending there are a range of other activities that need to be done so that attendance becomes participation.

In the lecture I now use various coloured markers to collect the large list into relevant smaller groups, just to make things more manageable, and to find metalevel activities or headings that can be used. These usually (though not always, it does arise from the list the students have generated) include top level activities such as research, contributing, asking questions, reading, and depending on the particular subject, writing, or making. Under these are collected all that are relevant so that, for example, research might include things like read the dossier, visit the library, read other relevant media, find other references and find relevant websites.

Now that we have a list of activities which defines participation the lecture moves to a more dialogic mode where I use two probes to begin to work through the actual assessment of participation. The first probe is to ask who would know if you had done these tasks? This conversation seems to always follow the same course for my first year students. Someone might suggest that I would know if it had been done. If this is raised then I ask how would I know if these things had been done. This often leads to a more lively discussion as Im told that in the things asked in class, the work that is submitted, and the contributions in class then I, as the teacher, would have a good sense of how much everyone had done. While agreeing that this might be the case, I wonder out aloud about a student who already knew quite a bit about whatever the subject is, and I ask the class whether a student who already knew quite a bit would presumably have plenty to say in class, and in general their work would indicate a very good understanding of the material. Well, of course, Im told, and most are quite happy with this.

However, I point out that we are discussing participation as they have just defined it and that the scenario just described is not assessing participation, it is assessing how much someone knows, and that is a different sort of thing altogether. This also lets them see that the sorts of indices they thought might let the teacher be able to judge participation dont actually work, and so I repeat the probe and ask again so who would know if you had done these tasks? This time there will usually be a cautious we would? from someone and I reply with a usually resounding yes! This often requires a little more elaboration, but most students by this point realise that they are the ones who would know if the things listed as participation had been done or not, particularly since so many of them happen outside of class. This leads to the second probe which is the obvious so who should assess your participation then? which leads to another cautious we should? and my second resounding yes! This always requires more conversation and often the steps used to get to this point will be reiterated. The key premises that the students need to understand are that participation is made up of a range of significant activities, that they are the ones best placed to actually know if, and how well, they have been done, and therefore if it is to be assessed then the only fair way to do so is for them to do it. This inevitably leads to the excited realisation that they could give themselves a top mark, and I assure them that yes, they could but as hard as it might be to believe now very few ever do because when you realise that you really have been given this responsibility the vast majority of you actually step up to it. There are, invariably, some who think this might delegitimise the academic integrity of the subject and I usually have two standard replies. The first involves me pointing out that everyone in the lecture is old enough to drive a car, drink alcohol, vote, get married without their parents permission and own a gun, theyre adults now and so can be trusted with these sorts of responsibilities, including this one. The second involves the importance of reflective practice to their future professional practice and their expectations for their own careers. This involves pointing out the importance of being able to judge for themselves what they need to do to achieve something, and whether theyve done it well enough that the majority of them will not want to be in a workplace where someone senior always looks over their shoulder to tell them if it is good enough or not, and that in most of the industries that they will work in they will be expected to be able to know if their work is good enough. The self assessment of participation is one small step towards this larger reflective understanding.

The list of activities that the students have made is documented and after the lecture summarised into a participation diary. Essentially, I use headings and list key activities under these headings. It is important that the items listed under each are quite explicit, so that the participation diary can not only be used as a record of activity but also as a guide for the sorts of activities that need to be undertaken. In addition, it is important that all the descriptors used for the activities are concrete, measurable actions so that it is easy for the student to not only know what to do, but whether or not they have done it. For example, while do research is an admirable goal such an item, by itself, is too woolly to be meaningful, whereas something like find two relevant references describes a research activity (providing direction and focus) and is also easily measurable.

This participation diary is limited to approximately ten individual items. Any more than this and participation gets lost in micromanagement and the role of the diary in supporting and enabling participation becomes confused with auditing rather than enabling student learning. This diary is photocopied and distributed in the second class and the very first activity then undertaken in each class is to complete the participation diary for the previous weeks activities. I have found that it is important for the diary to be completed first of all because it models its importance and it also means is done properly. If left to the end of class then a) you usually run out of time to do it, and b) it is treated as a minor after thought to the actual class.

Recording participation weekly in this way allows each student to get a sense of history to their activities. It means they have an informal audit trail of what they have been doing and also provides the opportunity to run a trial assessment at any time during the subject. This, particularly when first trying to use this self assessment protocol, is very useful as it helps make the actual assessment and participation activities concrete for the students. A trial that I will often run is to get individual students to go over their assessment diaries in small groups, discussing what they think theyve been doing pretty well, what they havent been doing well, and what mark they believe they would give themselves for participation at this point. This conversation often enables peer support for students where others offer advice about how to do whatever activities an individual may feel they are not doing well, and it also helps reinforce the key idea that participation is made up of a series of activities that must be regularly undertaken, it is not just attending class and completing the diary!

Finally, during the last week of classes everyone is invited to look through their individual participation diary and to write down what they have done well, what they think they have learnt to do better, and what they could have done better. On this basis they are to determine an overall mark for participation for the semester.

This involves a lot of conversation with some of it about whether or not I will moderate the marks in anyway (I wont), but also to think about how to actually assess their diaries. For example, I point out that it is a personal assessment, it is not about how they think they have performed in relation to another student but quite simply for the list of things we have as participation how well and how often have they done them. This returns us to some of the opening comments from the first lecture, but the three questions described above explicitly scaffold the activity as they do not relate to content mastery but are explicitly about their role and activities as learners. Each student then presents their answers to these questions to the class and the mark they are giving themselves for participation. Other students are allowed to ask questions of the presenter, and to make suggestions regarding the mark (for example it being too high, or too low) and the grading scale used is the same as the universitys and so consists of fail, pass, credit, distinction and high distinction.

Outcomes and Problems
In practice this final class has been exemplary. Students generally give themselves marks that are reasonable, and they are able to articulate relevant and incisive answers to the three key questions, even those students who have been unengaged by the subject who I might expect to not take the process seriously and abuse it have, without fail, given themselves either a pass, or at most a credit, willingly acknowledging that in terms of participation they did very little furthermore in some instances these are students who have performed quite well academically.

In general very few students give themselves high distinctions. Anecdotally, this appears to be a consequence of what in Australia is described as the tall poppy syndrome, though it also reflects the fact that while excellent academic students would probably receive high marks for participation if teacher assessed, such students can do well quite easily which also means that when they are asked to assess their own participation they recognise that they have not done all that they could have, or even should have. Additionally, the third question asking students what they could have done better? also provides a robust sounding board for them as this requires them to acknowledge that there will be things that they did not do properly or perfectly.

Students who may not excel academically also can do very well through this protocol as it is not assessing content expertise but their participation and engagement as active learners. Hence, students may still struggle with the academic components of the subject but can be rewarded appropriately for their active and engaged participation. It ought to be clear that this provides support for not only a diversity of learning styles within a subject, but can also complement those students who may be introverted, shy, or culturally intimidated by the group class room context.

I have used this protocol for six years, and in my experience it has been explicitly abused three times. The first was a mature age coursework Masters student who literally described the assessment as bullshit and gave themselves a high distinction. Interestingly this was a very small cohort, all participants were professionally employed, and while everyone else seemed to appreciate its value this student did not. In retrospect I understand that this student was a classic high achiever, and what frustrated him the most was that he wanted me to tell him what he needed to do to achieve a high mark. I had no doubt whatsoever that he would meet any benchmark that I set, but he absolutely required me to define this for him he was literally incapable of being able to define any of these qualities himself.

The second event was in a second year program where a student awarded themselves a high distinction, in spite of many articulate (and offended) comments and protests from her peers. In this case the class looked at me to amend the mark (the student certainly did not deserve the mark they were giving themselves) but I insisted that no, I was not going to intervene. She pragmatically described how she was very anxious about her results for the academic (content) aspects of the course and that if she didnt give herself high marks for participation she believed she may fail the course. I pointed out in the class that this student was willing to do this, knowing that everyone else thought it unreasonable, and that if she were comfortable with this then it was her reputation that she was playing with. She was.

Finally, a third example occurred in a class that I did not teach. A student who apparently had not attended any classes through the semester attended the last one (where the participation assessment is undertaken) and gave themselves a high distinction for participation. The class, quite reasonably, where upset by this and did not think it was appropriate. The teacher, who had not been mentored in the rationale of the protocol, was of the same view, and intended to provide a teacher directed grade for this student. The original result for the student was reinstated after a conversation with the teacher where it was stressed that the protocol depended very strongly on the students concrete understanding and experience of ownership of it. However, in a context where it is abused in this way there are several things that need to be borne in mind and also made explicit in the class. The first is that the teacher and the class know that what the individual student has done is wrong. The student knows this indirectly, by virtue of how the class has responded, but it is reasonable to think they dont understand or get it simply because they continue to insist that they are giving themselves the highest mark possible. By not participating through the semester it is too late to try and teach the student why reflection and participation matters, but to modify the mark as a consequence of this treats assessment as punishment and moves the teachers role from being mentor and collaborator towards that of judge and external validator. In place of this a very clear conversation can be had in the class about why the ability to appropriately assess your own contribution matters as a student, a learner and professionally. From this you might also have a conversation about what it might mean to stand up and give yourself such a mark in this context what it means you are saying about yourself and your peers and how you regard them and their efforts. I would also ask the class to think about (and answer) how they would value working in a group with such an individual. Personally, while I would be angry with a student who did this, and Id feel aggrieved that my efforts at teaching had been so mistreated, as long as they heard that their actions would significantly compromise their reputation (this was a first year course with a clear understanding that a significant amount of group work would be done in future semesters), and make others unwilling to work with them, I would hope that the consequences of such decisions would eventually become concrete at some later point for the student.

However, others have had mixed results in using this protocol and have not had the same general success that I have experienced. This is generally evidenced by a lack of rigour in the assessment, usually expressed as a large number of very high results. That I dont have this experience could be a consequence of my personality as a teacher, however I suspect it is more simply that I have a great deal of faith in the protocol, it is discussed in great detail, often over several classes, and considerable commentary is offered as to why being able to judge yourself appropriately in this manner is both pedagogically and professionally sound. Because of this scaffolding, and the implicit trust I construct and give to the students (for example that I will not moderate the result), when it comes to the evaluation it is conducted with impressive integrity and often surprising honesty.

Conclusion
The use of this protocol has had significant outcomes for students in terms of their development as reflective practitioners. It makes participation a more concrete and pedagogically grounded activity for the students and also makes the assessment of student participation more relevant, accurate and authentic. This model shifts what is being assessed away from content expertise towards a critical engagement and reflection on their own practice as learners, and while this may be problematic such skills are relevant and transferable across disciplines, and importantly into the professional lives that they are receiving training for. This is a small activity that encourages an autonomy of practice within our students that helps move their understanding of knowledge, and our role as teachers, away from an industrial, audit centred system towards a more reflexive, post-industrial model.

References
Geertz, Clifford. “Thick Description: Toward an Interpretive Theory of Culture.” The Interpretation of Cultures: Selected Essays. New York: Basic Books, 1973. 3-30.
Schon, Donald A. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions. San Francisco: Jossey-Bass, 1987.
Schon, Donald A. The Reflective Practitioner: How Professionals Think in Action. Basic Books, 1983.

One thought on “Draft: Authentic Self Assessment of Participation: A Protocol”

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

video blogging, et al.