You are currently browsing the category archive for the ‘Progress Report’ category.

After my first trial of using the blogging rubric, I’ve decided that the rubric and the process both need tweaking.

In a post I made last December, I talked about our process of assessing blogging.  I decided that we would have two submission dates.  On the first submission date, the students would submit a draft, and I would give them feedback on if they had met competetncy (based on their demonstrated knowledge of the subject area).  They would then have a chance to polish their post & I would regrade it at the second submission date.  Sounds complicated?  Well surprisingly enough it is.  It seemed like a good idea to me at the time, but after running through it once I’m going to revert to our standard approach which is allow them to submit an assessment, mark it completely, then if anyone is marked as not competent they are allowed one resubmission.  Simpler for the students.  Easier for me.  (I don’t know what I was thinking).

The other thing that needs tweaking is the actual rubric.

After using it once I’ve decided that grading of community involvement is over-weighted.  In fact, requiring this has just made a natural process into an unnatural process.  It hasn’t seemed to increase authentic community involvement at all, but rather has led to a few students incorporating references into their blogs, and making comments on others blogs which are fairly pointless apart from the gaining of marks (I know Leigh, I know).

Another problem is that the use of reflection isn’t particularly relevant to this assessment, so I’ve modified the rubric to create

Oh well, one step at a time.  We’ll get there in the end.  😉

I’m getting into writing assessments for next year, and it’s clear that some aspects of our assessment model need to change. The main drivers for me are the need to increase engagement in online learning activities, workload reduction, and improving feedback.

Assess them and they will come

In my review of how things have gone this year, one of the things that’s really stood out for me is the fact that the level of participation in the learning activities that I set for my students this year was not even close to a level that I would be satisfied with. It’s clear to me that their learning has been impaired as a result (or at least their learning of the material that I wanted them to learn), and I’m pretty sure that the one thing that would have led to more participation would be more assessment.

Taming the workload beast

But we already spend too much time marking assessment! In a recent staff meeting, we talked at length about workload reduction. One thing that takes up a considerable amount of our time is marking assessment. I’m sure that I can design assessments to involve less workload for the assessor.

Anderson describes a range of methods that may act to reduce assessment-related workload for teachers (2008)

  • Automated assessment processes – ranging from formative tests (simple) to virtual labs and simulation exercises (complex)
  • Online automated tutors
  • Use of neural networks & other artificial intelligence methods
  • Peer review (of either students within a specific course, or students within a network of similar courses)
  • Student creation of open educational resources which are then assessed by lifelong learners who are using the resources (Farmer, 2005 as cited by Anderson, 2008)

Formative tests are fairly straightforward to implement. They take some time to set up, but then they’re there to use year on year. I have thought about creating a simulated clinical environment in second life, but at this point, the creation of automatically marked simulations is well out of my financial ballpark, so I’ll move on.

The next two are also a bit too high tech, and high budget.

The last two options are possible if the students of the course are a part of a learning network. (Anderson, 2008). One of my goals for the future is to develop this network, but I think it’ll take at least a couple of years of students moving through the programme before this happens to any particular degree.

Feedback

Feedback is crucial to the learning process, and this is something that we can definitely improve on. Formative tests that provide feedback directly following the student’s performance provide a wonderful development opportunity for students, and I believe that this is one of the real strengths of online education. According to Shepard (2000 as cited in Caplan & Graham, 2008), and Wiggins (2004), providing detailed feedback as close as possible to the performance of the assessed behaviour enhances student learning.

We should strive to “create assessments that provide better feedback by design” (Wiggins, 2004). I was inspired last year by the way in which Montessori school activities are based on this principle. Learning activities can be designed to provide feedback to students in the absence of the teacher. This can be facilitated through instructional design (Wiggins, 2004), or through social networks (Anderson, 2008). In my experience when courses I’ve been engaged with have required blogging, a community of learners has developed, where the learners have begun to support each other in their learning.

3 phase assessment process

After considering all of this, I’ve come up with a three phase assessment process that I think would be fairly ideal for most of our online courses. Phase 1 and 2 here test different grades of knowledge (simple/moderate complexity) & overlap in temporal space.

  1. Automated formative testing to test knowledge of discrete chunks of knowledge.
    Facilitator’s role: establish test, monitor results
  2. Reflective blogging on key concepts in the first ½ of the course. Students required to post on each topic, rewarded for commenting, updating the work they’ve done based on future learning, and referencing.
    Facilitator’s role: Monitor class activity, encourage engagement, Provide generalised feedback
  3. Final theoretical assessment which integrates learning.
    Facilitator’s role: Mark assessment, provide feedback & opportunity for resubmission

Students are therefore rewarded for acting as good community members, are given feedback on their developing understanding & are assessed for their integration of knowledge.

The one slight issue with this model is that if anything, I can see myself doing more assessing in this than I was doing previously. However the formative assessment that I’ll do in the early stages of the courses will be integrated with my teaching, so in effect I believe I could save time with this approach.

What do you think? Can you see any big holes in my thinking here?

References

Anderson, T. (2008). Towards a theory of online learning. In T. Anderson (Eds.). The theory and practice of online learning (2nd ed., p. 45-74). Canada: AU Press, Athabasca University.

Caplan, D., Graham, R. (2008). The development of online courses. In T. Anderson (Eds.), The theory and practice of online learning (2nd ed., p. 245-264). Canada: AU Press, Athabasca University.

Wiggins, G. (2004). Assessment as feedback. Retrieved December 11, 2008 from http://www.newhorizons.org/strategies/assess/wiggins.htm.

We’re through the second practical block, which has been another period of intense work. The task of getting assessments ready for three of the the four courses which have just started, led me to work until after 1pm two nights last week. Getting up at my usual 6am left me feeling like the Living Dead on Friday.

But I got there, and after a bit of a sleep in on Saturday, am feeling a bit better.

I get the sense that online delivery will be a bit like this – work peaks and troughs, as defined by the timing of practical blocks. I’m now through what I think will be the most intense period of the year for me. I’m about to launch into teaching Fundamentals of Massage as well as Study skills, and focussing a bit more on the second year students (it’s case-study season, and I need to get into developing the Business Studies course for them). I think I’m starting to get into some kind of rhythm instead of the frantic scrabbling to get everything sorted that has characterised my last month or so.

I haven’t managed to get the survey instrument out to the students yet, so I’m not sure exactly where the class is as a whole, but from a small survey of the students it seems that most of them have been doing far less than the amount of work they really should have been doing so far. To some degree it’s their cake that they’ll now need to eat (with four other courses starting up the demands on their time will increase quite dramatically), and the Study Skills programme has been designed to be self-paced, so it’s not the end of the world, but I am a little concerned. I’m going to send out the first survey on Monday, so I’ll get a bit more of a sense of where the class is at.

This is the first of what I hope will be many progress report logs.

We had the first practical block from Friday 22 – Monday 25 Feb. The students mostly seem to be pretty happy. There are several people within the class who are finding the idea of using computers fairly daunting. But they’re willing to give it a go, so I’ve been spending a bit of time reassuring them and pointing out the support options which are available to them (i.e. me, CLCs, Helpdesk, Open Access Suite).  It’s not helping that almost all students have had problems with their initial passwords not working, but we’re working through that.

We have 23 students enrolled.  Of these 5 are part-time and 4 live outside of Dunedin.  One of our students was planning to be full-time, but has had to move to Auckland at the last minute, and so is planning to study Bioscience 1 & 2 this year (which he can do by distance), then to come back next year or the year after & pick up the rest.

The online programme started on Tuesday. We’ve decided to only have Study Skills online in between Block 1 & Block 2 to give the students some time to get comfortable with communicating in the online environment.  I’ve been communicating purely via email so far, and today have started nudging the students into the use of Windows Live Spaces & messenger.

The students have a wikipage which contains all of the learning modules which make up the course. It’s a work in progress, but it’s not far away from being completed.  I’ve been working late most nights trying to pull it together – I’m just trying to keep a few steps ahead of the students. 🙂

The other resource which they have to guide their learning is their assessment. For the study skills course, the students have ten assessment tasks to complete. Click here to see the assessment overview.  Each assessment task has learning modules that support it. The students have been told that most of the learning modules are optional.  If they can complete the assessment without running through the learning modules, then I’m happy for them to do this.

We’re now three days into the programme.  Yesterday I did an audit of our email group, and found that four students were not receiving the emails that they should be, so I’ve added them in. Through marking the first assessment task I’ve picked up that 6 of the students are not participating as yet.  Some of those will be composed of the 4 that were not in the email group, but there are some others as well. So first thing tomorrow I’m going to ring them all, and find out what’s going on.

I’ve also realised that I need a regular feedback process to determine how the students are progressing through the programme, how they are feeling about the programme, and what problems they’re experiencing (if any).  It’s hard to get a sense of these things from where I’m sitting right now, but they’re so important in these early stages.  So I’ve drafted a survey up & am running it by our local expert before sending it out to the students.

So I’m mostly happy with how things are going.  I am a little concerned that so many students have not managed to participate at all, but I’m fairly sure that we should be able to work through whatever issues are there.  I’m also a little worried that some of the more competent computer users may be getting a bit bored, however the self-paced nature of most of the activities should suit them.  It’s definitely a learning curve.  🙂