I was going to tackle the list of skills first.  In fact I did.  Then I found myself waffling over how much info makes up a skill.  If the pieces are too small, there will be a thousand of them to grade.  If they’re too big, it will be impossible to summarize them in a single grade.  I kept thinking about whether it was assessable — which meant knowing how I would assess.

So I decided to try the rubric first.  Here’s a bit of blogosphere roundup:

 Max Score To get full marks: Continuous Everywhere but Differentiable Nowhere 4 Demonstrate the skill  Use algebra correctly Write clearly Show work Use correct notation Teaching Statistics 4 Demonstrate skill  Solve a complex problem Solve independently (lower scores for solving with assistance Brain Open Now 4 Demonstrate the skill  Write clearly Use algebra correctly Use correct notation Reason efficiently Draw valid conclusions MeTA Musings 4 Demonstrate full understanding Sarcasymptote 5 Demonstrate comprehensive knowledge  Use it in novel situations Point of Inflection 4 Demonstrate mastery  Connect to other skills Apply creatively

A lot of rubrics differentiate the score based on algebra (3 if you get part of the idea but make conceptual errors; 4 if you get the idea but make algebra errors).  This makes sense in a math class, but since I’ll be testing things like “can predict circuit behaviour”, I’m tempted to make algebra a separate standard.  If you make an algebra mistake that’s serious enough to draw wrong conclusions from, it’s a conceptual error.  But a lot of algebra mistakes have such small effects on the goal that they don’t change your predictions or problem-solving strategy.  There’s a whole other course that tests algebra, so maybe it’s wasteful to make it the main criterion in my grading scheme.  I think writing is also a separate standard.  There will be chances to assess the students’ ability to integrate their own writing with their own measurements/troubleshooting as part of project work (which I think I’ll grade on a separate rubric).

It’s possible to score based on how much assistance someone needs, but I’ll be assessing only in situations where assistance is not available.  “Demonstrate full understanding” and other similar language like “proficient” is not specific enough for me — I worry that students would have no definitive way of knowing what I consider proficient, short of asking me a million times a week (causes anxiety for them; causes insanity for me).

The last two, though, seem reasonable to me.  What I’m really testing — if I had to pick one thing — is ability to apply the knowledge to a situation that’s not exactly like anything in the textbook (though it might be a mix-and-match combination of previous problems).  This is also the hardest thing for students to understand.  I get a lot of pushback from students who are angry that I’m testing them on things I “haven’t taught” them — this despite practise problems, in-class time Q&A or group work, and hands-on activities practising the skills from various angles.  There’s a high correlation between that attitude and the Fs that came through on the last test.  So I’d like clarify that, in fact, that is what I teach… and what I expect them to aim for.

I’m tempted to use a 5 point system.  Someone wrote recently that they didn’t like 5 because it gave a “middle ground” score of 3 that was neither good nor bad.  I disagree: there is always a score of 0.  So it’s the 4-point scale that has a fence-sitting score.  In the final weighting, 2/4 will probably translate into 50%, which is the cutoff grade for supplemental exams.  That’s definitely something I want to avoid, both for my sanity and the students’.  They will want to know which side of passing (60%) they’re on: a 3 means yes, a 2 means “not close”.  So I think my synthesis so far is this:

1. understands something about the concept
2. understands lots of things about the concept but not everything
3. gets the concept, has problems with strategy or application of skill
4. applies the skill to a previously practised situation
5. chooses appropriate concepts to combine with and/or applies to novel situations

The wording needs some work, but basically breaks down the way I want it to.  Assuming I’ll average these for the final grade, a 4 is 80% (this is technically considered “honours” by the school, so I might have to make this worth 79% or something like that).  To get a 5 you’d have to do something cool.  If you did something cool and it mostly worked, I’d be ok giving a 4.5.   Some students will probably be confused about what the “appropriate” other skills are for getting a 5, but I think I can draw up some guidelines for that.  On the other end of the scale, if you don’t have a good grip on what the point is, you are below 60%.  Seems sensible so far.

Next up: some really interesting ideas I’ve stumbled across for evaluating rubrics.  I’ll put this one through the ringer and see what survives.

 Max Score To get full marks: Continuous Everywhere but Differentiable Nowhere 4 · Demonstrate the skill · Use algebra correctly · Write clearly · Show work · Use correct notation Teaching Statistics 4 · Demonstrate skill · Solve a complex problem · Solve independently (lower scores for solving with assistance Brain Open Now 4 · Demonstrate the skill · Write clearly · Use algebra correctly · Use correct notation · Reason efficiently · Draw valid conclusions MeTA Musings 4 · Demonstrate full understanding Saracasymptote 5 · Demonstrate comprehensive knowledge · Use it in novel situations Point of Inflection 4 · Demonstrate mastery · Connect to other skills · Apply creatively

1. What about the case where they can apply the skill to previously practiced exercises, but don’t understand the concept? That is, what about the classic “memorized all the homework” approach to taking tests?

• Yes, that’s a problem. I think that will be a 2/5 (or 1 or, theoretically, 0). I should ask that question on the blogs quoted above.

In a project or lab, it’s easier to discern than on a written test. I talk it over with each student, and over the past year, I’ve found it quite apparent when they don’t know what they’re talking about (I have the luxury of 12 students per class).

I think I will have to handle this partly through my choice of what constitutes a skill, and partly in the conversation I have with the class about how we’ll attack this. I’ll get to work on the skills post and see if I can chip away at it there. Thanks for the great questions.

2. […] so far, check out their most recent test scores in my first post.  I’d score myself at a 3/5 on my own proposed rubric, so far.  But don’t worry — I have plans to reassess myself […]

3. […] the level 2 stuff (although I switched to a 5-point system, for reasons that have not changed since this post).  I like this setup because it suggests an order in which to tackle things.  At the same time, […]

4. As a result, cars movie online itt is better you go in for more complex and advanced systems.
Anyway, so we decided to do that very competent in doing that.

5. Hi! Quick question that’s totally off topic. Do you know how to make your site mobile friendly?
My weblog looks weird when viewing from my iphone.
I’m trying to find a theme or plugin that might be able to resolve this issue.
If you have any suggestions, please share. Thanks!

6. Both quantity and quality will be greatly enhanced.
Froestad, analytical chemistry review 2006When residents were questioned
about the current situation, which at times has
been described as a pandemic. So that acid rain is a little
cooler than Venus is, but smaller than Earth; so small that the most
of the time in 2008 and consistently testing among the dirtiest
beaches from 2005-2008. It is ridiculous when you really think of the ruibbish that may end up in a township.