You are currently browsing the category archive for the ‘Grades’ category.
I’m thinking about how to make assessments even lower stakes, especially quizzes. Currently, any quiz can be re-attempted at any point in the semester, with no penalty in marks. For a student who’s doing it for the second time, I require them to correct their quiz (if it was a quiz) and complete two practise problems, in order to apply for reassessment. (FYI, it can also be submitted in any alternate format that demonstrates mastery, in lieu of a quiz, but students rarely choose that option).
The upside of requiring practise problems is eliminating the brute-force approach where students just keep randomly trying quizzes thinking they will eventually show mastery (this doesn’t work, but it wastes a lot of time). It also introduces some self-assessment into the process. We practise how to write good-quality feedback, including trying to figure out what caused them to make the mistake.
The downside is that the workload in our program is really unreasonable (dear employers of electronics technicians, if you are reading this, most hard-working beginners cannot go from zero to meeting your standards in two years. Please contact me to discuss). So, students are really upset about having to do two practise problems. I try to sell it as “customized homework” — since I no longer assign homework practise problems, they are effectively exempting themselves from any part of the “homework” in areas where they have already demonstrated proficiency. The students don’t buy it though. They put huge pressure on themselves to get things right the first time, so they won’t have to do any practise. That, of course, sours our classroom culture and makes it harder for them to think well.
I’m considering a couple of options. One is, when they write a quiz, to ask them whether they are submitting it to be evaluated or just for feedback. Again, it promotes self-assessment: am I ready? Am I confident? Is this what mastery looks and feels like?
If they’re submitting for feedback, I won’t enter it into the gradebook, and they don’t have to submit practise problems when they try it next (but if they didn’t succeed that time, it would be back to practising).
Another option is simply to chuck the practise problem requirement. I could ask for a corrected quiz and good quality diagnostic feedback (written by themselves to themselves) instead. It would be a shame, the practise really does benefit them, but I’m wondering if it’s worth it.
All suggestions welcome!
My standard (informal) course feedback form asks,
- What do you like or dislike about the grading system?
- How does the grading system affect your learning?
- What do you love about this course?
- What do you hate about this course?
- What would you change about this course?
The 2nd-year courses are less science and more engineering, so my approach is less inquiry and more project-based. In particular, in the course they’re evaluating, there’s an independent project where students must define their project, set their own deadlines, set their own evaluation scheme, then grade themselves. It’s worth a quarter of their grade. I reserve the right to veto a mark, but I’ve never done it. Here’s a sample of the feedback I got from 2nd year students last week.
1. Grading system
- Love reassessment (2)
- Feel dependent on ActiveGrade
- Need quicker way of knowing when a test is corrected
- Love the independent project
- Make reassessment deadline start when grade is updated?
- Ability to do skills on your own time. But they can also pile up.
- Clearly shows what you need to know
- Retests help a lot with understanding because you know what you need to improve on
- Showing improvement helps solidify thoughts
2. Effects of Grading System
- Reassessing forces you to gain understanding instead of “I failed that let’s move on”
- I can thoroughly explain certain circuits from my head, I could not do that before.
- Helpful — I can choose to not finish a lab if I do not understand it fully, then ask questions and come back to it
- I knew nothing about electronics before this course but skill based learning has really helped me understand many topics
3. Love
- Reassessing forces you to gain understanding instead of “I failed that let’s move on”
- Lab work — hands on feel
- Making things work and understanding what they do
- Freedom
- Retests, doing something more than once makes remembering it easier.
4. Hate
- Lack of info on notch filter (2)
- Lack of time
- Nothing
5. Change
- Hands on – when you don’t quite understand something, lab work refines understanding
- It’s a pretty refined, good system. Once you know something, it sticks with you.
- More time to learn. 3 years?
- Reassessment deadlines
The teacher’s skill sheet was a success (thanks, Dan). Today was our third day with the first-year students, and my first time explaining skills-based-grading to an incoming class. Our reassessment period is Thursdays from 2:30 – 4:30, so in this morning’s shop class I dropped a skill sheet on their benches and we started using it. By the time I started explaining how I grade this afternoon, they already had a skill signed off.
I handed out their skills folders and the first two skill sheets for DC circuits. You should have seen their jaws drop when I explained that they can choose if, when, and how often they reassess. They asked great questions and gave thoughtful answers. We talked about how everyone progresses, the many ways of getting extra help, learning at your own pace, and the infinite ways of demonstrating improvement or proficiency. They wanted to know what is proof of improvement (required when applying for reassessment), and had suggestions (quiz corrections, practice problems, written explanations). They wanted to know what level 5 questions are, where to find some, and how to prevent them from getting too big. Many of them had ideas in mind already and we bounced those around to see if they meet the criteria (at least two skills, and you have to choose the problem-solving approach yourself, so it can’t be the same as something we’ve done in class).
We talked about how and why you couldn’t get credit for level 4 until you’ve completed level 3. I explained it in terms of employers’ expectations about basic skills. One student explained it back to me in terms of “levelling up your character” in role-playing games. We talked about feedback, from me and from themselves. I gave examples of feedback that does and does not help you improve (“I need to figure out why V and I are different” compared to “I don’t get it.”). We talked about how many points homework is worth (none). My get-to-know-you survey tells me there are a lot of soccer players in the room, so we talked about practices and push ups. “Do you get points in the league standings for showing up to practice? What about for going to the gym?” I asked. Of course they said no. “So why do it if it’s not worth points?” They got this right away. “It helps you win the game.” “It makes you stronger.”
I enjoyed this conversation:
Student A: “So homework is just for learning.”
Me: “What are you talking about? I thought homework was for sucking up to the teacher.”
Student B: “I thought so too. That’s why I never did it.”
Student C: “I thought homework was for keeping kids in their homes at night.”
Once the questions had died down, I gave them a copy of a skills sheet that looks just like the ones I use to assess them, except that all the skills relate to my teaching. I asked them to sign and date next to any items they had evidence that I had done. I did this so I could find out if they really understood how to use the thing. But it had unexpectedly positive side-effects. From a quick glance, they could tell that I was going to get a “failing” grade. It never occurred to me that they would be upset by this.
They had barely started reading when I started hearing gasps. “You’re failing!” someone called out. “Is our assessment of you going to affect your assessment of us?” someone else half-joked. “Of course I’m not passing yet,” I replied reasonably. “It’s the second day of class. There’s no possible way I could have done 60% of my job by now. That’s how it works: you start at 1, then you move up to 2.” I walked around and peeked over shoulders to make sure they got the mechanics of what to fill in where. I stopped a couple of times to talk to people who seemed to have overly generous assessments. “How have I demonstrated that?” I asked.
We reviewed it together. We got to practice technical reading in tiny, learning-outcome-sized pieces. The highly condensed text on a skill sheet changes meaning if you miss a preposition. Another unexpected side-effect: my students had noticed me doing things that I hadn’t noticed myself. They had evidence to support most of their claims, too. There were a few that I disagreed with because I had only demonstrated part of the skill, and I modelled the kind of feedback that my “teacher” could have given me to help me improve.
Overall, they seemed very concerned about my feelings about “failing;” we calculated my current topic score at 0.5/5 and filled in the bar graph on the front of the skill sheet with today’s date. I got a chance to model a growth mindset. I made sure to let them see how proud I am of having achieved a 0.5 in only two days’ work, and mentioned that this is an improvement over two days ago, when I had a zero. The usual running commentary of tongue-in-cheek jibes had a disarmingly earnest, reassuring tone. “I know that you can improve your score the next time you reassess,” one student said. Another student chimed in with “feel free to drop in to my office anytime if you want to get some feedback.”
As I get ready to launch into my second September, I’ve gone over the feedback from last spring. If you’ve read since the beginning, you know that last December, half of my class was failing and the rest were bored. There was a lot of “why do we have to learn this?” and “is this on the test?”
By the end of this semester, no one failed, and there were some remarkable changes in our classroom culture. One of my colleagues said “when I check labs now, they show me which findings they think are important, instead of waiting for me to tell them what important things they should have found.”
I did some informal evaluations (I stole these questions from Robert Talbert at Casting Out Nines, and they worked well.) I started getting feedback that sounded like this.
What do you like/dislike about the grading system?
Like: Keep trying skills until you understand it
I’ve actually grown pretty fond of the skill system. I like that you actually make us demonstrate our knowledge of the individual skills, it actually helps me remember better sometimes, specially when going over quizzes. The only thing I don’t like is that to get a skill checked off, mainly in the shop, it can take a long time.
The grading system works very well although I think using the skills for every aspect of the course is a little too flexible. Using skills for the lab and going back to regular marked assignments. I need more room, I will talk to you later.
Skills for quiz bad idea. I had no ambition to study for test/quizzes. I like the shop skills tho.
I dislike the unstructured feel of it, simply because I do better with the assignments/tests, but I do like the ability to retest on a skill if you don’t get it the first time.
Independent learning project was fantastic and incredibly valuable in the long run.
I really appreciate you trying something new, and already there is a huge improvement. I hope you continue to innovate and improve the system.
I think the skills are very straight forward, they let us know exactly what you’re looking for.
It all encourages independence, which is great, unless you’re unmotivated.
There needs to be more communication.
Without marks to fuel my ego, I lost my drive to excel.
I think it helps focus more on the important stuff, and less on just completing useless lab stuff.
I was able to learn more with a smaller [work] load. This gave me time to play and experiment, by approaching labs in a way that was helpful to me.
Yes. It’s taken the good parts out of the lab book and made them easy to learn.
It certainly kept me on my toes to make sure that I understood what was needed to do the labs and the tests.
Yes. Previously, I would be missing a small piece of the “puzzle,” this way I know what I need to do.
What do you LOVE about this course?
A lot more feedback this semester, understand concepts easier
The learning environment, the flexibility…
I love that I am actually doing well in this course…
The ability to work at your own pace (even though you have to remember not to procrastinate)
Hands-on
Designing my own labs
I feel that education has in general become stagnant, and I was delighted to have a teacher who was willing to try something new. I know this takes courage and a lot of hard work. Having 25% of my mark based on a project I was able to pick and have it graded in a way that suited me was a blast.
Electronics 🙂
All the freedom
The instruction and the easy feeling that one understands what is being taught.
I liked the independent learning project, even if I had been a bit too ambitious in my designs and dreams
What do you HATE about this course?
I wouldn’t say I hate anything really except there’s a lot of work sometimes.
Nothing.
Quizzes! don’t do well on them, if get one part wrong, all wrong
Other students asking questions on things we have already covered in class, then interrupting the instructor when trying to respond
If you could change ONE THING about this course, what would it be?
More level 5 questions on tests. It is necessary to go above and beyond to get 100% on most modules.
Give assignment due dates.
More availability with students during lab time.
Harder deadlines, required milestones for the self-directed project
Level 5 questions: being bonus because sometimes difficult or busy time schedules to get one ready and do research
Include marked assignments somehow
To have a mix of skills and assignments
Points for homework so I’m more motivated to do it
More hands-on and practicing circuits
Any other comments about the course or the teacher?
Keep on getting better, you are doing a service to your students by furthering education.
I really enjoyed the year. I just wish we had the skill program for the first year as well.
I like this semester better than last semester. Keep up the good work!
Summary
My students are awesome, and almost as invested in developing me as I am in developing them.
Students really get reassessment. Not a bad place to start when introducing the “sales pitch.”
They want more feedback, and they’re asking for it explicitly. This is fantastic. I require work samples as part of an application for reassessment now, so that should help. I’ll also be experimenting with BlueHarvest.
Reassessment changed the concept of “studying.” I think this is a good thing. I suspect that what they mean by “study” is “do a long series of identical problems until you’ve got the procedure memorized,” and I’m ok with letting go of that. At the same time, I need to spend more time helping them learn to test themselves, so that they’re not relying exclusively on my tests as a way to diagnose and learn.
It made them look hard at who they are, what they want, and why they do what they do. I need to be ready for that. Students probably could use some preparation for it too.
It exposed the squirming, seething reality of the differences between my expectations about teaching and their expectations about learning. Dan Goldner’s got a great idea about how to clarify what the teacher’s job is, and I’m going to try it.
But hands-down the most fascinating thing that happened this past semester was that my students begged for homework. Many interesting conversations ensued (post about this forthcoming). Removing points for homework may have been the single most useful thing I did all year. To be continued.
Can my students use their skills in real-world situations? Heck, can they use their skills in combination with any single other skill in the curriculum? When I was redesigning my grading system, I needed a way to find out. It’s embedded in the “levels” of skills that I use, so I’ll explain those first.
What are these “levels” you keep talking about?
For every curriculum unit, students get a “skill sheet” listing both theory and shop skills. Here’s an example of the “theory” side of a unit of my Electric Machines course. (For a complete skills sheet, showing how theory skills correspond to shop skills, and the full story of how I use them, see How I Grade). If I were starting this unit over, I would improve the descriptions of each skill (“understand X, Y, and Z” isn’t very clear to the students) and make the formats consistent (the first four are noun phrases, the last one is a complete sentence; things like that annoy me). But this should give enough info to illustrate.
So, about synthesis…
Realistically, all skills involve synthesis. The levels indicate complexity of synthesis, not whether synthesis is involved at all. My goal is to disaggregate skills only as far as I need to figure out what they need to improve — and no further.
L2 Example
For example, in the unit shown above, wound-rotor induction motors are at level-2. That’s because they’re functionally almost identical to squirrel-cage motors, which we studied in the previous unit, and the underlying concepts help students understand the rest of the unit.
Quiz question: List one advantage and one disadvantage of wound-rotor induction motors compared to squirrel-cage motors.
Danger: a student could get this wrong if they don’t understand wound-rotor or squirrel-cage motors. But the question is simple enough that it’s pretty clear which one is the problem. Also, I have a record of the previous unit on squirrel-cage motors; both the student and I can look back at that to find out if their problem is there.
L3 Example
Synchronous, split-phase, and universal motors require a solid understanding of power factor, reflected load, and various ideas about magnetism (which the students haven’t seen since last year, and never in this context) so that puts them at level-3.
Quiz question: Synchronous motors can be used to correct power factor. Explain in 1-2 sentences how this is possible.
L4 Example
The level-4 skill in this unit is to evaluate a type of motor for a given application.
Quiz questions: “Recommend a motor for [scenario]. Explain why.” Or “you need to replace a 3-phase AC motor. Give 3 questions you should ask to help you select the best type. Explain why.”
Why this is an improvement over last year
Last year I would have put only the level-4 problem on a test. The solutions were either excellent or incoherent. I couldn’t help people get better, and they couldn’t help themselves.
Level 5 Questions
You’ll notice that there are no level 5 skills on the skill sheet, even though the unit is graded out of 5. Level 5 is what others might call “Mastery,” where Level 4 might be called “Proficiency.” I teach up to Level 4, and that’s an 80%. A Level 5 question is the name I give to questions that are not exercises but actually problems for most of the class. There are a number of ways to get a 5/5. All of them include both synthesis and a context that was not directly taught in class. So the main difference between L4 and L5 isn’t synthesis; it’s problem-solving.
I occasionally put level-5 questions on quizzes; but not every quiz. I might do it to introduce a new unit, or as a way of touching on some material that otherwise we won’t have time for. Other ways to earn a level 5: research a topic I haven’t taught and present it to me, or to the class. Build something. Fix something. I prefer these to quiz questions; they’re better experience. So I put examples of project topics on the skill sheet. I also encourage students to propose their own topics. Whether they use my topics or theirs, they have to decide what exactly the question is, how they will find the answer, and how they will demonstrate their skill. We’ve had a ton of fun with this. I’ve sometimes put questions on quizzes that, if no one solved them, could be taken into the shop and worked on at your leisure.
I wrote lots in this post about level-5 questions that are independent projects, not quiz questions. But I didn’t give any examples of level-5 questions that are on quizzes, so here are a few.
This is a reduced-voltage manual starter on a DC shunt motor. If I gave this question now, it would be trivial, because we’ve done a whole unit on starters. But it was on a the second quiz of the semester, when the students had barely wrapped their heads around DC motors. It’s a conceptually tough question because the style of drafting is unfamiliar to my students, there’s an electromagnet sealing-in the switch that doesn’t make sense unless you’re thinking ahead to safety hazards caused by power failures, and we hadn’t discussed the idea that there was even such a thing as a reduced-voltage starter. But we had discussed the problem of high current draw on startup, and the loading effect that it causes, and the dangers of sudden-startups of machinery that wasn’t properly de-energized. Those are the problems that this device is intended to solve. One student got it.
Here’s one that no one solved, but someone built later in the shop.
Draw a circuit, with a square-wave power supply, where the capacitor charges up almost instantly and discharges over the course of 17 ms.
You may use any kind of component, but no human intervention is allowed (i.e., you can’t push a button or pull out a component or otherwise interfere with the circuit). You do not need to use standard component values.
This requires time-constant switching, which means combining a diode and a capacitor. They had just learned capacitors that week in one course, and diodes the previous week in a second course. The knowledge was pretty fresh, so they weren’t really ready to use it in a flexible way yet. But the diode unit was all about time-constant switching, and it’s a hard concept to get used to, so this question got them thinking about it from another angle.
Other examples: find total impedance in a parallel circuit, when all we’ve studied so far is series circuits. If they followed the rules for parallel resistance that we studied last year, it will work out; but they had just learned vectors, many of them for the first time, so most people added the vectors (instead of adding the inverses and inverting). Or, find total impedance of a resistor-capacitor-inductor circuit, when all we’ve studied is resistors and capacitors. Amazingly, most of the class got that one. I was really impressed. Again, it’s a question where the conclusion follows logically from tools that the students already have; but they might have to hold the tool by the blade and whack the problem with what they think is the handle.
A few people have asked about implementing the “2-copy quiz,” so I thought I would write a bit about what I’m doing, what’s going well so far, and what I realize in hindsight I should have done differently.
Also, I want to say thanks and welcome to the new readers who’ve joined since that post was “Freshly-Pressed.” I’m delighted that you’ve decided to stay. Don’t hesitate to comment on the older items if you are interested — none of these conversations are finished, by a long shot.
Backstory of the 2-Copy Quiz
I got intrigued by the idea of immediate feedback. It’s easy with after-class make-up quizzes, and I was trying to figure out how to do it with in-class quizzes where a large group of people was likely to finish all at once.
1. I could grade the quizzes and hand them back the next day
Too late — students have already forgotten why they wrote reactance when they should have thought about resistance. Also, since the paper’s already graded, they know whether everything’s right or wrong. It takes the question away.
2. I could collect their work on one piece of paper, and they would still have the sheet of questions while we discuss the answers
Better, but still not what I want. They will have forgotten the details of what they wrote and that’s where the devil is. If I present the correct answers in a “clear, well illustrated way, students believe they are learning but they do not engage … on a deep enough level to realize that what was is presented differs from their prior knowledge.” This is a quote from a video about superficial learning made by Derek Muller, of Veritasium science vlog fame. Derek goes on to say that those misconceptions can be cleared up by “presenting students’ misconceptions alongside the scientific concepts.” It was the alongside part I wanted. It’s not until their thoughts and their actions are suddenly brought into focus at the same time that they realize there is a contradiction.
3. I could collect their papers, run to the staff room, photocopy them, and come back to review the answers.
And while I was gone, they squeezed all the burning curiosity out of their questions among themselves. Which is what they normally do in the hallway.
So the conclusion followed: we needed two copies of the quiz. One for me to grade later, one for them to keep while we reviewed the answers right away. One thing I like about this method is that it doesn’t interrupt the learning. It actually removes an interruption that would normally happen (students having to walk out into the hall to talk about the test). By inviting the conversation into the classroom, I can be a part of it if that’s helpful, or I can organize the students into groups and get out of the way.
Goal: for students to assess the goodness of their answer
We often met this goal. Using class time to discuss “rightness” directs their point-chasing energy toward the good judgement I want them to develop (would this be considered educational judo?). If your students are like mine, they will stop at nothing to find out if they “got the right answer.” Sometimes this makes me tired, what with the assumption that there’s a single right answer, and the other assumption that rightness is all that counts. But then I realized that motivation is motivation, and I could probably teach them to jump through flaming hoops or walk on a bed of nails if I put those things between a student who’s just written a test and the “right answers.”
So I put some self-assessment in the way instead. Their desire to “get the right answer” extends to their self-assessment, of course, but the conversations became more nuanced throughout the term. At first there was a lot of “will you accept this answer” and “will you accept that answer.” I tried to help them make inferences about whether an answer is good enough. I also opened myself up to changing my definition of the right answer if they could substantiate their arguments for an alternate perspective. Hell, alternate perspectives and substantiating their thinking are more important than whatever was on the quiz. Later on in the term, I started hearing things like, “No, I don’t think this answer is good enough, it’s a true statement but it doesn’t answer the question,” or “I think this is too vague to be considered proof of this skill.” They’d rather say it before I say it. Which means I have to be really careful what language I use during this conversation. They will repeat it.
Logistics
I expect the students to write feedback to themselves on their quiz paper. It can be praise or constructive criticism, but there has to be something for each question. They see the value of this later when they’re studying to reassess, but it’s a hard sell at first, and I realized after a few weeks that my students actually had no idea how to do it. For a while, I collected their worksheets at the end of class to read and write back to them. But I don’t pass back the answer sheets that I correct. If they know that I’m going to give the answer and some feedback, it takes the responsibility off of them to do it for themselves.
What worked well
- It’s easy and cheap. Just print off 2 quiz papers for every student, and have them fill out both.
- It’s flexible. You could have them make two full copies of their work. You could ask them to make a full copy for themselves and an answer copy for the teacher (my tactic at the moment). You could ask them to make an answer copy for the teacher, and some rough notes for themselves so they can remind themselves of their thinking (what my students actually do).
- In keeping with the idea of going with the flow of the learning, I let the class direct the questioning. There’s no reason we have to review the first question first. Often there’s one question that everyone is dying to know the answer to, so we talk about that one.
- I get an instant archive of student work. Good for preparing my lesson plans next year, reconstituting my gradebook when a computer crashes, turning over the course to another instructor, submitting documentation to accrediting agencies, etc. etc.
What didn’t always work well
- It’s time-consuming to have to copy things to another page. For numerical answers, it’s pretty easy to copy the final answer, but then you can’t see their work. For short-answer/essay questions, it’s going to get seriously annoying for students to copy them in full to another page (I make them do it anyway). Multiple-choice is pretty painless, but it’s a pain to feel limited to one kind of question.
- Students don’t always see the value of having their own copy, so they fill out my copy and leave theirs blank. See Backstory #2 above.
- Students don’t always see the value of showing their work, so they fill out two copies with nothing but answers. See Backstory #2 above.
- Students don’t always see the value of assessing their work at all. The teacher is going to decide the final grade, and the teacher might disagree with their self-assessment, so why not just wait and let “the experts” make the judgement call.
- Students don’t always see the value of writing feedback to themselves.
- Students sometimes have no idea how to write feedback to themselves.
I struggled with the attitude of “wait for the teacher to decide if it’s good enough.” I should have made it clearer that improving their ability to evaluate their answers was the point, not a side-effect. I deliberately held off updating my online gradebook, so that they had to depend on themselves to track their skills (just got my student evals back today and my “poor tracking” of their grades is the #1 complaint). It’s said best by Shawn Cornally from Think Thank Thunk: “I am not your grade’s babysitter.” In fact I sometimes wondered if I should stop using the online gradebook altogether. Yes, sometimes I disagree with their self-assessment; that’s why it’s important for them to take part in the group discussion after the quiz. That’s where I discuss what I’m looking for in an answer and help them figure out if they’ve provided it. This is hard on them, and makes them feel insecure, for lots of reasons, and I need to keep thinking about it.
One reason is that writing feedback is something I realized (a bit late) that I had to teach. I did this in a hurry and without the scaffolding it deserved. Kelly O’Shea of Physics! Blog! broke it down for me:
How often do you think they’ve practiced the skill of consciously figuring out what caused them to make a mistake? How often do we just say, “That’s okay, you’ll get it next time.” instead of helping them pick out what went wrong? My guess is that they might not even know how to do it.
What’s Next
- I’m still not sure how to teach them to create feedback for themselves, but it goes to the top of the pile of things to introduce in September next year, not February.
- I’m toying with the idea that the students should keep an online gradebook updated. Then I could check up on their scoring (and leave them some feedback about it), instead of them checking up on my scoring, and being annoyed that it’s not posted yet. Not sure logistically how to do this. (Edit: ActiveGrade is already working on this)
- A portable scanner. For $300 I could solve Didn’t-Work #1, 2, and 3. Just scan their quiz papers as they finish. Makes it extra-easy for me to annotate the electronic copy and maybe make a screencast for a particular student, if warranted. Saves trees, too.
Update, July 29, 2011: If you already own a smartphone, the portable scanner is free, and it’s called CamScanner.
In the new grading system, the skills list for each unit ends at 4/5. Any student who wants a 5/5 must apply their skills to a novel context (not explicitly taught in class), choose their own problem-solving strategy, and combine ideas from at least two units.
I put a L5 question on a quiz at least once per unit as a way to assess problem-solving and synthesis. They’re doing that quite well. But they have had a host of unexpectedly positive benefits for the class. Top 10 reasons I love the “L5 question”:
1. I can put anything on a quiz. Since L5 questions by definition include synthesis, the students understand that anything is fair game: skills we’ve learned in other units, in co-requisite courses, in pre-requisite courses. So L5 questions free me from the compartmentalization that the skills-based grading scheme might otherwise enforce.
2. Students use it to practise “trying something” even though they don’t know the right answer. A L5 question on a quiz feels like a bonus question, so there’s less stigma attached to getting them wrong. Unlike other levels, your score on L5 questions can not go down. So, you can write any wacky thing that goes through your head, and there’s no penalty. I give 30 minutes for quizzes, and deliberately choose the questions so that even the slower students finish in about 25 minutes. That means there’s nothing left to do except think about the L5 question. This helps students practice creating representations, choosing symbols, and thinking about unfamiliar things in a low-stakes environment. (Who would have thought that a quiz would become a low-stakes environment??)
3. It’s great for introducing a new unit. Since every unit builds on the previous one, a student who has mastered the tricky questions from Unit 1 probably has all the skills to do the easy questions from Unit 2, if they can figure out how to apply them. I throw these on the quiz and one of two things happen: some students get them right, in which case they’re primed to make sense of the new unit; some students get them wrong, in which case I’m introducing Unit 2 at the exact moment when they’re dying of curiosity to know how it works.
4. It doesn’t have to go on a quiz. A L5 question can be a research project or an invention or a presentation to the class or an interpretive dance or a graphic novel, if it meets the synthesis/problem-solving criteria.
5. It’s a great response to tangential questions in class (“Interesting, I’m not sure of the answer… How could you find out? Sounds like a great L5 question.”)
6. It’s a good way bring up neat topics that don’t quite fit in the curriculum. I make a list of some of them at the bottom of each skill sheet. Any student who is curious can learn more about one of those topics. It’s then up to them to propose both a question and the assessment of its answer.
7. It’s an instant way to incorporate fix-it projects, service-learning opportunities, and inter-program collaborations that cross my desk every semester.
- The head chef from the culinary program went to Europe and fried the power supply of his fancy sous-vide cooker, so a student traced the problem, selected and ordered a replacement for the obsolete part, and put it back together.
- A student in Disability Services needs help building a rehabilitative technology toy for developing fine-motor skills, so a team of four 1st-year students are working together to help him out.
- The Academic Chair’s Roomba isn’t finding its dock properly anymore. I ask for volunteers, and voila — Level 5 question.
I don’t need the thing to work at the end; I expect the student to have developed a sensible problem-solving strategy and synthesized their skills. (Proving to me that it shouldn’t be fixed — for economic or other reasons — might be perfectly legitimate. It depends on whether you have enough evidence to convince me).
8. The students are free to propose a problem. About anything. As long as it requires them to synthesize and problem-solve. They can bring in something broken from home and work on it. They can decide to experiment with something they read about in a trade journal or diy magazine.
- The other day a student completed his assigned exercise early (using an inductor to light a 120V lamp using a 12V supply). So he went out to the parking lot, removed the relay from his trunk latch, wired it into the lamp circuit as a crude boost chopper, and used a signal generator to energize the relay fast enough to make it look like the light was on continuously.
- Two students figured out how to test a transistor before I taught the unit — so they asked for permission to destroy one to test their algorithm. I agreed, on the condition that they teach their methods to the class.
The assessments don’t have to be involved or time-consuming; they just have to deepen a student’s thinking. About 3/4 of my students have at least one L5 question.
9. They are a built-in back-up plan for students who finish their work in class early.
10. The students get stoked about them.
In the “why didn’t I think of this before” category: my students now grade their own quizzes. No, they don’t get to give themselves whatever mark they want. Here’s how it goes.
I used to learn a ton from going over their tests with a fine toothed comb, trying to figure out what they were having trouble with. It finally dawned on me that I didn’t need that learning opportunity nearly as much as my students did. So my students now do quizzes like this.
- Write quiz.
- Copy all answers to the provided answer sheet.
- Hand in answer sheet to teacher. Keep question sheet with all your work on it!
- Twiddle thumbs for a few minutes waiting for everyone to finish. Wish you could ask someone “what did you get for #5?”
- All papers are in. Burst at the seams and ask “what’s the answer to 5!?”
- Participate in class or small-group discussion of questions and answers. Compare your problem-solving approach to others’. Figure out what you did right. Figure out what you did wrong. Make notes about how to do #5 differently, since now that you’ve found your misunderstandings or found new ways to tackle the question, you’re already planning to reassess next Wednesday.
- Put checkmarks on skill sheet.
I love this because it allows the students to go over the problems immediately after the test, during those 7 minutes of burning curiosity, yet still have their own test paper in front of them. They remember what they did and why they did it (by tomorrow it’ll be gone into the ether of “oh, just a careless mistake” or “I understand it now”). The downside is that they have to copy their answers from the question sheet to the answer sheet, which can take time. I collect the answer sheets before the discussion/review, grade them on a complete/incomplete basis, and update my gradebook.
Because each person needs to know whether their answer meets the standard, they share. This gets into a great discussion of the many possible right answers. If I hand the tests back already graded, there’s no incentive to share. Downside: you must risk speaking up and exposing a possibly flawed answer. Upside: everyone else is doing it. Sometimes the top students get things wrong, and take a good-natured drubbing, and it becomes more clear that smartness isn’t a magical quality that enables you to skip the “learning” phase of the learning.
Every once in a while, I collect the work sheets after we’re done reviewing. It lets me see how they’re doing with grading their tests and writing feedback to themselves. It also lets me have a look at the common misconceptions and confusions (although mostly I collect in-class work for that kind of intel).
Unexpected discovery: their negative self-talk shows up in their corrections. I was blown away by the brutality of the things they were writing to themselves. (“Stop being a moron!” “Stupid, stupid, stupid!”). Collecting the work sheets gives me a chance to write back to them, try to interrupt negative self-talk, and do some coaching about self-assessment.
When they request reassessments, the web form I use has a spot for “What have you learned about this skill that you didn’t know before.” The answers there are almost as enlightening, and have evolved from “I learned that I am stupid” to “I learned that capacitors in parallel do not get added up if they have been converted to reactance.” All of these things become the beginnings of conversations.
It’s the end of my third semester teaching. It’s the first semester that I’ve used a radically different grading policy. I’ve got a pile of stuff in the “drafts” folder that I’d like to clear out; so, this is the first in a series of Semester In Review posts. Since the major change that I made was about assessment, it makes sense to start there.
Test Schedule Before
3-4 major tests, each 2 hours long, stressful, difficult, full of challenging synthesis problems. Answers were rarely numbers; more often they were recommendations, arguments in favour/against a position, etc.
Result:
Students who entered the program with strong problem-solving skills were able to display those skills. The rest found that the best strategy was to throw as much stuff at the paper as they could think of, and hope that something stuck. The tests were awful to grade. It was often unclear whether students actually believed their own train of thought. If they were on the wrong track, it was hard to tell which part they didn’t understand. There were often many defensible answers, and it was easy to award full marks to any one of them; but there was no really fair way to award fractional marks. The students took one look at their mark, shrugged, and filed the test paper in the “forget” pile. If they didn’t have any insight into their problem-solving habits before the test, they had none after. Those who did poorly had no idea what to improve — or that there was any point in improving, since the points couldn’t be gotten back and they couldn’t know that those skills would be needed next semester or maybe next week.
Test Schedule After
Now I give shorter quizzes every week instead of long tests/exams. The quizzes include both “exercises” (applying standard procedures) and “problems” (questions requiring students to synthesize, choose or evaluate procedures, etc.). For any synthesis skill, I also assess the components individually. There are no fractional grades; you’ve either demonstrated the skill or not. I store each skill’s status (complete or incomplete) as a 1 or 0 in my gradebook. If you miss a question, you can ask to retry it any Wednesday afternoon. As many Wednesdays as you want (I reserve the right to redirect students who are reassessing without making progress).
Result:
I have a better grip on what each student does and doesn’t know. They have a better grip on their own understanding. Grading is easier and faster. Students actually go back and fix up their misunderstandings. This makes the rest of the courses infinitely less painful — for all of us.
Then there are the Wednesday reassessments. Mostly they’re just a quiz buffet. As each student finishes their quiz(zes), they sit down with me to find out how they did. We review the quiz together, which has all the above-mentioned benefits about immediate feedback. But there’s another hidden gem here: this is an instant tutoring session — on exactly what they need to learn. Some students who would not otherwise book a tutoring session with me end up sitting down and having conversations about all kinds of things.
Quizzes have a new lease on life. If students don’t necessarily salivate at the thought of another one, at least quizzes don’t seem like “the enemy” any more. A class discussion recently concluded “the more quizzes, the better.” Yes, there’s some point-chasing and question-memorizing tucked in there. But there’s some willingness, maybe even eagerness, to discover progress. It’s a start.
Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in /hermes/web07/b2567/moo.scornally/SBGBook/getScoresTable.php on line 22
It’s the end of the semester. My students are starting to think about their final grades. Some decide it’s time to scramble and get those numbers up, for various reasons that I may or may not agree with. In the past, scrambling to get the numbers up had the following top-two effects:
- asking for “extra work” that would compensate for missed deadlines
- giving up — they feel so far behind that there’s no point trying to do well on the rest of the semester either
The new grading scheme means that the top-two strategies are no longer even in the top ten. The end-of semester scramble now results in students, get this, going back over their semester’s work with a fine-toothed comb to improve their grasp of underlying concepts.
That’s because underlying concepts are “Level 2” skills, and they can’t get credit for a unit’s higher level skills until lower levels are complete.
Consequence: struggling students are uncovering their misconceptions and misunderstandings and practising their basic skills. They’re also injecting new insight into group discussions; students who “got” those underlying concepts way back in January may not have thought about them explicitly since then. Since the struggling students are talking about the basics that they’re working on, the rest of the class is seeing the basics again, through their new “Semester 2” eyes, and having important revelations. (One of the top students in the class just realized today that everything is always in parallel with an open circuit).
I might stop hating the end of the semester.
But not until after I finish making up 20 reassessments for tomorrow.
