You are currently browsing the category archive for the ‘Assessment’ category.
I wrote recently about creating a rubric to help students analyze their mistakes. Here are some examples of what students wrote — a big improvement over “I get it now” and “It was just a stupid mistake.”
The challenge now will be helping them get in the habit of doing this consistently. I’m thinking of requiring this on reassessment applications. The downside would be a lot more applications being returned for a second draft, since most students don’t seem able to do this kind of analysis in a single draft.
Understand What’s Strong
-
“I thought it was a parallel circuit, and my answer would have been right if that was true.”
-
“I got this question wrong but I used the idea from the model that more resistance causes less current and less current causes less power to be dissipated by the light bulbs.”
-
“The process of elimination was a good choice to eliminate circuits that didn’t work.”
-
“A good thing about my answer is that I was thinking if the circuit was in series, the current would be the same throughout the circuit.”
Diagnose What’s Wrong
-
“The line between two components makes this circuit look like a parallel circuit.”
-
“What I don’t know is, why don’t electrons take the shorter way to the most positive side of the circuit?”
-
“I made the mistake that removing parallel branches would increase the remaining branches’ voltage.”
-
“What I didn’t realize was that in circuit 2, C is the only element in the circuit so the voltage across the light bulb will be the battery voltage, just like light bulb A.”
-
“I looked at the current in the circuit as if the resistor would decrease the current from that point on.”
-
“I think I was thinking of the A bulb as being able to move along the wire and then it would be in parallel too.”
-
“What I missed was that this circuit is a series-parallel with the B bulb in parallel with a wire, effectively shorting it out.”
-
“What I did not realize at first about Circuit C was that it was a complete circuit because the base of the light bulb is in fact metal.”
-
“I thought there would need to be a wire from the centre of the bulb to be a complete circuit.”
-
“I wasn’t recognizing that in Branch 2, each electron only goes through one resistor or the other. In Branch 1, electrons must flow through each resistor.”
-
“I was comparing the resistance of the wire and not realizing the amount of distance electrons flowed doesn’t matter because wire has such low resistance either way.”
-
“My problem was I wasn’t seeing myself as the electrons passing through the circuit from negative to positive.”
Improve
-
“In this circuit, lightbulb B is shorted so now all the voltage is across light bulb A.”
-
“When there is an increase in resistance, and as long as the voltage stays constant, the current flowing through the entire circuit decreases.”
-
“After looking into the answer, I can see that the electrons can make their way from the bottom of the battery to the middle of the bulb, then through the filament, and back to the battery, because of metal conducting electrons.”
-
“To improve my answer, I could explain why they are in parallel, and also why the other circuits are not parallel.”
-
“I can generalize this by saying in series circuits, the current will stay the same, but in parallel circuits, the current may differ.”
-
“From our model, less resistance causes more current to flow. This is a general idea that will work for all circuits.”
This year I’ve really struggled to get conversation going in class. I needed some new ways to kick-start the questioning, counter-example-ing, restating, and exploring implications that fuel inquiry-based science. I suspected students were silent because they were afraid that their peers and/or I would find out what they didn’t know. I needed a more anonymous way for them to ask questions and offer up ideas.
About that time, I read Mark Guzdial’s post about Peer Instruction in Computer Science. While exploring the resources he recommends, I found this compelling and very short PI teacher cheat sheet. I was already curious because Andy Rundquist and Joss Ives were blogging about interesting ways to use PI, even with small groups. I hadn’t looked into it because, until this year, I’ve never been so unsuccessful in fostering discussion.
The cheat-sheet’s clarity and my desperation to increase in-class participation made me think about it differently. I realized I could adapt some of the techniques, and it worked — I’ve had a several-hundred-percent increase in students asking questions, proposing ideas, and taking part in scientific discourse among themselves. Caveat: what I’m doing does not follow the research model proposed by PI’s proponents. It just steals some of their most-easily adopted ideas.
What is Peer Instruction (PI)?
If you’re not familiar with it, the basic idea is that students get the “lecture” before class (via readings, screencasts, etc), then spend class time voting on questions, discussing in small groups, and voting again as their understanding changes. Wikipedia has a reasonably clear and concise entry on PI, explaining the relationship between Peer Instruction, the “flipped classroom”, and Just-In-Time Teaching.
Why It’s Not Exactly PI
My home-made voting flashcards
- I don’t have clickers, and don’t have any desire for them. If needed, I use home-made voting cards instead. Andy explains how effective that can be.
- I prefer to use open-ended problems, sometimes even problems the students can’t solve with their current knowledge, rather than multiple-choice questions. That’s partly because I don’t have time to craft good-quality MC items, partly because I want to make full use of the freedom I have to follow students’ noses about what questions and potential answers are worth investigating.
- Update (Feb 19): I almost forgot to mention, my classroom is not flipped. In other words, I don’t rely on before-class readings, screencasts, etc.
What About It is PI-Like?
- I start with a question for students to tackle individually. Instead of multiple-choice, it could be a circuit to analyze, or I might ask them to propose a possible cause for a phenomenon we’ve observed.
- I give a limited amount of time for this (maybe 2-3 minutes), and will cut it even shorter if 80% of students finish before the maximum time.
- I monitor the answers students come up with individually. Sometimes I ask for a vote using the flashcards. Other times I just circulate and look at their papers.
- I don’t discuss the answers at that point. I give them a consistent prompt: “In a moment, not right now but in a moment, you’re going to discuss in groups of 4. Come to agreement on whatever you can, and formulate questions about whatever you can’t agree on. You have X minutes. Go.”
- I circulate and listen to conversations, so I can prepare for the kinds of group discussion, direct instruction, or extension questions that might be helpful.
- When we’re 30 seconds from the end, or when the conversation starts to die down, I announce “30 more seconds to agree or come up with questions.”
- Then, I ask each group to report back. Usually I collect all the questions first, so that Group B doesn’t feel silenced if their question is answered by Group A’s consensus. Occasionally I ask for a flashcard vote at this point; more often, collect answers from each group verbally. I write them on the board — roughly fulfilling the function of “showing the graph” of the clicker results.
- If the answers are consistent across the group and nothing needs to be clarified, I might move on to an extension question. If something does need clarification, I might do some direct instruction. Either way, I encourage students to engage with the whole group at this point.
Then we’re ready to move on — maybe with another round, maybe with an extension question (the cheat-sheet gives some good multi-purpose prompts, like “What question would make Alternate Answer correct?”). I’m also a fan of “why would a reasonable person give Alternate Answer?”
Why I Like It
It doesn’t require a ton of preparation. I usually plan the questions I’ll use (sometimes based on their pre-class reading which, in my world, actually in-class reading…). But, anytime during class that I feel like throwing a question out to the group, I can do this off the cuff if I need to.
During the group discussion phase (Step 4), questions and ideas start flowing and scientific discourse flourishes. Right in this moment, they’re dying to know what their neighbour got, and enjoy trying to convince each other. I don’t think I buy the idea that these techniques help because students learn better from each other — frankly, they’re at least as likely to pseudoteach each other as I am. I suspect that the benefit comes not so much from what they hear from others but from what they formulate for themselves. I wish students felt comfortable calling that stuff out in a whole group discussion (with 17 of us in the room, it can be done), but they don’t. So. I go with what works.
No one outside the small group has to know who asked which questions. The complete anonymity of clickers isn’t preserved, but that doesn’t seem to be a problem so far.
Notes For Improvement
There are some prompts on the cheat sheet that I could be using a lot more often — especially replacing “What questions do you have” or “What did you agree on” with “What did you group talk about,” or “If your group changed its mind, what did you discuss?”
There’s also a helpful “Things Not To Do (that seemed like a good idea at the time)” page that includes my favourite blooper — continuing to talk about the problem after I’ve posed the question.
If I was to add something to the “What Not To Do” list, it would be “Shifting/pacing while asking the question and immediately afterwards.” I really need to practice holding still while giving students a task, and then continuing to hold still until they start the task. My pacing distracts them and slows down how quickly they shift attention to their task; and if I start wandering the room immediately, it creates the impression that they don’t have to start working until I get near enough to see their paper.
I expect students to correct their quizzes and “write feedback to themselves” when they apply for reassessment. The content that I get varies widely, and most of it is not very helpful, along the lines of
I used the wrong formula
I forgot that V = IR
It was a stupid mistake, I get it now.
I was inspired by Joss Ives’ post on quiz reflection assignments to get specific about what I was looking for. This all stems from a conversation I had with Kelly O’Shea about two years ago, back when I had launched myself into standards-based/project/flipped/inquiry/Socratic/mindset/critical thinking/whatnot all at once and unprepared, that has been poking its sharp edges into my brain ever since:
Me: Sometimes I press them to be specific about what they learned or which careless mistake they need to guard against in the future. It’s clear that many find this humiliating, some kind of ingenious psychological punishment for having made a mistake. Admitting that they learned something means admitting they didn’t know it all along, and that embarrasses them. Does that mean they’re ashamed of learning?
Kelly: How often do you think they’ve practiced the skill of consciously figuring out what caused them to make a mistake? How often do we just say, “That’s okay, you’ll get it next time.” instead of helping them pick out what went wrong? My guess is that they might not even know how to do it.
Me: *stunned silence*
So this year I developed this.
Phases of Feedback
- Understand what you did well
- Diagnose why you had trouble
- Improve
Steps 1 and 3 can be used even for answers that were accepted as “correct.”
This has yielded lots of interesting insight, as well as some interesting pushback. Plus, it gave me an opportunity to help my students understand what exactly “generalize” mean. In a future post I’ll try to gather up some examples. Overall, it’s helped me communicate what I expect, and has helped students develop more insight into their thinking as well as the physics involved.
Sometimes I need to have all the students in my class improve their speed or accuracy in a particular technique. Sometimes I just need everyone to do a few practice problems for an old topic so I can see where I should start. But I don’t have time to make (or find) the questions, and I definitely don’t have time to go through them with a fine-toothed comb.
One approach I use is to have students individually generate and grade their own problems. They turn in the whole, graded, thing and I write back with narrative feedback. I get what I need (formative assessment data) and they get what they need — procedural practice, pointers from me, and some practice with self-assessment.
Note: this only works for problems that can be found in the back of a textbook, complete with answers in the appendix.
Here’s the handout I use.
What I Get Out of It
The most useful thing I get out of this is the “hard” question — the one they are unable to solve. They are not asked to complete it: they are asked to articulate what makes that question difficult or confusing.
Important Principles
- Students choose questions that are easy, medium, and hard for them. This means they must learn to anticipate the difficulty level of a question before attempting it.
- If they get a question wrong, they must either troubleshoot it or solve a different one.
- They turn in their questions clearly marked right or wrong.
Advantages
- I don’t have to grade it — just read it and make comments
- The students get to practice looking at things they don’t fully understand and articulating a question about it
- I get to find out what they know and what they (think they) don’t know.
- Students can work together by sharing their strategies, but not by sharing their numbers, since everyone ends up choosing different problems.
- It makes my expectations explicit about how they should do practice questions in general: with the book closed, page number and question number clearly marked, with the schematics copied onto the paper (“Even if there’s no schematic in the book?!” they ask incredulously — clearly the point of writing down the question is just to learn to be a good scribe, not to improve future search times), etc.
Lessons Learned
I give this assignment during class, or at least get it started during class, to reduce copying. Once students have chosen and started their questions, they’re unlikely to want to change them.
My students use the same assessment rubric for practically every new source of information we encounter, whether it’s something they read in a book, data they collected, or information I present directly. It asks them to summarize, relate to their experience, ask questions, explain what the author claims is the cause, and give support using existing ideas from the model. The current version looks like this (click through to zoom or download):
Assessment for Learning
There are two goals:
- to assess the author’s reasoning, and help us decide whether to accept their proposal
- to assess one’s own understanding
If you can’t fill it in, you probably didn’t understand it. Maybe you weren’t reading carefully, maybe it’s so poorly reasoned or written that it’s not actually understandable, or maybe you don’t have the background knowledge to digest it. All of these conditions are important to flag, and this tool helps us do that.
The title says “Rubric for Assessing Reasoning,” but we just call them “feedbacks.”
Recently, there have been a spate of feedbacks turned in with the cause and/or the “support from the model” section left blank or filled with vague truisms (“this is supported by lots of ideas about atoms,” or “I’m looking forward to learning more about what causes this.”)
I knew the students could do better — all of them have written strong statements about cause in the past (in chains of cause and effect 2-5 steps long). I also allow students to write a question about cause, instead of a statement, if they can’t tell what the cause is, or if they think the author hasn’t included it.
So today, after I presented my second draft of some information about RMS measurements, I showed some typical examples of causal statements and supporting ideas. I asked students to rate them according to their significance to the question at hand, then had some small group discussions. I was interested (and occasionally surprised) by their criteria for what makes a good statement of cause, and what makes a good supporting idea. Here’s the handout I used to scaffold the discussions.
The students’ results:
A statement of cause should …
- Be relevant to the question
- Help us understand the question or the answer
- Not leave questions unanswered
- Give lots of info
- Relate to the model
- Explain what physically makes something happen or asks a question that would help you understand the physical cause
- Help you distinguish between similar things (like the difference between Vpk, Vpp, Vrms)
- Not beg the question (not state the same thing twice using different words)
- Be concrete
- Make the new ideas easier to accept
- Use definitions
Well, I was looking for an excuse to talk about definitions — I think this is it!
Supporting ideas from the model should…
- Help clarify how the electrons work
- Help answer or clarify the question
- Directly involve information to help relate ideas
- Help us see what is going on
- Give us reasoning so we can in turn have an explanation
- Clarify misunderstandings
- Allow you to generalize
- Support the cause, specifically.
- Be specific to the topic, not broad (like, “atoms are made of protons, electrons, and neutrons.”)
- Not use a formula
- It helps if you understand what’s going on, it makes it easier to find connections
The Last World
Which ones would you emphasize? What would you add?
My standard (informal) course feedback form asks,
- What do you like or dislike about the grading system?
- How does the grading system affect your learning?
- What do you love about this course?
- What do you hate about this course?
- What would you change about this course?
The 2nd-year courses are less science and more engineering, so my approach is less inquiry and more project-based. In particular, in the course they’re evaluating, there’s an independent project where students must define their project, set their own deadlines, set their own evaluation scheme, then grade themselves. It’s worth a quarter of their grade. I reserve the right to veto a mark, but I’ve never done it. Here’s a sample of the feedback I got from 2nd year students last week.
1. Grading system
- Love reassessment (2)
- Feel dependent on ActiveGrade
- Need quicker way of knowing when a test is corrected
- Love the independent project
- Make reassessment deadline start when grade is updated?
- Ability to do skills on your own time. But they can also pile up.
- Clearly shows what you need to know
- Retests help a lot with understanding because you know what you need to improve on
- Showing improvement helps solidify thoughts
2. Effects of Grading System
- Reassessing forces you to gain understanding instead of “I failed that let’s move on”
- I can thoroughly explain certain circuits from my head, I could not do that before.
- Helpful — I can choose to not finish a lab if I do not understand it fully, then ask questions and come back to it
- I knew nothing about electronics before this course but skill based learning has really helped me understand many topics
3. Love
- Reassessing forces you to gain understanding instead of “I failed that let’s move on”
- Lab work — hands on feel
- Making things work and understanding what they do
- Freedom
- Retests, doing something more than once makes remembering it easier.
4. Hate
- Lack of info on notch filter (2)
- Lack of time
- Nothing
5. Change
- Hands on – when you don’t quite understand something, lab work refines understanding
- It’s a pretty refined, good system. Once you know something, it sticks with you.
- More time to learn. 3 years?
- Reassessment deadlines
Series circuits are one of the foundational concepts in electrical work, and one of the first things students build/think about/get assessed on in their first months at school. My definition of two series components:
- Two components are in series if all the current in one flows into the second, and all the current in the second comes from the first
Things I have heard about series components:
- Components are in series if they’re in a square shape
- Components are in series if all the current in one flows into the second
- Components are in series if they’re both connected to the power supply
- Components are in series if they’re aligned in a straight line
In the first year of the program, we spend a lot of time refining our ideas about which circuits have which behaviours. We refine and revise and throw out ideas. By the end of December we should have something fairly strong.
Last week, I had a second-year student tell me he knew that two components were in series because of reason #3 above. I’m struggling to make sense of this, and the accountability of teaching in a trade school hangs over my head like the razor-edged pendulum in the pit. In May, some of these students will be working on large-scale industrial robots. These things weigh tons, carry blades and torches, and can maim or kill people in an instant. Electronics is not an apprenticeable trade. Grads will not carry tools for a journeyman for three years — they get put right to work. Also, electronics is not a construction trade — it is a repair trade. That means that work is almost always done under pressure of short timelines and lost money — the electronics tech doesn’t get called out until something is broken.
I have two years to make sure they are ready to at least begin their industry-specific training. It’s not good enough for them to sometimes make sense of things — they need nail these foundational concepts every time in order to to use the training the employer provides and make good judgement calls on the job. Please, no comments about how education is about broadening the mind and this student is learning lots of other valuable skills. While that’s true, it’s not currently the point. When that electronics tech does some repairs on the heart-rate monitor keeping tabs on your unborn child, you are not going to be any more interested in the tech’s broad mind than I am.
What does it mean if a student can spend 4 months in DC circuits, not fully integrate the concept of series components, pass the course, and 8 months later still have an unstable concept?
Here are all the ideas I can think of at the moment. Don’t panic — I don’t think these are all equally likely.
- Their experience in DC circuits is not doing enough to help them make sense of this idea
- The assessments in DC circuits are not rigourous enough to catch students who are still unsure about this
- This student is incapable of consistently making sense of this idea, and should not have been accepted into the program in the first place
- It’s normal for students to form, unform, and reform their ideas about new concepts. It’s inevitable, and sometimes students will revert to previous ways of thinking even after the fantastic course and the rigourous assessments.
If it’s #1, I’m not sure what to do. I’ve already given over my courses to sense-making, critical thinking, and inquiring. Do they need more class hours, more time outside class hours, or just different kinds of practice? Maybe the practice problems are too consistent, failing to address students’ misconceptions.
If it’s #2, I’m not sure what to do. I feel pretty confident that I’m assessing their reasoning rather than their regurgitating. More assessments might help — not sure where to get the time. A final exam might help. I can’t see my way clear to passing or failing someone on the strength of a final exam, but I’d at least know a bit more about which concepts are still shaky. I’ve sometimes given a review paper in January on the concepts learned in the previous semester, and worked through multiple drafts — I could start doing that again.
If it’s #3, I’m definitely not sure what to do.
If it’s #4, how do I reconcile this with my sense of personal responsibility to not send them out to get injured or injure someone else? I realize I’ve framed this in a fairly dramatic way, and not every student who’s unsure of what a series circuit is will end up harming someone. It’s much more likely that they’ll end up on the job and start to consolidate their knowledge and clear up their misconceptions. However, it’s also likely that they’ll end up on a job where they suddenly realize that they don’t understand the basic things they’re being asked to do. This bodes poorly for the grad’s confidence and enjoyment of their career, the employer’s willingness to hire future grads, and of course the quality of our biomedical equipment, manufacturing equipment, navigational equipment, power generation instrumentation, … . It also bodes poorly for my ability to believe that I am doing a reasonable job.
Thoughts?
How I got my students to read the text before class: have them do their reading during class.
Then, the next day, I can lead a discussion among a group of people who have all tangled with the text.
It’s not transformative educational design, but it’s an improvement, with these advantages:
- It dramatically reduces the amount of time I spend lecturing (a.k.a. reading the students the textbook), so there’s no net gain or loss of class time.
- The students are filling in the standard comprehension constructor that I use for everything — assessing the author’s reasoning on a rubric. That means they know exactly what sense-making I am asking them to engage in, and what the purpose of their reading is.
- When they finish reading, they hand in the assessments to me, I read them, and prepare to answer their questions for next class. That means I’m answering the exact questions they’re wondering about — not the questions they’ve already figured out or haven’t noticed yet.
- Knowing that I will address their questions provides an incentive to actually ask them. It’s not good enough to care what they think if I don’t put it into action in a way that’s actually convincing to my audience.
- Even in a classroom of 20 people, each person gets an individualized pace.
- I am free to walk around answering questions, questioning answers, and supporting those who are struggling.
- We’re using a remarkable technology that allows students to think at their own pace, pause as often/long as they like, rewind and repeat something as many times as they like, and (unlike videos or podcasts) remains intelligible even when skipping forward or going in slow-mo. This amazing technology even detects when your eyes stray from it, and immediately stops sending words to your brain until your attention returns. Its battery life is beyond compare, it boots instantly, weights less than an iPod nano, can be easily annotated (even supports multi-touch), and with the right software, can be converted from visual to auditory mode…
It’s a little bit JITT and a little bit “flipped-classroom” but without the “outside of class” part.
I often give a combination of reading materials: the original textbook source, maybe another tertiary source for comparison — e.g. a Wikipedia excerpt, then my summary and interpretation of the sources, and the inferences that I think follow from the sources. It’s pretty similar to what I would say if I was lecturing. I write the summaries in an informal tone intended to start a conversation. Here’s an example:
And here’s the kind of feedback my students write to me (you’ll see my comments back to them in there too).
Highlights of student feedback:
Noticing connections to earlier learning
When I read about finite bandwidth, it seemed like something I should have already noticed — that amps have a limit to their bandwidth and it’s not infinite
Summarizing
When vout tries to drop, less opposing voltage is fed back to the inverting input, therefore v2 increases and compensates for the decrease in Avol
Noticing confusion or contradiction
What do f2(OL) and Av(OL) stand for?
I’m still not sure what slew-induced distortion is.
I don’t know how to make sense of the f2 = funity/Av(CL). Is f2 the bandwidth?
In [other instructor]‘s course, we built an audio monitor, and we used an op amp. We used a somewhat low frequency (1 KHz), and we still got a gain of 22.2 If I use the equation, the bandwidth would be 45Hz? Does this mean I can only go from 955 Hz to 1045 Hz to get a gain of 22.2?
Asking for greater precision
What is the capacitance of the internal capacitor?
Is this a “flipped classroom”?
One point that stuck with me about many “flipped classroom” conversations is designing the process so that student do the low-cognitive-load activities when they’re home or alone (watching videos, listening to podcasts) and the high-cognitive-load activities when they’re in class, surrounded by supportive peers and an experienced instructor.
This seems like a logical argument. The trouble is that reading technical material is a high-cognitive-load activity for most of my students. Listening to technical material is just as high-demand… with the disadvantage that if I speak it, it will be at the wrong pace for probably everyone. The feedback above is a giant improvement over the results I got two years ago, when second year students who read the textbook would claim to be “confused” by “all of it,” or at best would pick out from the text a few bits of trivia while ignoring the most significant ideas.
The conclusion follows: have them read it in class, where I can support them.
To introduce the incoming students to my grading system, I’ll spend a class explaining, having them practice using a grading sheet, and doing some Q&A. Last year I had them assess me using the “Teacher Skill Sheet,” and I will do that again. It helped students understand the reasoning behind the system.
But I found that students often submitted incomplete applications for reassessment, and I wanted to create a resource they could turn to.
It’s a bit too much to digest, I think… a lot to absorb in one bite. So I’ll introduce the components one by one: how to do a good-quality quiz correction (with inspiration from Joss), how to update the bar graph (their current grade), how to find an appropriate practice problem in the textbook.
But I’m going to put a handout in the front of their skill folders, nonetheless — for future reference. Here’s the draft so far — comments encouraged, especially about how to make it shorter or more student-friendly.
In the same vein as the last post, here’s a breakdown of how we used published sources to build our model of how electricity works.
- I record questions that come up during class. I track them on a mind-map.
- I pull out the list of questions and find the ones that are not measurable using our lab equipment, and relate to the unit we’re working on.
- I post the list at the front of the room and let students write their names next to something that interests them. If I’m feeling stressed out about making sure they’re ready for their impending next courses/entry into the work world, I restrict the pool of questions to the ones I think are most significant. If I’m not feeling stressed out, or the pool of questions aligns closely with our course outcomes, I let them pick whatever they want.
- The students prepare a first draft of a report answering the question. They use a standard template (embedded below). They must use at least two sources, and at least one source must be a professional-quality reference book or textbook.
- I collect the reports, write feedback about their clarity, consistency and causality, then hand back my comments so they can prepare a second draft.
- Students turn in a second draft. If they have blatantly not addressed my concerns, back it goes for another draft. They learn quickly not to do this. I make a packet containing all the second drafts and photocopy the whole thing for each student. (I am so ready for 1:1 computers, it’s not funny.)
- I hand out the packets and the Rubric for Assessing Reasoning that we’ve been using/developing. During that class, each student must write feedback to every other student. (Note to self — this worked with 12 students. Will it work with 18?)
- I collect the feedback. I assess it for clarity, consistency, and usefulness — does it give specific information about what the reviewee is doing well/should improve. If the feedback meets my criteria, I update my gradebook — giving well-reasoned feedback is one of the skills on the skill sheet.
- If the feedback needs work, it goes back to the reviewer, who must write a second draft. If the feedback meets the criteria (which it mostly did), then the original goes back to the reviewer, and a photocopy goes forward to the reviewee. (Did I mention I’m ready for 1:1 computers?)
- Everyone now works on a new draft of their presentation, taking into account the feedback they got from their classmates.
- I collect the new drafts. If I’m not confident that the class will be able to have a decent conversation about them, I might write feedback and ask for another draft. (Honest, this does not go on forever. The maximum was 4, and that only happened once.) I make yet another packet of photocopies.
- Next class, we will push the desks into a “boardroom” shape, and some brave soul will volunteer to go first. Everyone takes out two documents: the speaker’s latest draft, and the feedback they wrote to that speaker.
The speaker summarizes how they responded to people’s feedback, and tells us what they believe we can add to the model. We evaluate each claim for clarity, consistency, causality. We check the feedback we wrote to make sure the new draft addressed our questions. We try to make it more precise by asking “where,” “when,” “how much,” etc. We try to pull out as many connections to the model as we can. The better we do this, the more ammo the class will have for answering questions on the next quiz.
Lots of questions come up that we can’t answer based on the model and the presenter’s sources. Sometimes another student will pipe up with “I think I can answer that one with my presentation.” Other times the question remains unanswered, waiting for the next round (or becoming a level-5 question). As long as something gets added to the model, the presenter is marked complete for the skill called “Contribute an idea about [unit] to the model.”
We do this 4-5 times during the semester (once for each unit).
Example of a student’s first draft
I was pretty haphazard in keeping electronic records last semester. I’ve got examples of each stage of the game, but they’re from different units — sorry for the lack of narrative flow.
This is not the strongest first draft I’ve seen; it illustrates a lot of common difficulties (on which, more below). I do want to point out that I’m not concerned with the spelling. I’ve talked with the technical writing instructor about possible collaborations; in the future, students might do something like submit their paper to both instructors, for different kinds of feedback. I’m also not concerned with the informal tone. In fact, I encourage it. Getting the students to the point where they believe that “someone like them” can contribute to a scientific conversation, must contribute to that conversation, or indeed that science is a conversation, is a lot of ground to cover. There is a place for formal lab reports and the conventions of intellectual discourse, but at this point in the game we hadn’t developed a need for them.
Feedback I would write to this student
Source #1: Thanks for including the description of what the letters mean. It improves the clarity of the formula.”
Source #2: It looks like you’ve used the same source both times. Make sure to include a second source — see me if you could use some help finding a good one.
Clarity: In source #1, the author mentions “lowercase italic letters v and i…” but I don’t see any lower case v in the formula. Also, source #1 refers to If, but I don’t see that in the formula either. Can you clarify?
Cause: Please find at least one statement of cause and effect that you can make about this formula. It can be something the source said or something you inferred using the model. What is causing the effect that the formula describes?
Questions that need to be answered: That’s an interesting question. Are you referring to the primary and secondary side of a transformer? If so, does the source give you any information about this? If you can’t find it, bring the source with you and let’s meet to discuss.
Common trouble spots
It was typical for students to have trouble writing causal statements. I’m looking for any cause and effect pair that connect to the topic at hand. I think the breadth of the question is what makes it hard for students to answer. They don’t necessarily have to tell me “what causes the voltage of a DC inductor to be described by this formula” (which would be way out of our league). I’d be happy with “the inductor’s voltage is caused by the current changing suddenly when the circuit is turned on,” or something to that effect. I’m not sure what to do about this, except to demonstrate that kind of thinking explicitly, and continue giving feedback.
It was also common for students to have trouble connecting ideas to the model. If the question was about something new, they would often say “nothing in the model yet about inductors…” when they could have included any number of connections to ideas about voltage, current, resistance, atoms, etc. I go back and forth about this.
In the example above, I could write feedback telling the student I found 5 connections to the model in my first three minutes of looking, and I expect them to find at least that many. I could explicitly ask them to find something in the model that seemed to contradict the new idea (I actually had a separate section for contradictions in my first draft of the template). That helped, but students too often wrote “no contradictions” without really looking. Sometimes I just wait for the class discussion, and ask the class to come up with more connections, or ask specific questions about how this connects to X or Y. This usually works well, because that’s the point at which they’re highly motivated to prevent poorly reasoned ideas from getting in to the model. Still thinking about this.
Example Student Feedback
(click through to see full size)
I don’t have a copy of the original paper on “Does the thickness of wire affect resistance,” but here is some feedback a classmate wrote back.
Again, you can see that this student answered “What is the chain of cause and effect” with “No.” Part of the problem is that this early draft of the feedback rubric asks, in the same box, if there are gaps in the chain. In the latest draft, I have combined some of the boxes and simplified the questions.
What’s strong about this feedback: this student is noticing the relationship between cross-sectional area of a wire (gauge), and cross-sectional area of a resistor. I think this is a strong inference, well-supported by the model. The student has also taken care to note their own experience with different “sizes” of resistor (in other words, resistors of the same value that are cross-sectionally larger/smaller). Finally, they propose to test that inference. The proposed test will contradict the inference, which will lead to some great questions about power dissipation. Here the model is working well: supporting our thinking about connections, and leading us to fruitful tests and questions.
Example of my first draft
Sometimes I wrote papers myself. This happened if we needed 12 questions answered on a topic, but there were only 11 students. It also happened when we did a round of class discussions only to realize that everyone’s paper depended on some foundational question being answered, but no one had chosen that question. Finally, I sometimes used it if I needed the students to learn a particular thing at a particular time (usually because they needed the info to make sense of a measurement technique or new equipment). This gave me a chance to model strong writing, and how to draw conclusions based on the accepted model. It was good practice for me to draw only the conclusions that could be supported by my sources — not the conclusions that I “knew” to be true.
I tried to keep the tone conversational — similar to how I would talk if I was lecturing — and to expose my sense-making strategies, including the thoughts and questions I had as I read.
In class, I would distribute my paper and the rubrics. Students would spend the class reading and writing me some feedback. I would circulate, answering questions or helping with reading comprehension. I would collect the feedback and use it to prepare a second draft, exactly as they did. If nothing else, it really sold the value of good technical writing. The students often commented on writing techniques I had used, such as cutting out sections of a quote with ellipses or using square brackets to clarify a quote.
Reading student feedback on my presentations was really interesting. I would collect their rubrics and use it to prepare a second draft. The next day, I would discuss with them my answers and clarifications, and they would vote on whether to accept my ideas to the model. At the beginning of the year they accepted them pretty uncritically, but by the end of the year I was getting really useful feedback and suggestions about how to make my model additions clearer or more precise.
I wish I had some student feedback to show you, but unfortunately I didn’t keep copies for myself. Definitely something I will do this year.
How It’s Going
I’m pretty satisfied with this. It might seem like writing all that feedback would be impossible, but it actually goes pretty quickly.
Plan for improvement: Insist on electronic copies. Last year I gave the students the choice of emailing their file to me or making hard copies for everyone and bringing to class. Because bringing hard copies bought them an extra 12 hours to work on it, many did that. But being able to copy and paste my comments would help. Just being able to type my comments is a huge time-saver (especially considering the state of my hand-writing).
The students benefit tremendously from the writing practice, the thinking practice and, nothing to sneeze at, the “using a word-processor correctly” practice. They also benefit from the practice at “giving critical feedback in a respectful way,” including to the teacher (!), and “telling someone what is strong about their work, not just what is weak.” If their writing is pretentious, precious, or unnecessarily long, their classmates will have their heads. And, reading other students’ writing makes them much more aware of their own writing habits and choices.
I’m not grading the presentation, so I don’t have to waste time deliberating about the grade, or whether it’s “good enough.” I just read it and respond, in a fairly conversational way. It’s a window into my students’ thinking that puts zero pressure on me, and very little pressure on the students — it’s intellectually stimulating, I don’t have to get to every single student between 9:25 and 10:20, and I can do it over an iced coffee on a patio somewhere. I won’t lie — it’s a lot of work. But not as much work as grading long problem sets (like I did in my first year), way more interesting, and with much higher dividends.
Resources
MS Word template students used for their papers
Rubric students used for writing feedback. Practically identical but formatted for hand-written comments
