Today at the American Association of  Physics Teachers summer meeting, I presented a 30 min talk on how assessment practices can start to transform our relationships to power — inside and outside the classroom.  I started by talking about how to balance rigour, accessibility, and instructor sustainability. The assessment techniques I presented were

• Standards-Based Grading
• Critical thinking-based rubric for assessing evidence
• Emergent curriculum

I’ve written a lot about all of those; see the post categories (to the right or below) for my progress on these.

Those led me to collective struggle against the status quo, liberatory practices, and decolonization.  To find out how I got there, a rough transcript is below.

I created screencasts of the Introduction, the Land Acknowledgment, and the presentation of the three assessment practices I focused on.  If you watch the screencasts, you’ll get a tour of the mind map of resources that I created to accompany the talk.

My heartfelt thanks go to Andy Rundquist and Deepak Iyer, who invited me to attend, and made my experience of giving this talk a wonderfully supportive one.

Assessment Practices for Rigour, Accessibility, and Sustainability: You Can Have All Three

Presented at: American Association of Physics Teachers Summer Meeting 2019, Provo, Utah

Introduction

I teach algebra-based circuit analysis and embedded systems in a 2-year community college.  I got there after working as a software developer, serving with the Canadian Coast Guard, and doing electronic R&D in an oceanographic institute.

Assessment Techniques

The focus for my 10 years of teaching has been on assessment practices that bring together rigour, alignment, accessibility, growth mindset, sense-making, and, diversity.  That required me to study conflict mediation, pastoral care, and techniques for making student thinking visible. I also had to learn to incorporate liberatory philosophies, and balance self-determination with practices of collectivity.  And try not to burn myself out.  You know, just a light weekend project.

I’ve tried lots of things, but there are three assessment practices that overlap with all of those:

• Standards Based Grading (I call it Skills Based Grading)
• Explicit instruction in critical thinking
• Emergent curriculum

In this presentation, for each of those techniques, I’ll propose an exercise for exploring whether and how they could be adapted in your context.  Those exercises are linked electronically in the section of the resource mind map called Assessment Techniques; the same thing is on paper at the stations around the room.

Lots of people do those things.  In the interests of time, I won’t talk much about the aspects of those things that are fairly well documented; I’ve left lots of resources in this  mind map.  I’ll mostly discuss the ways I’ve modified them to work together.

Assessment, Power, and Control

The theme is this: I found that increasing the rigor AND accuracy of my assessments required me to reconsider my relationship to power and control.  After all, how can I accurately tell what students can do, if they don’t have the power to show me the full range of their knowledge and capacity? I invite you to notice, throughout the presentation, where I have shared control, where I have relinquished control that instructors usually take, and where I have taken control that instructors usually don’t take.

In order to make good decisions about what forms of power to give up, I had to be accurate about the access to power I have; in other words, I had to correctly assess myself. I make my decisions as a queer and trans person who experiences homophobia, transphobia, misogyny, and occasionally some forms of male privilege, along with white privilege and a complicated relationship to ableism and class. I have been stalked, threatened, sexually harassed, or gay bashed at least once in every job I’ve ever had.  Figuring out how to make your decisions about transforming power will be different, based on how much access to power you have, and also how much exposure to violence you have.  My project is to find new ways every day, small and large, to transform power; I look forward to joining with others who are interested in sharing this learning.

Land Acknowledgement

This presentation is being given in Provo, Utah; to the best of my understanding, this is the traditional, current, and unsurrendered territory of the Ute, Goshute, Paiute, Navaho, and the Shoshone, although it may be the contemporary territory of other Indigenous nations as well; some of these nations were relocated to reservations here by the US government, and many of them used this territory for seasonal hunting and gathering.

In talking to Franci Taylor (from the University of Utah American Indian Resource Center)  and others from Indigenous organizations, there are some reports that there were treaties made on this land, but none were ever upheld by the US government.

In 2016, a coalition of tribes lobbied to have the land around Bears Ears Buttes designated as a national monument. But last year, the federal government slashed it by 85% in order to open it up to uranium mining.

I come from Mi’kmaki, the traditional and never-surrendered territory of the Mi’kmaw.  We also have peace and friendship treaties that guarantee the Mi’kmaw the right to fish, hunt, and practice their culture without interference, which have never been upheld by the Canadian government. As someone who inherits my legal status on that land from the British who signed the Peace and Friendship Treaty of 1725, I am part of that treaty relationship.

Keep that in mind, it will come back later.

Using This Presentation

I welcome you to interact with the material in any other way that works.  You can follow along with me, or go straight to the exercises, or just poke around the mind map. Every little icon clicks through to something. Feel free to work on your own, or not, — whatever makes the most sense for you to get what you came here for.

I’ll try to answer the most common questions that come up. Please add them to the web form or the paper slips at each station; in the interests of time I’ll take them from there instead of taking verbal questions from the floor. That way, by the end of today, we’ll have our own curiosity-tracking spreadsheet mapping the emergent curriculum of our community, which I will share with anyone who’s interested.

How I Got Started: Rigour

I started down this path because I became concerned, early in my career, with the rigour and validity of my assessments. The grades I gave didn’t seem accurate.  People were struggling to pass who had very strong skills, or who I thought had the capacity to develop very strong skills; people were getting very good grades who didn’t seem to understand the material and who I worried might actually be dangerous in the workplace.

Standards-Based Grading (SBG)

I started using SBG to get accurate info about what skills students need help with, and what they can actually do by the end of the semester.  I use tracking sheets that list the skills I will assess, and some of skills are mandatory; you can’t pass without them.  That raised the level of rigour; it’s no longer possible to pass if you can 60% do 100% of the things.  You have to be able to 100% do 60% of the things.  And I control which 60%.

But the heart of SBG is reassessment: if a student’s demonstration of a skill doesn’t meet the requirements, they can demonstrate again next week, or the week after, until the end of the semester.  There’s no penalty for how many tries it takes.

The skill itself should be the only bar; there should be no other barriers.

Self-Determination and Universal Design for Learning (UDL)

To remove as many other barriers as I could, I needed to make it possible for students to have as much self-determination as possible. If there was any way that they could meet the criteria, I wanted to see it. I started using a framework called Universal Design for Learning that emerged from disability rights advocacy, which uses the slogan “tight goals, loose means.”

One possible interpretation of UDL is that, instead of the instructor needing to provide multiple paths, students are “allowed, supported, and encouraged” to invent and control the path themselves. I’m using Zaretta Hammond’s language here; she created the Ready4Rigor framework for culturally responsive pedagogy, building on the work of Gloria Ladon-Billings among others, and it dovetails easily with UDL.

It shouldn’t matter whether students read the textbook, or watch videos, or tinker, or interview their grandma who works for NASA, to get their information.

Similarly, any way of demonstrating mastery that meets the requirements should count.

That means my rubrics have to be as flexible as possible. I don’t want to have to create a rubric for papers and a different rubric for song lyrics and one for crocheted models of subatomic particles.  I also want our rubrics to become habits of thought that help students practice the science-and-engineering ways of thinking, so that it becomes a reliable routine; in other words, they are criteria by which we assess thinking – or “critical thinking”.  One example is our rubric for assessing evidence, which is one of the techniques I’ve proposed an exercise for, if people are interested in adapting it. The rubric for assessing evidence is a set of ways of thinking can be applied to any topic, so I use it over and over.

The tricky part for me, was, how to you combine the predetermined structure of SBG with self-determination of UDL?  The way I bring those forces into balance is using an emergent curriculum.

Emergent Curriculum

At the beginning of the year, the class works on question-generating exercises – usually goal-less problems that require few safety precautions.  I have my students make working circuits from conductive play dough, or experiment with small light bulbs and AA batteries to see what they can do.  They take note of what they notice and what they wonder.  I add the questions to a spreadsheet – keeping them in the students words. That curiosity-tracking spreadsheet is the third assessment technique that I propose people might consider adapting. The next day, I project the spreadsheet and each student chooses a topic to investigate.  They find evidence, assess it, choose what they think is strongest, and present it to the class.  The class then peer reviews it – again using the rubric for assessing evidence.  Ideas that are well-supported by the evidence get added to what we call the “class model” – a shared reference of what the class has found out so far.  The model isn’t just something students can use on tests and other assessments; it’s something they are required to connect to, either to show support or contradictions for new ideas.

There are always some ideas that don’t have enough evidence to accept, or generate contradictory results.  These generate new questions, which go in the spreadsheet, and then the cycle starts again.

If I find it necessary to take control of the topics, I can simply choose to be present data myself.  I can assign the relevant textbook section and Wikipedia article, and have students assess it against the rubric for assessing evidence, the same way they assess their own and each other’s data.  I can also vet the curiosity tracking spreadsheet, by focusing on one topic before another, or by excluding some questions from our regular class cycle.

Effectiveness

I have only anecdotal data, but so far it appears that men and women complete the program in the same proportions, as do students who self-disclose as having disabilities and those who don’t.  Indigenous, Black, and students of colour seem to complete the program at a higher rate than average.  We see a similar phenomenon with women in the trades sometimes, where there are so many barriers that the only women who attempt the program are the ones who are most committed or have the most support.  This isn’t a good thing.  While we can all aim for excellence, we don’t have true equality if it’s possible to be a middle-of the-road man in this program but women have to be top achievers.

Still, this is encouraging data, and it’s also encouraging that I consistently hear students with disabilities say that the flexibility and high standards make the program both accessible and rewarding.

That tells you something about accessibility.

As far as rigor goes, I use the DIRECT concept inventory, which is a conceptual test of circuit understanding with 29 measures.  The authors publish the results of students who’ve completed honours high school physics, and students who’ve completed calculus-based university circuit analysis, probably in their second or third year.  My students, four months into their algebra-based program, consistent outperform university students on 21-25 of 29 measures.  Often by a large margin.

Try It

I hope that gives you an idea of how these practices work together.  I’m going to take a short break for people try things on their own. You can try one of the three exercises linked in the map and posted around the room.  Or, use this time to do what works for you.  We too can have self-determination in our learning!

if you need help, please congregate at the corresponding station; help each other if you can, and I’ll come around too. And don’t forget to add your questions to the web form.

Conclusions

There are 2 main reasons I find assessment so fruitful in contributing to the values that I prioritize.

1. Assessment can help generate data that exposes structural injustices.
2. Assessment is where I’ve been given the power to police the borders of what is a valid contribution to learning and science.  Where I have that power is where I can transform or relinquish it.

For those two reasons, the question that’s been keeping me up at night is this: Have you ever wondered what science would look like today if it had developed with everyone taking their full and rightful place at the table?

Of course, enormous contributions to science have been made by every group, at various places and times in history.  But the dominant idea of science today, which grows out of what we sometimes call the Scientific Revolution, is only about 500 years old.  And that is exactly the period during which the groups we today try to “include” got excluded in the first place.

How Did Exclusion Start 500 Years Ago?

Between 1350 and 1533, in Europe, same gender sex went from being not on the books to being punishable by death.  The borders of what was considered acceptable gender expression also became much more rigid, narrow, and binary than they had ever been before.

Women, who had previously participated in guilds, got pushed out.  Midwives and alewives were both suddenly criminalized.  Ending a pregnancy was suddenly criminalized.  But women knew what plants to eat if they didn’t want to be pregnant any more. I can tell you from personal experience that it’s not that difficult.

So there were massive revolts and mass executions.  Thousands of women died. We call them witch trials. Criminalizing the science of midwives made that knowledge much more difficult to spread and more dangerous to use.

Also emerging at the same time: private ownership of land in fee simple. Before that you couldn’t buy land any more than you could buy a river. Also, the nation state, and the modern form of policing that was needed to enforce it. In other words, rigid control over the borders of land.  European aristocrats did this first in Europe, then extended it around the world, in combination with another new invention: whiteness and white supremacy. These were used to justify chattel slavery and colonialism.

The Scientific Revolution Begins In a Sheltered Bubble

It’s important to know all this because the other thing that happened about 500 years ago is that Copernicus published On the Revolutions of the Celestial Spheres, marking the beginning of what we consider the scientific revolution: the emergence of the form of science that is dominant today.

That science developed in a sheltered bubble, protected from the kind of challenge and critique it would have faced if people of all cultures and genders had been taking their full and rightful place at the table.  If today’s science, as a seedling, had been required to be accountable to everyone, there’s no doubt in my mind that it would be something entirely different today.  It owes its default settings of individualism, hierarchy, and focus on control over natural forces to that early crucible of violence and suppression in which it grew.

And, it has accomplished amazing things!  Relativity is amazing.  Quantum physics is amazing.  But those things are necessarily a pale shadow of what could have been accomplished.  Imagine what that could have been.

So now that our current view of science has grown deep thick roots, and is extremely difficult to reshape, NOW we invite the pushed out people to return and participate in it?

Diversity and Inclusion: Where Are They Leading Us?

I’m going to suggest that diversity and inclusion are the Bohr model of scientific justice.  Powerful, accessible to many people, important in our path of changing ourselves and our institutions.  But very, very far away from the whole story.

Policing Borders of Knowledge, Bodies, and Land

Assessment, or the policing of the borders of knowledge, is a place where I have found it fruitful to question which aspects of power I hold that are illegitimate, and how I can relinquish and transform them.

And because that way of policing the border of knowledge arose together with and for the same reasons as policing the borders of acceptable genders, bodies, races, nations, and land, we’ll know we’re doing it right when our efforts on one of those support the efforts of others.  I want to be plain: I am saying that we cannot have true diversity and inclusion in science until we end the way we police the borders of the land we do it on. How can physics truly “include” women if women who are trans or Indigenous or both are dying and being disappeared? Eve Tuck and K Wayne Yang say it as “Decolonization is Not a Metaphor“; Indigenous sovereignty isn’t just possible; it’s necessary for all of us to move forward into a just future.

What Moves Us Toward Liberatory Goals?

And if you’re not sure whether that tree, planted 500 years ago, is really such a problem, consider that Gloria Ladson-Billings, originator of culturally-relevant pedagogy, gave a speech just last week concluding that we have not, in 65 years, accomplished the desegregation called for by Brown v Board of Ed.  She concluded, “we need to think differently of who we are as a nation.” Along with causing persistent inequality in our physics classrooms, this 500 year old tree motivates concentration camps at the border, telescopes that are forced on communities by national guards, it is also putting us on a collision course for climate destruction.  The bad news is that diversity and inclusion will not be enough: a complete shift in our relationships to each other and the land will be needed.  The good news is that, because everything is connected to everything else, we could work collectively toward a shared goal while taking a step from where we are. Assessment is where I’m taking some of my next steps; and when I’m deciding what to do next, I try to ask, which steps move us closer to liberation? I hope you’re found a new step you can take.  And I look forward to learning from each other how to take bigger steps together.