jump to navigation

Tests are Good for You January 21, 2011

Posted by gordonwatts in Teaching, university, University of Washington.

The New York Times had an article the other day talking about a discovery that is making rounds:

Taking a test is not just a passive mechanism for assessing how much people know, according to new research. It actually helps people learn, and it works better than a number of other studying techniques.

I’m here to tell you: duh!

In fact, we’ve institutionalized this in our physics graduate schools. Most university physics departments have the mother-of-all tests. Here at UW we call it the Qualifying Exam. Others call it a prelim (short for preliminary). And there is a joke associated with this exam, usually said with some bitterness if you’ve not passed it yet, or some wistfulness if you long since have passed it:

You know more physics the day you take the qual than you ever do at any other time in your life.

The exam usually happens at the end of your first year in graduate school. The first year classes are hell. Up to that point in my life it was the hardest I’d ever worked at school. Then the summer hits, and you get a small rest. But it is impossible to rest staring down the barrel of that exam, often given at the end of the summer just before the second year of classes start. You have to pass this exam in order to go on to get your Ph.D. And for most of us, it is the last (formal) exam in our career that actually matters. So physiologically, it is a big hurdle as well.

How hard is it? My standard advice to students is that they should spend about one month studying, 8 hours a day. For most people, if they study effectively, that is enough to get by. Some need less and some need more. This is about what it took me. What is the test like? At UW ours is 2 hours per topic, closed book, and all it is is working out problems. No multiple choice here! It lasts two days.

So, how do you study? There is, I think, really only one way to get past this. For 30 days, 8 hours a day, work out problems. There are lots of old qualifier problems on websites. Our department provides students with copies of all the old exams. Even if you don’t know the solution, you force your self to try to work it out with out looking it up in a book – break your brain on it. Once you can solve those problems with out having to look at a text book, you know you are ready. Imagine trying to study by reading a text book, or by reviewing your first year homework problems. There is no way your brain will be able to work out a new problem after that unless you are a very unique individual.

Note how similar this is to the results shown in the article:

In the first experiment, the students were divided into four groups. One did nothing more than read the text for five minutes. Another studied the passage in four consecutive five-minute sessions.

A third group engaged in “concept mapping,” in which, with the passage in front of them, they arranged information from the passage into a kind of diagram, writing details and ideas in hand-drawn bubbles and linking the bubbles in an organized way.

The final group took a “retrieval practice” test. Without the passage in front of them, they wrote what they remembered in a free-form essay for 10 minutes. Then they reread the passage and took another retrieval practice test.

The last group did the best, as you might imagine from the theme of this post!

This is also how you know more physics than at any other time in your life. At no other time do you spend 30 days working out problems across such a broad spectrum of physics topics. If you study and try to work out a sufficiently broad spectrum of problems you can breeze through the exam (literally, I remember watching one guy taking it with me just nail the exam in about half the time of the rest of us).

Working out problems  – without any aids – is active learning. I suppose you could follow the article and say that forcing the brain to come up with the solution means it organizes the information in a better way… Actually, I have no idea what the brain does. But, so far this seems to be the best way to teach yourself. You are actively playing with the new concepts and topics. This is why homework is absolutely key to a good education. And this is why tests are good – if you study correctly. If you actively study for the test (vs. just reading the material) then you will learn the material better.

And we need to work better at designing tests that force students to study actively. For example, I feel we are slipping backwards sometimes. With the large budget cuts that universities are suffering one byproduct is the amount of money we have to hire TA’s to help grade our large undergraduate classes is dropping. That means we can’t ask as many open-ended exam questions – and have to increase the fraction of multiple choice. It is much harder to design a test that goes after problem solving in physics using multiple choice. This is too bad.

So, is this qualifier test hazing process? Or is there a reason to do it? Actually, that is a point of controversy. Maybe there is a way to force the studying component without the high-anxiety of the make-or-break exam. Certainly some (very good) institutions have eliminated the qual. Now, if we could figure out how to do that and still get the learning results we want…


1. Claire - January 21, 2011

Hi Gordon

I try to give my first years a multiple choice section in each test (of which I do 5 per semester). However, I almost never ask problems in multiple choice, rather I go for conceptual questions. You can write them so that the students have to really think, and understand their work to be able to answer them. Something like “Alice says xxx about something, while Bob says yyy” and the choices are: “Alice is correct because… Bob is correct because…. Alice is incorrect because…” etc and various permutations of those.

This has a two-fold advantage… first, the marking is made easier for me (grin), and second, I can still ask rather difficult conceptual questions to a class where the majority has English as their second, or (often) third language.

Wrt testing as a learning tool, in adult education the “teach 5, test 5” prescription is well known to aid in the learning process (or at least, that’s what my parents have been teaching for years). Basically, the theory is that the human brain can only store on average 5 new pieces on information in its short-term memory. So after 5 new “bits” you should do a test or reflection exercise, to enable the student to put those into long-term memory. Then you can move to the next 5 bits. (the tricky thing is that one “bit” for you may be a number of bits for your students). I try to structure each lesson, and my course as a whole along that principle.

2. Andy Rundquist - January 21, 2011

When I took my qualifying exam (4 days long, 4 hours each day and then a 5th day with an oral exam) I felt just as you describe in this post. I thought I’d never understand physics better.

However, now that I’ve taught undergraduate physics for 10 years, I realize that I understand physics much better now than then. I’ve had the time, while preparing lectures, to really get down to the fundamental concepts. When I teach a new course, homework solutions come quite easily to me because I know what fundamentals to apply.

I remember that whole semester of 8 hours/day studying and now I feel like it was a little too close to what my students do: cram it in and spit it out.

I will say, though, that from a community building perspective it was great. We were all in the same boat and we’d come in and say things like “did you get number 1 from 3 years ago?” It taught me how to use resources and to flex my math muscles, that’s for sure.

Thanks for the post. When I read the NYTimes article I didn’t see the grad school connection.

3. Chip Brock - January 25, 2011

Hi Gordon,

One of the things I was determined to do when I was chair was to change that whole thing. I distinctly remember the “one shining moment” when I knew more physics than ever before or ever after (although I absolutely agree that teaching makes you learn more than you thought you would). I frankly never saw the sense in it.

Now, we have a committee that prepares the exams for the standard subjects at the beginning of the year. The committees include the faculty who teach those graduate courses: E&M, QM, Mechanics, Stat Mech. They’re put away and then the regular courses are taught and the exams are rolled out as both the final exams for that semester and that subject portion of the prelims. So, over the course of 3 semesters, all of the subjects are tested. They’re all done in a way that reflects the desires of the faculty since the exams are collectively produced. And nobody has to freak for that awful 3 days of writing. Students must have an overall average on the “subject exams” above a (rarely) tuned threshold (that’s pretty high). They have 2 tries on each exam. We also have a set available for people to take at the beginning of their graduate careers if they come with a masters or equivalent. We’re convinced that the same people pass who would have before and that the same people fail who would have before. We also solved the problem of students not taking the core courses and trying to just get by with the exams.

It rid us of the problem of random faculty deciding to see if they could construct problems that nobody could solve. It also rid us of the occasional problem where someone would decide that E&M needed to be taught with quaternions or something (well, that never happened)…so, it inadvertently made those core graduate courses more uniform without dictating it be so.

We’ve done this since about 1995 and we like it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: