Sunday, March 19, 2017
If you want to help your bad students DO NOT give an easy exam
1) When I was a grad student TAing Formal Lang Theory we had a final ready to give out but noticed that one problem was too hard. So we changed it. But we made it too easy. Whoops. My thought at the time was this will help the bad students. I was wrong. Roughly speaking the students who got 70-80 on the midterm now got 90-100 on the final whereas the students who got 30-40 on the midterm got 35-45 on the final. So the bad students improved, but the better students improved more.
2) When I teach Discrete Math to LOTS of students we have a policy about midterm regrade requests. Rather than have them argue in person they have to:
In writing make a clear concise argument as to why it was mis-graded
If your argument displays that you really don't know the material, even when you can reflect on it, you can lose points. (True Story: We ask for an example of a Boolean Function with two satisfying assignments. They gave us a formula with only one, so they got -5. In the regrade request they try to still argue that it has two satisfying assignments. They lost 2 more points.)
In reality the policy is more preventative and we rarely remove points. However even this policy benefits the better students more than the poor ones who have a hard time even articulating why what they wrote is actually right (likely it is not).
3) Just this winter teaching a 3-week 1-credit course we were grading a problem and giving lots of 15/25 since the students were all making the same mistake. Half way through I got suspicious that maybe WE were incorrect. Looking at the exact wording of the question I realized WE were wrong, and, given the wording and what they would quite reasonably think we wanted, they were right. So we went back and upgraded many students from 15 to 25. And again, this lifted students in the 70's to 90's, but did NOTHING for the students below 50 since none of them had anything like a correct answer to any way to view the question.
Okay, so what does all of this mean? It means that an easy exam or a generous grading policy is devastating for the bad students.
However, that's just my experience- what are your experiences with this?
Subscribe to:
Post Comments (Atom)
I'm currently a TA for the formal languages class at UC Irvine, and I've noticed the same thing. The top students on our tests get perfect scores, and the bottom students still manage to miss a lot of the easy questions.
ReplyDeleteMake the exams impossibly hard so everybody gets 0!
ReplyDeleteThen everybody will do just as well as the best students!
You are prob kidding but I will make a intelligent comment out of this: Some teachers give incredibly hard exams where the top score is (say) 60. To me this is a waste- you are wasting a large portion of the grade spectrum. I've given exams where there were at least 10 people in EVERY ten point inteverval starting at 20-29. This is a good use of the grade spectrum. Everyone getting a zero.... is not
DeleteA good use of the grade spectrum is if no more than one person got 100 and 0, IMHO.
DeleteMaking exams really hard can benefit all students as follows. If the top score is 60% then there will have to be a grade curve, which is equivalent (if we are just shifting the mean) to adding certain number of points to each work. However, the students can get 60 out of 100 by selecting a SUBSET of the problems for which they know the answer. It is like saying here is 10 problems, it suffices to solve any 6 out of 10 to get a high score, and there is no downside to trying them all. Since various students have strengths in different subareas covered in the course, it is not immediately clear that this style would benefit strong students more than it would benefit weak students.
DeleteThis is only true assuming time is unbounded, and that students will not panic and fail to do even the problems they know how to solve.
DeleteOnce gave a question that I intended to be difficult (to separate the A students from the B students). It was in a class for all majors, so one Education student complained that this was unfair, that I should expect every student to be able to get every question correct. Needless to say, I don't agree with that and still give questions of varying difficulty. But as with Bill, any easy question with even a bit of substance still trips up the weak students. One consequence of this is that I have started giving shorter exams over the years, since the reliability of a 10 question exam is as good as a 20 question exam, and allows students more time per question.
ReplyDeleteMy exams are notorious for their LACK of time pressure. Even the hard questions- more time does not help.
DeleteAnd again- this lack of time pressure helps the good students some and the bad students much less.
Hmmmm … perhaps one lesson is that it's convenient for professors to assume implicitly that "this [better test-scores] will help the bad students" and conversely that relatively worse test-scores will be "devastating for the bad students" (the quoted passages are verbatim from the OP).
ReplyDeleteAre there examples of students who went on to outstanding achievements in research domains where they had poor grades, or poor teachers, or no undergraduate-level training at all?
Ed Witten maybe? Alexander Grothendieck? Marsha Linehan perhaps?
By what process(es) did these marginal students learn what they learned, and achieve what they achieved?
"Are there examples of students who went on to outstanding achievements in research domains where they had poor grades, or poor teachers, or no undergraduate-level training at all?"
ReplyDeleteGalois of course..
My policy on regrades is to regrade the entire exam, not just the one question they want regraded. Excepting a clear error on my part (which does occasionally happen), I often find that regrades of specific questions boil down to the student explaining to me why their wrong answer can be viewed as correct via some tortured logic, but then I look through the rest of the exam and see I gave them partial credit for questions where they really had no idea what they were doing.
ReplyDeleteWhile I agree with most of the content, I find easy exam questions a great way to distinguish between weak students from bad ones. The definition of bad student shouldn't include those who are actually trying but are having a hard time with the material. These people will at least be able to answer to questions that are similar to homework, or require less creativity, because they study for the exams. The bad students however will miss even these kinds of questions.
ReplyDeleteAnd that is good why? You are simply rewarding people for meaningless memorization and effort that doesn't help them in any way but wastes their time and energy.
DeleteThe REASON we teach people mathematics is to convey a useful skill or in higher math for the love of the subject and knowledge. Being able to rotely apply a bunch of rules doesn't do anyone any good. After they leave your class no one is going to come up and surprise them with a homework problem so if they can't use anything they learned in the actual world why celebrate putting in tons of effort for no benefit?
I understand why it feels good to reward the student who tries hard and fails rather than the slacker but if they end up at the same point isn't it (other things being equal) better not to spend all that effort for naught?
Moreover, I would argue that as an instructor one has a MORAL DUTY to issue grades based on mastery of the material qualified only by as much effort incentive as is absolutely necessary to incentivize actual learning (as students age they need less handholding to study and less of the grade should be effort based).
Deliberately boosting the grade of students simply because they try hard and you feel sympathetic seems to me to be the same kind of immoral act that giving your cousin a better grade in the class because you like him would be. You are no longer in the business of either instructing or evaluating performance but are simply handing out favors based on sympathy and affinity for their character.
Peter- At first I couldn't tell if you were arguing with Pooya or with me or with both. But you raise a good issue beyond what I was saying (which is great!)
DeleteMy point was a limited one- I speculated that an easy exam is a bad thing for bad students and wondered if others made the same observation. Mostly yes (though your experience with Calclus, a comment below, might be an exception).
You are raising a better question: SHOULD we give easy exams? How easy? If all a student does is memorize HWs and we pass them for this is that bad? (I think you think yes.)
All good questions. I may blog on it later, but here are some thoughts for now
1) I like to have a mix of easy and hard questions, where there are some hard questions to sep the A's from the B's, but also some questions that are straightforward. I sometimes err on the side of being too easy, which is where my observations/speculation came from.
2) I DO try to ask questions that require MASTERY of the material, but not really quick or clever thinking. An extreme case of this is a take home exam where I TOLD the students WHICH article in the literature to read to get the answer- but they still had to be able to explain it to me.
3) Is the lack of CLEVER questions a bad thing? This is debatable.
If an "easy" exam boosts some "B" students into "A" and some "C" students into "B" and removes all doubt about "D" and "F" students is that a bad thing?
ReplyDelete
ReplyDeleteMy experience with calculus courses is that hard (conceptual/proofish) questions absolutely crush the poor students while the good students muddle through. This is a combination of intimidation and learning styles/choices. The poor students in intro calculus tend to treat it as a series of recipes they plug problems into and any problem which can't be fit into one of those existing recipes might as well be written in an alien language and they are too intimidated to even try. Indeed, the most substantial barrier many students in intro math courses seem to face is working up the courage to try things and fail (I've spent entire office hours trying to get a couple of students to even try guessing a relationship between epsilon and delta). The good students are more confident and are more willing to give problems they don't recognize a chance.
I think we are talking at cross-purposes. You assume that "easy" means "measuring mechanics" and "hard" is "measuring understanding." One can give easy conceptual questions -- ones that students who understand the material will breeze through, and one that blind memorizers will miss.
DeleteI also like a mix of "easy" questions and hard ones. The former for the benefit of exactly this group: has worked at the material but not mastered it.
ReplyDeleteIMHO, the first-order problem is giving "35-45" (post-curved) scores, for example, on ANY exam in the first place. An "F" is typically defined as "59% and under." A student who gets an F on exam 1 and then recovered on exam 2 with an A should get a C avg for the two exams. That seems a "fair" summary of the overall performance, and motivates striving for A's versus giving up. The result should not depend whether that first "F" is 10 or 59 (e.g. (10+93)/2 ~ 52 --> F avg) or a 59 (e.g. (59+93)/2 ~ 76 --> C avg).
ReplyDelete[BTW, FWIW, I got all A's, from K thru PhD, including at top/competitive schools -- so, I am not some "disgruntled student." I'm just someone who thinks many profs tend to assume way too much precision / validity actually holds for their test scores / testing methods, seemly based in large part that their class grades seem (cough, central limit theory, cough ...) to "work out" nicely, on average, in the end (i.e. their get the desired distributions of numbers of final A,B,C,D,F's). That doesn't mean that there isn't unnecessary / undesirable statistical variance -- aka "unfairness", in student lingo ...]
It is exactly the other way around: it is students who believe grades should have no error. Profs know this is impossible. But they also know it's counter-productive to argue about it with the students.
DeleteLet's make grade spectrums great again!!!!!!!
ReplyDeleteTests are about grading Spectra!!!!
The issue is not easy or hard exam. You have to design better exams. You should be clear about what you expect them to take away from the course and then test for that. It is important to understand who we are teaching. We are not teaching only for future famous theoreticians, we are teaching also a large number of average software engineers who need to know when to use algorithm A and when not to use algorithm A, etc.
ReplyDeleteIn my exams I test them for basic knowledge about how to use data structure and algorithms, how they work, etc. and I have a few questions which go into fine points like what would go wrong if we did Y in place of X which needs a deeper understanding of how and why things work and the proofs, and then a few question that need creativity like solving a new algorithm problem and even in those cases it is not a good idea to go too far from examples they have seen. A 2 hour exam with a lot of stress and pressure are not situation we tend to be creative. Any average student who has attended lectures and spend some time reviewing them for the exam should be able to get a fair amount of points from the early sections of the exam to pass the course. Give difficult question which need creativity for assignments where they are not under 2 hour time pressure, not for exams. Those who are exceptional will show themselves in the assignments and if they are interested you can give them extra problems to think about or maybe a summer research project.