One of the comments on my blog on Types of question for exams
A True/False math question where they have to prove their answer. A student who picks the wrong answer can figure that out during the proof and then correct their answer. A student who picks the wrong answer and proves it has proven they really don't have a clueActually I once did an experiment about this! It's only one so I don't know what to read into it, but I will describe it and speculate.
CMSC 250 is the Sophomore Discrete Math course, required for all majors. CS 3 is a co-req. It's a course on how to prove simple things. We DO go over how a FOR ALL statement can be true vacuously (E.g.,all of the students over 10 feet tall will get an A+). Around 150 students take the course. In the spring there is an honors section of about 20. I PLANNED the following:
- In Spring of 2008 one of the questions on the final was a set of FIVE statements where the students had to, for each statement, say if its TRUE or FALSE and NO JUSTIFICATION NECC. One of the statements was If A is a set of natural numbers such that the powerset of A has 5 elements then A is infinite.
- In Spring of 2010 one of the questions on the final was a set of FIVE statement where the students had to, for each statement, say if it's TRUE or FALSE and IF TRUE THEN GIVE A SHORT JUSTIFICATION, IF FALSE THEN GIVE A COUNTEREXAMPLE.
Note that the statement is TRUE since there are NO such sets A.
So, how did they do?
- When NOT needing to justify or give a counterexample, of the 150 students in the class, 14 got it right. There was no correlation (or perhaps a very weak one) between those who got it right and those who did well in the course or those that were in the honors section.
- When the DID need to justify or give counterexample, of the 152 students in the class, 19 got it right. Slightly stronger correlation to those who got it right and those who did well in the course and to those in the honors section.
tricky question which some non-theory faculty members had trouble with when I explained this story to them. Exam Pressure was likely NOT a factor as my exams have very little time pressure- by the end of the exam
there were only about 30 students left taking it.
Here are the answers I got:
- FALSE- clearly A is finite.
- FALSE- too obvious to say why.
- FALSE- there is no such A
- Variants of the above.
- Incoherent things that may be similar to the above.
A clearly has ~2.32 elements and is thus finite.
ReplyDeleteI found this to be a terrible question (though I admit that, on first reading, I got it wrong). What exactly were you trying to test? Their knowledge of power sets, or their ability to parse logical statements? If the former, I would have just asked "Is there a set whose power set has 5 elements?" If the latter, I would have asked "Is the following statement true or false: for all A, (P(A) has 5 elements) => A is infinite."
ReplyDeleteAnon 1- I suspect someone has devised a set theory where a set really
ReplyDeletecan have a powerset of 5 elements. Fuzzy set theory? Some sort of Prob?
Anon 2- I was trying to test BOTH things, which may be why so many got it wrong. Was it a terrible question? It did not correlate with other measures of who the good students are, so in that rigorous sense YES it was a terrible question. It might be good as a HW/discussion question. Since some said it
was OBVIOUSLY false it could also lead to a good discussion about the need
for rigorous thinking.
If I had been taking your exam, I would've walked up to you and asked, "Do you mean exactly 5 elements or at least 5 elements?" The wording is ambiguous, I think; a set can have 5 elements in it without those being the only 5.
ReplyDeleteFor a introductory math subject, one quiz question was a two-parter, where the first part required them to do some simple algebra to obtain a result, and the second part required them to use a different technique to verify the result. One student made a mistake in the first part, so got the wrong result. They then fudged the verification, essentially by skipping the penultimate step and going straight to the conclusion that the original result was correct.
ReplyDeleteOf course, this "verification" didn't deserve any credit: either the student didn't know the technique, or missed the point of the question. Surprisingly, the student came up to me after receiving his graded quiz and attempted to argue that he should have got credit since the error in part b was a consequence of the error in part a. Talk about missing the point! He would have gotten credit if he'd pointed out that his part a answer was incorrect, but he didn't see satisfied with that!
I second Anon 2: this is a very poor question.
ReplyDeleteWhat about a universally quantified statement that is false and if you try to prove it true you get to a point where "5 < 3" needs to be true. A student who gets there and puts a QED after that had a chance to see their answer was wrong but just mechanically did a proof without understanding what they were doing and moved on.
ReplyDeleteOff topic again: does anybody know when are the STOC 2014 and CCC 2014 call for papers announced? (The level of unprofessionality is astounding.)
ReplyDeleteThe question hinges around whether the respondent is expected to include the "if" as part of the question. That is, are they expected to say whether "|- X => Y" is true, or are they expected to choose between "X |- Y is true" "and X |- Y is false".
ReplyDeleteAll three of these statements are correct, so I don't think it's a fair question to eliminate "false" as a correct answer.
If they have to explain their answer, then of course you might say they answered "false" for the wrong reason.
I had a sophomore math class MIT (18.25: groups, rings, fields) where our grade was based on 3 open-ended, open book exams consisting of 10 True / False questions. Scoring for each exam was +10 for the right answer, -10 for the wrong answer, 0 for no answer, but +200 for the exam if you had 10 wrong answers. Needless to say, there were more than a few -80 scores.
ReplyDelete