tag:blogger.com,1999:blog-3722233.post175012156525970133..comments2023-03-20T21:49:52.361-05:00Comments on Computational Complexity: Is Guessing a good idea?Lance Fortnowhttp://www.blogger.com/profile/06752030912874378610noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-3722233.post-31332409059848610412010-04-19T23:43:35.212-05:002010-04-19T23:43:35.212-05:00Ask the student to give a probability for each pos...Ask the student to give a probability for each possible answer ( summing to one ). If the correct answer is (i), the score for that question is log(p_i). If students want to maximize their expected score, their incentive is to describe exactly their belief. It penalizes random guesses but not informed guesses.<br /><br />Of course, it's always dumb to assign a probability of 1 to a choice, because if you're wrong you'll score -infinity, yet some student will still do that.Arthur B.https://www.blogger.com/profile/07960779431956791836noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-83140950072677219012010-04-19T14:40:52.754-05:002010-04-19T14:40:52.754-05:00The problem of assigning scores to the various com...<em>The problem of assigning scores to the various combinations of correctly or incorrectly ticked boxes has been completely solved...</em><br /><br />...provided you agree with the authors' axioms, in particular:<br /><br />(1) Guessing should be punished but wrong answers given in good faith should not.<br /><br />(2) The expected outcome from random guessing should be zero.<br /><br />But shouldn't making an <em>educated</em> guess be rewarded more than confidently asserting the wrong answer? Leaving a question blank transmits more information than a random guess; shouldn't it result in a higher expected score?<br /><br />When I give multiple choice exams, my rubric for five choices is A=1, B=-1/2, and C=+1/4. Multiple answers are wrong by definition, and negative totals are rounded up to zero at the end. Despite the "I don't know" option, a few students always leave some questions blank (= 0 points). <br /><br />Anyway, the short answer to Bill's question is "Apply linearity of expectation!" On my exams, you're better off writing "I don't know" if you can only eliminate 2 of the 5 options, but guessing is slightly better if you can eliminate 3.JeffEhttps://www.blogger.com/profile/17633745186684887140noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-59615530130387021142010-04-19T13:15:39.647-05:002010-04-19T13:15:39.647-05:00displaymame- the post you point to
is the standard...displaymame- the post you point to<br />is the standard case where you CAN<br />get a negative score. My problem<br />is that you can't get a negative<br />score, which I think makes the problem harder. <br /><br />GASARCHGASARCHnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-48861428862621463722010-04-19T11:14:24.426-05:002010-04-19T11:14:24.426-05:00See Tanya Khovanova's posts from January: To G...See Tanya Khovanova's posts from January: <a href="http://blog.tanyakhovanova.com/?p=203" rel="nofollow">To Guess or Not to Guess?</a> and <a href="http://blog.tanyakhovanova.com/?p=207" rel="nofollow">How to Boost Your Guessing Accuracy During Tests</a>. Although they are about a specific test (AMC), the analysis extends to general exams.<br /><br />I see nothing wrong with guessing; life and mathematics are often about making educated guesses and taking chances anyway.displaynamehttps://www.blogger.com/profile/09068351772472305473noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-1412711196379673012010-04-19T11:13:25.412-05:002010-04-19T11:13:25.412-05:00This comment has been removed by the author.displaynamehttps://www.blogger.com/profile/09068351772472305473noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-60499824319951981812010-04-19T10:45:18.139-05:002010-04-19T10:45:18.139-05:00There are good reasons for giving partial credit i...There are good reasons for giving partial credit in multiple choice exams for students who tick several boxes, provided the correct answer is among them. This avoids one of the major criticism of multiple choice exams. <br /><br />The problem of assigning scores to the various combinations of correctly or incorrectly ticked boxes has been completely solved, see the reference below. A remarkable paper (by two theoretical computer scientists) that every educator should know.<br /><br />A singular choice for multiple choice by Gudmund S. Frandsen and Michael I. Schwartzbach, Annual Joint Conference Integrating Technology into Computer Science Education archive<br />Working group reports on ITiCSE on Innovation and technology in computer science education table of contents. Bologna, Italy. Pages: 34 - 38 <br />2006. ISBN:1-59593-603-3 <br /><br />Online at http://portal.acm.org/citation.cfm?id=1189164<br /><br />Or at http://www.brics.dk/~mis/multiple.pdfThore Husfeldthttps://www.blogger.com/profile/14937584058954877153noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-35056663394874880282010-04-19T10:00:14.158-05:002010-04-19T10:00:14.158-05:00Reminds me of an introductory abstract algebra cou...Reminds me of an introductory abstract algebra course (groups, rings, fields) I took at MIT. Grades were based the total points from 3 open-ended, open-book true / false exams of 10 questions each. Questions were scored +10 / 0 / -10 for right / no / wrong answer. But if you got all 10 true / false answers wrong, you got +200 for the exam. It generously rewarded excellence and confidence, but severely punished hubris.Barryhttps://www.blogger.com/profile/15601407170512828091noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-26373070836093275662010-04-19T09:58:34.888-05:002010-04-19T09:58:34.888-05:00"These are multiple choice exams where you ge..."These are multiple choice exams where you get (say) 4 points for getting it right but -1 for getting it wrong. If there are 5 choices and you have no idea then you are better off NOT guessing and leaving it blank. If you can eliminate one of the choices then your guessing has expected value of 0, so either guess or don't guess. If you can eliminate more than one then you should guess. "<br /><br />you erred big time in the base case. If there are 5 options and you get +4 for a correct answer and -1 for an incorrect your expected value will be zero if you have no clue and you guess. Check again.Anonymousnoreply@blogger.com