Sunday, May 07, 2017

Students try to memorize rather than understand! Who knew! (everyone)


Discrete Math. Required for CS majors, taken mostly by Sophmores.  Goal is to teach them how to think  rigorously. Topics are logic, number theory (not much), induction, sets, functions, relations, combinatorics (includes Pigeon hole prin, henceoforth PNP), prob, countability, uncountability.

We taught the Pigeon Hole Principle and gave MANY examples and HW of the following type:

Let A be a subset of {1,...,50} of size 10. Show there are two subsets of A that have the same sum.

ANSWER: There are 2^{10} = 1024 possible subsets.

MAXSUM is  41+..+50 = (1+...+ 50 )-(1 +...+ 40) = 50*51/2 - 40*41/2 = 455

MINSUM is 0 (the empty set). So the NUMBER OF SUMS is 456

Since 1024 > 456 there are two subsets of A of the same size.

The EXAM covered PHP, combs, prob, and induction. Hence they should know n choose k

On the HW and in class we NEVER did a problem where we only cared about the subsets of a fixed size. Conceptually this is really the same problem but if you had MEMORIZED  the proof template and tried to apply it you would get it wrong.

I asked the following on the exam: which was worth 20 points.

(20 points) Let A be a subset of {1,...,21} of size 8. Show that A has at least two subsets of size 3 which have the same sum.

ANSWER: There are (8 choose 3) = 56 subsets of A of size 3.
MAXSUM = 19+20+21 = 60, MINSUM = 1+2+ 3 = 6,
 So the NUMBER OF SUMS is 60-5 = 55.

Since 56> 55 there are two subsets of A of size 3 that have the same size.

Grading rubric:

If they got (8 choose 3) thats 3 points.(Many said 2^8- I suspect incorrect memorization)

If they got the MAXSUM and the MINSUM both right (aritj errors- NO penalty) then 3 points
(Many had MINSUM=0- I suspect incorrect memorization).

If they knew to use these PHP then 3 points.

If they got all three right then 20 points

So they got 0,3,6, or 20.

Here is the final tally:

0 points: 85

3 points: 190

6 points: 71

20 points: 167

(For the entire exam: 39 100's, 55 90-99, 77 80-89, 77 70-79, 76 60-69, 70 50-59, 47 40-49, 24 30-39, 24 30-39, 17 20-29, 3 10-19, 3 1-9)

This all raises the much harder question- how can we get students to UNDERSTAND rather than MEMORIZE

Telling them: DO NOT MEMORIZE! TRY TO UNDERSTAND!- I did this. Oh well.

Allowing a cheat sheet (which I did) is both good and bad for this issue.

Giving them a much wider variety of problems of this type. Either they would understand OR they would memorize several different templates.

I WELCOME your thoughts on either my grading or on how to get them to try to UNDERSTAND rather than MEMORIZE.


46 comments:

  1. "There are 2^{10} = 1024 possible subsets of size A." should be "There are 2^{10} = 1024 possible subsets of A."

    More importantly, the "MAXSUM" in this example should be a sum of ten elements (41 + ... + 50) not five.

    ReplyDelete
  2. thanks, fixed
    I have my TAs proofread and prooftake my exams VERY carefully so that this kind of mistake would not happen on a real exam.

    ReplyDelete
  3. For my entire academic career, I always memorize by means to understand. That is in fact what tests/exams are right? An assessment on a students ability to MEMORIZE concepts. When it came to math, I always had to remember theorems and axioms, some of which I do believe led to a better understanding. This is a rather gray area, however so I had to do some self reflecting.

    Memorizing certain problems helped me UNDERSTAND the procedures to solve them. And of course, if I identified something similar to what I have seen before on a hw or quiz again on an exam/test, I was VERY confident that I could solve it, perhaps overconfident to the point where I lost focus got ahead of myself and well...completely shot myself in the foot and jeopardized my entire future...maybe...

    I believe memory is a FUNDAMENTAL tool of the learning process. But upon my research on the topic, I found that A LOT of scholars vary in position. Some favor memorization while others favor understanding and vilify the former. Ultimately, memorization is what most of a student's academic career is built off of. If a teacher asked me "What is sine of pi/2?", I would fervently answer, "one!". Not because I understood WHY but because I made it a point to memorize the Unit Circle....I memorized that sine meant the y value and cosine meant the x value, etc.

    As a student, this troubles me, and makes me question my true understanding of what I actually DO know. It also strikes me with doubt and fear. It's an old habit to shake. A student might say, "Do I have time to understand the concept of what I am memorizing, or will that simply be enough to help me pass this course. And for some, memorization is not even their strong suit! Cheat sheets or formula sheets may help out once in a while, but if constructed correctly they can be very helpful. For example. I listen to what my instructors say, not to memorize a problem from an assignment and hope it shows up again on a test, so I might copy a problem I answered successfully and be sure as to mark it such that the PROCESS of how I solved it is explained and WHAT the logic behind it is. For example, a Pigeonhole problem. One could write down the answer to an old problem but take CAREFUL notes on how he ACQUIRED that answer. Or perhaps writing down the probabilities of poker hands? One could write them all down, again WITH CAREFUL notes explaining WHY the answer is what it is in hopes to understand the procedure on how to solve another problem like it...

    Perhaps this is all just a futile effort to justify a colossal mistake. But upon my reflecting I remember a prolific quote that resonated with me and still fuels me to learn more: "I know one thing, and that is the fact of my own ignorance."

    ReplyDelete
    Replies
    1. _That is in fact what tests/exams are right? An assessment on a students ability to MEMORIZE concepts._

      Nope. They're an assessment of a student's ability to UNDERSTAND and APPLY concepts, especially to NEW problems. Remembering the concepts is only the first step, and probably the least important one.

      Delete
    2. @Jeff Erickson

      I believe we don't do a good job of teaching theory courses because we focus them too much on what is useful for someone who would pursue a career in theoretical computer science research rather than what is useful for most of the students taking the course. Yes, problem solving skills is important, however it is not the only thing that is important. Ability to know when to use basic data structures and algorithms is as important if not more important than ability to rigorously prove the correctness of an algorithm is correct for average software engineer. Yes, if they could also prove the correctness that would make them better software engineers but they can still be pretty good software engineers without that as well. In reality a software engineer seldom needs to design a new data structure or algorithm or prove their correctness, their job most often is to combine existing implementation of them. So for vast majority of people taking an algorithms or data structure course it is way more useful to understand how and when to use what already exists and where to find relevant information than the ability to design new data structures or prove their correctness.

      Let me tell that vast majority of software engineers in top companies like Microsoft, Facebook, Amazon, Google, Apple, etc. (the crust of what exists in the industry) cannot even clearly state the problems and model them mathematically. This tells me that we are doing a very bad job in how we are teaching theory courses. They don't even have skills that are the base of what you want them to have. When we get that part right I would have no problem with having more questions like those in your algorithms course lecture notes.

      Delete
  4. Hi, Bill. I'm one of your Teaching Assistants and had some comments. I think that the grading on this question was quite harsh, and I think it boils down to the question being pretty simple for a whopping 20 points. It's too easy to make a small slip up and lose all the points. I would recommend something like this:
    Part A: Let A be a subset of {1...21} of size 8. Find some number X such that there MUST be at least X subsets of A with the same sum. This should maybe be 5 out of the 20 points because it's exactly like a homework problem.
    Part B: Show that A has at least 2 subsets of size 3 with the same sum. Putting myself in a student's shoes, this part sets off an alarm in my head. Clearly there must be some difference between this problem and the last problem. Now the student is forced to realize that this is NOT a problem they've done before and that they have to think critically. Without this Part B, I think it's too easy for a student to slip into his/her comfort zone, or skim over the problem and assume it was like the homework, or gloss over the fact that the min sum is 6 and not 0, etc.
    New rubric: 5 points all or nothing for Part A. If you can't recreate the answer from the homework then no mercy. 15 points for Part B. 7 points if you counted the number of subsets as 8 choose 3, 3 points if you found the max sum as 60, 3 points if you found the min sum as 6, 2 points if you used PHP at the end. I think that this is a more fair distribution that allows students to still demonstrate some semblance of knowledge even if they can't put all the pieces together. The addition of Part B is kind of like a hint, but it doesn't really give anything away about how to actually solve it, just that you have to think harder than on Part A.

    ReplyDelete
    Replies
    1. You think they got it wrong from not understanding the problem.

      I think they got it wrong from trying to memorize rather than understand.

      Not at all clear who is right about that, of course ther eare some of each, but not clear how many...

      Giving two problems so that the first one is of the type on the HW and the second one is ``new'' might be a way around the issue of understanding what I am asking for, as you suggest.

      Delete
    2. Why not give some points to those who recognized it was a combinatorics question but had used a wrong number for 8 choose 3? Isn't there some merit to that vs a student who did 2^8?

      Delete
    3. "You think they got it wrong from not understanding the problem. I think they got it wrong from trying to memorize rather than understand."

      Sounds like you're in violent agreement.

      Delete
    4. "You think they got it wrong from not understanding the problem.

      I think they got it wrong from trying to memorize rather than understand."

      One way to tell of which of these would be to later check with them and tell them "Your MINSUM is wrong, and it is not an arith error". I think most will now pause to think what they are doing and now get it right.

      Delete
  5. "Since 1024 > 241"
    should be
    "Since 1024 > 456"

    ReplyDelete
    Replies
    1. I fixed the post for Bill. Wonder what kind of grade Bill would have given himself?

      Delete
    2. Would his grader have thought he was memorizing?

      Delete
  6. Get them to write out 100 times in longhand the sentence: "I must UNDERSTAND stuff and not merely follow rules blindly." If that fails, try again using 200 repetitions.

    ReplyDelete
  7. I think you'll have a tough time getting students to stop memorizing. It's something so ingrained in our very being ever since we started 1st grade. Spelling tests, history tests, english tests, even early math tests all involved rote memorization of facts, rules or readings to get good grades thus memorization is rewarded and reinforced. I think the problem is largely with how overvalued memorization is prior to college which makes it incredibly difficult to break that cycle once you get there and professors want you to think critically.

    ReplyDelete
  8. One of the problem is, students only get a limited amount of time for a test. (And for good practical reasons). The more answers you can answer from memory, the more time you have to answer the questions where you have to show some insight.

    I understand the wish to test people on how well they understood, but I learned 40 years ago in high school that deriving everything just doesn't leave you with enough time.

    Perhaps if you take the time to test each student individually, during an oral examine with a white board, you can get a better insight of how well the student understands the subject. But I guess you aren't willing to invest the time it will take, and it may not fly well with the department either.

    ReplyDelete
    Replies
    1. I try to make my exams have NO time pressure- and based on how many hand it in early I am partially successful.

      With 510 students cannot give each one an oral exam.

      Though my experience when I've had smaller classes andI do give Oral exams is that students seem to know LESS than I thought they did.

      Delete
  9. This post is a little too students versus teachers for my liking. My guess is that some of the students who got 6's really did understand the material, but just didn't think carefully enough and so made a stupid mistake on the test. As far as the grading itself goes, rubrics are hard to make and I can understand why the rubric was made this particular way. The rubric can't see inside the students' heads; it can only grade based on what they wrote. Overall, though, I don't agree with this blog post because I don't think that "because some students wrote some wrong things on an exam means they just memorize everything and don't actually understand anything" is the right conclusion to make in this situation.

    ReplyDelete
    Replies
    1. I have been ASSUMING a rather monolithic reason for why they got it wrong-memorization rather than understanding.
      You think this is incorrect. Well, to be fair

      I'm right

      You're wrong

      JUST KIDDING (this is a popular refrain in my class, but used on cases where I really am right- like about Naturals and Evens being the same cardinality).

      But YES I agree that students may be getting it wrong for a far more complex set or reasons than I present.

      As for the post being too students vs teacher- again that is a valid point- and (even before reading your post) I emailed by class ABOUT this post and invited them to comment.

      Delete
  10. Hi, one of the students here. I missed all 20 points on this question. It feels like I am being condescended upon. I certainly understood all steps of having to do this problem. Had I not had the time constraints or pressure involved in taking a midterm I would have likely not made the mistakes I made. Whether or not that means I should receive a 0 on the question I leave for another debate, but it certainly does not necessitate that I didn't understand how to do this type of problem.

    ReplyDelete
    Replies
    1. I have been assuming (and you are telling me I am wrong, and you may be correct) that just because a student gets a low score on the problem does not mean that he does not understand the problem. A few thoughts that neither agree nor disagree with you
      1) I try to give exams with no time pressure.
      2) Despite that, students often put pressure on themselves
      3) Owen (one of the TA's who commented above) things that some students misread the problem.
      4) sorry if you take this post to be condescening. The real question (aside can `can bill spell words like condensengin') is how to get students to understand rather than memorize. You are bringing up a different valid question- are exams a good way to measure this?

      Delete
    2. Probably the most reasonable way given the constraints. I was disappointed with the exam score but understanding of how it was graded, and considered it to be my fault that I made such crucial mistakes. It's not like a grader can know why I made the mistakes or how much I do understand if I don't show them.
      Someone above mentions oral examination with a white board. That would indeed be much more thorough if possible.
      And just for the record I do think that the time given for the exam is plenty. That may seem contradictory to what I was saying before. I think it's more just the nature of exams that it is a lot of pressure for some people, and even if you have a half hour extra - that time can be spent very quickly inside one's own head.

      Delete
  11. First off- I thought the question was 100% fair. It was a mouthful to read and take in, but it was actually the easiest question on the exam (I had no doubt I got it right)

    Anyone with a baseline understanding of sets and pigeonholes could piece this one together. And bill is right- he never time crunches his tests, which is great for me (Im slow) and I would argue that this problem is actually very close to the homework examples. Its just slightly modified.

    HOWEVER!

    I find the grading to be absurd. Only options for grade either 20 or <=6? What the heck?

    It's like you're purposefully trying to remind students why testing is a terrible metric for your actual understanding of a subject.

    With no other question on the test to measure your pigeonhole knowledge, getting a 6/20 for a slipup is just obnoxious. The reason the problem was hard to understand was because of the wording of the sets, I think, not because of understanding pigeonholes:

    For reference, I got a 99. Ultimately I think the test was fair- but the rubric was harsh. Half of the class failed because of stuff like this. Just my thoughts, not trying to be obnoxious myself.

    ReplyDelete
  12. Hi Professor Gasarch,

    I too am one of your students, and I also admittedly got a 0 on this question. Before writing this post, in my natural Italian way I wrote an enormous post where I talked a figurative ear off for several paragraphs. As you well know, WE ARE BUSY PEOPLE. Straight to the point, and leaving out a lot of detail, this course is somewhat polarizing according to myself and many of my peers. Some people enjoy this course thoroughly, such as Math majors who may be taking the course, or CS students who enjoy this sort of math. On the other hand, many students are CS students who are in the program to learn how to be computer scientists. I myself am a computer engineering student, so I get to be an even more extreme example to prove my point. I am doing so because I wanted to be an electrical engineering student, but would also like to receive the kind of top quality education and skills that the CMNS school provides (for reference, I am in the A James Clark Engineering School, and as there are no low level dedicated CE classes, I take a mix of Elec. Eng. and Comp. Sci. courses). Your course, clearly, has many good things to provide: an understanding of induction, probability, countability, complexity, and many others. Many of these concepts will help us in the Algorithms course (CMSC351), a very useful course for CS students indeed. However, some of the concepts in this course appear to be far less useful. Let me start with a positive example: the boolean logic unit in the beginning of the course. No other material in the course is like it, and I have heard many a CS student complain that they'll likely never need it. I, however, found it fascinating, mostly because that is much more related to my interests and intended career goals, and as such I paid a great deal of attention to it and got plenty out of it. However, from my point of view studying for this midterm (and what I can presume many other students feel as well), the PHP appeared to be trying to solve one kind of unique problem. (If this blog actually reads the HTML tags as italics this is a MAJOR security flaw and you may want to contact your admin immediately). As such, unlike say induction, it did not SEEM to hold any kind of great importance, and so given the choice between truly understanding probability, synchronous/asynchronous logic, assembly language, PHP, and many others (keep in mind that many of us had other class exams in the same time frame), the PHP did not appear to be worthy of my utmost attention, and so I memorized it. I understand that this was wrong, however, many students do it frequently, simply because I was uninterested and it appeared to be somewhat unhelpful. Hopefully my story can give you some insight into how students think sometimes; we may prioritize what we find interesting/useful, and merely treat things that appear to be "just a part of the class" with much less care, as I did.

    I wish you the best for the rest of the semester!

    ReplyDelete
    Replies
    1. (The italics are italics! Great!)

      The question of how much of a discrete math course shold be directly or even indirectly applicable is a good question worthy of another post (or perhaps I posted on that a while back- I've been blogging since 2007

      ANother question you raise which I have NOT heard before:
      Which material does a student study?
      1) The material which appeared on many HWs (PHP would fit that bill)
      2) Material that the students finds interesting (perhaps your case)
      3) Material the student things the professor finds interesting (related to (1)).

      For intellectual purposes I can see (2).

      For exams (1) (maybe (3) but its really close to (1)).

      Every course will have material in it that you do not think is useful. Sometimes you'll be right about that, sometimes you'll be wrong about that.

      Delete
  13. I agree that understanding in more important than memorization. The problem is that exams are super crunched for time and not many people will make breakthroughs like that with such limited time under all of that stress/anxiety. Partial credit was barely given either which seemed harsh.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. Whether the time given for the exam is enough or not, thats entirely up to individual students. I heard TAs took the second midterm in less than 20 minutes. Maybe the exam should have been 20 minutes after all. Wait, they ar TAs, they have experience in the course, Bill is a professor, he probably wrote the midterm in five minutes. My point is the HWs/quizzes had me practicing the same kind of problems such that when a unique question was brought, with wording that was a little difficult to understand, i thought the underlying question was " can u understand that you ar supposed to use the PHP to solve this question?" On my way home i realized i shouldn't have used 2^8. I should have found a way to choose 3 from 8. Of course we can never know that i actually thought about the question this way on my long drive to Baltimore. But assuming i did think about it this way, given a few more minutes i may have had a 6/20 instead of a 3/20. Exams are not about how much of the material we understand, its about how many correct answers we can write given the time constraint. To answer your question, we would be able to understand material even better if more practice problems where provided. I apologize to all students that might have more practice problems in their homeworks next semester if my suggestion is taken into consideration. I think students perform better if the material is constantly reinforced in different ways. Am not saying we should be given practice problems on all unique cases, but is ther a possibility that these unique cases can be categorized? My other suggestion is that lose those slides, go back to chalk n board. Reduce the topics to be covered in the semester or break the course into two parts, 251 n 252. I agree with Owen, please break the problem into sub parts next time.

    ReplyDelete
  16. WOw- lots to comment on here!
    Time pressure on exam- I judge that by what percent of the class finishes
    before the exam is over. For the first midterm (NOT the one this question was on) over 1/2 left early, about 1/4 left after one hour of a two hour exam. For this exam about 1/4 left early. However, my notion of `if X % leave early then its not time pressure' might not be correct. Do you have a better method to evaluate this (I am NOT being sarcastic.)

    There were plenty of HW with solutions, quizzes with solutions, and a textbook, and pointers to prior exams. We COULD give out more problems next semester, to be done but not collected, with solutions posted, which means you wont' have to apologize to next years students.

    Slides?- a bit off topic, but okay we'll go there.
    PRO- since they are posted the students can go back to them very easily later.
    CON- They can make a teacher go to fast. We are aware of that and are trying to slow down (this last section on countability we are doing VERY slowly).
    CAVEAT- in a class of 120 students a chalkboard talk might be infeasible
    SECOND CAVEAT- since only about 60 come to class, maybe it would work.

    Reduce topics- This goes with the slides/go to fast issue. Rather than make it two semesters, could dump some topics.

    Owen wanted to break it into pieces. I agree and disagree. Having a problem more similar to the HW and THEN a problem that was a variant- YES. Guiding the student through it, NO- they should already know to do number-of-sets, maxsum, minsum, from the many HW and example on this.

    Meta Comment- THANKS for your (and the other comments above) inteligent comments- I mostly do not comment on comments, but these comments were worth engaging with.

    ReplyDelete
    Replies
    1. Professor Gasarch

      You definitely had lots to comment on. I enjoyed reading your defense i mean your response (defending lecture strategies) up to the part you forgot to address the issue of PRACTICING THE SAME KIND OF PROBLEMS. Yes, there are lots of quizzes but how many homework problems went beyond the basics of the PHP. Why ar we not understanding if there is plenty of practice? Oh am glad u asked, how can you evaluate time pressure? Conduct a survey on all exams, not just 250 bt all courses. I thought we did probability already? Am not being sarcastic, i am doubting my sampling skills .. Time pressure propagates from the experience a student has in other exams. But now i know that there is enough time for your exams so i hope i can relax in the final. My last question is, (everyone is wise so we could probably all go back and forth with the debate all day, oh and I still have to apologize to future students because they will stil be some students bothered by having those extra practice problems) back to the question, will you adopt any of the above suggestions Made on this blog?

      Delete
    2. `defending'- really?- notice that I have agreed with all of the comments, hardly `defending'. Count how many of my statements are defending myself and you'll see its very little (not just on your comment).

      HW problems beyond he basics of PHP. Not sure what you are getting at here since the exam question was on the basics and in fact was very similar to the main problem we used as an example. So- clarify what your point is here please.

      I think that RIGHT NOW I don't want to waste the students time with a survey on time pressure for the midterms. Might do it for the final. Can't do it for courses I don't teach.

      Will I use the comments here- I will try to make the final have NOT time pressure, and I have done this in the past- but again, only measuring by how many leave early. And this time I may do a survey afterwards to check if there was any time pressure.

      I will still ask some questions that test TRUE UNDERSTANDING. I might use Owen's technique of leading up to them, or more to the point make sure the students know that its different. For example, on the problem being discussed perhaps should ahve said

      NOTE- WE ARE LOOKING AT SETS OF SIZE THREE. NOT ANY SIZE. REALLY! REALLY! DO NOT COPY AN ANSWER OFF OF YOUR CHEAT SHEET SINCE IT WILL BE FOR THE WRONG PROBLEM.

      I am serious- would that have helped?

      Delete
    3. By same problems i mean, all homeworks/quizzes with PHP had minimum sum of 0. 2^n subsets, max sum n(n+1)/2 and actually i still don't know how to find the number of different sums and the reason for that is that my TA said one thing and a solution in the homeworks had another thing and the slides also had its own version. A true experiment is if you CREATE A RANDOMIZED CONTROLLED TRIAL. Do the student that finish early in the exam get good grades? Do they actually finish the entire exam? Do they have to rush across compass for another exam/class? Are they quick writters? I could poke many holes in your survey. Leading us like that may have worked for me. I would have put more thought in my answers. I would have gotten the 8 Cr 3 part but i would have never gotten the number of different sums right since In the homeworks /slides the minimum sum was ALWAYS ZERO.

      Delete
    4. Patrick: I think you're illustrating the problem here. In complaining that the minimum sum wasn't zero, like it was on all the examples you saw before, you're essentially asking that the exam questions differ from previous examples in only trivial ways, so that memorizing the examples will be enough to get a good grade. But memorization should not be enough to get a good grade in a course involving mathematical reasoning, if the course is to be something other than a farce.

      Delete
    5. In order for me to understand, the min sum should not always be zero. If it isn't zero in all the examples, then something in my mind would be triggered to ask why it is zero in some cases and why it isn't zero in other cases. I pretty much know minimum sum is the value of the set contain the smallest elements. I only got to learn that after the exam. Before the exam, i thought it was always going to be the value of the empty set. But you are right and so is Prof Gasarch, i should have tried harder to understand. But what exactly would have made me realize i did not understand the PHP if all examples i practticed had minimum sum zero? Again if you look at the numbers, 2/3 failed the question. Why? We didnt understand. Why? Because the mim sum was always zero and since you know "we are busy people" we thought we understood.

      Delete
    6. (Comment on both this and your other comment)
      1) YES- doing a survey to find out about time pressure will require NOT just asking `did you feel time pressure'- Have to balance getting info with not wanting to bug the students too much. And yes a randomized cgtrolled experiement woudl be great- though I doubt I could manage that.

      2) Is there was a contradiction between the slides and HW solutions then please email me about that (not really of interest to the blog readers), or come by to discuss so I can see whats up with that.

      3) ADVICE (though you've heard it before) When going over HW or Slides or Quizzes by yourself or with a TA try to make sure you understand whats REALLY going on. For example- WHY is MINSUM=0? Because the empty set is being included. Try to vary the problems yourself and see what happens to enhance your understanding. And (tired phrase) understand, don't memorize.

      Delete
    7. Something off topic,
      I saw your BOLD emphasis on the final. I am not discussing any questions on the final but i just wanted to let you know that time was not the issue today. It was a great exam. I felt like you listened to students problems and somehow you brought questions that seemed to make many students happy. You seemed to have brought problems that most students understood. I say this because my friends and I were discussing answers, which is a bad habit, i know that but students always do it anyway. Everyone was able to contribute valuable input on how to solve problems and from the reasoning that was presented by others, i felt we somehow all understand how to solve the problems. We did not use memorization this time, well atleast for most of the problems..

      Delete
  17. Memory and understanding are both important. The question is what attitude you have: do you focus on remembering? or do you focus on understanding?

    In my experience, focusing on remembering leads to way better results for the test next week. It also leads to terrible results for the test two years from now.

    Students see that remembering "works", and they assume it's because the tests are not properly designed. But, remembering "works" better on almost any test. (It fails on very difficult ones, but then students will have something else to complain about.)

    Convincing students to not focus on memorizing is difficult because it means you have to convince them to give up immediate rewards for the promise of some vague and distant (but bigger) rewards.

    ReplyDelete
  18. I think my comment may be valuable from the perspective of the student, even though it's been a while since I graduated.

    When you take a subject or module in the university, you usually do it with noble intentions. You take it along with some 5 - 7 other subjects of varying levels of difficulty.

    But as the semester rolls along, life happens. Some of the subjects (operating systems!) are far more interesting than others, so you spend disproportionately more time on them. You have extracurricular activities that take up time (debating club! orchestra!). Personal drama at the undergraduate level unfolds (romance!). Or maybe you're in the course just to get a marketable degree and get a job, and you spend time on things that are truly more important to you (games! parties! or just plain good old loafing and enjoying your youth).

    At any rate, by the time of the finals, most students have very little time and too many topics to cover. So you make a rational decision to optimize your time, usually just before the exam, which mostly involves a combination of cramming and selectively ignoring topics, and hope that you're not hit too much. Usually most people make it through without much damage.

    I personally love problem solving, and especially have fond memories of solving this particular folklore problem on my own after learning of the pigeon hole principle. But as heartbreaking as it may be to our beloved educators, mastering the pigeonhole principle (for e.g.) doesn't figure among the top priorities of most students. I myself haven't had to use it in the real world even once, even though some of the other discrete math topics like graph theory came in quite handy.

    A wise scheme might be to resign to this reality, and design the course for students of varying levels of interest in the subject. For those who really love solving problems, perhaps you could set separate tests for them, the top scorers among whom will be rewarded with a non-grade item? One of my professors used to give a specially printed folder as an award to the top 5 performers in our algorithms course (which I won)!

    As to the rest of the students, well - who knows, maybe they are CEO and not quite computer scientist material!

    ReplyDelete
  19. I felt a bit bad about my previous comment because it didn't say anything about how to solve the problem at hand, which was:

    > I WELCOME your thoughts on either my grading or on how to get them to try to UNDERSTAND rather than MEMORIZE.

    Instead, I recommended that the educators accept the fact that a substantial number of students are never going to want to invest the kind of time and effort the professors would like into their particular subjects.

    Here are a few tips that I think actually might work. They would have definitely worked with me!

    * Highlight the importance of the course material to the student's career. Top software companies place a huge importance on the candidates mastery of core CS concepts and ability to crack challenging coding problems, and they are tested on this knowledge in the job interviews. A sound grasp of discrete maths is key in acing these interviews.

    * Stress the problem solving aspect of the course in addition to the specific content itself. Discrete Maths is a very fertile field for honing your problem solving skills. Problem solving - i.e., the ability to go from a known state to a desired state using the tools at hand - is a highly transferable skill in areas even beyond Computer Science.

    * Make it competitive and fun. Ask students to form groups of (say) 3, and have them compete in problem solving competitions. Maybe make this a running affair throughout the course, replete with constantly changing leaderboards and everything. That might increase the student engagement with the subject.

    ReplyDelete
  20. Had this been a 10 point question where the possible scores were 0,3,6,9,10 it looks like a good "are you an A student" question but the jump from 6 to 20 makes it a bad question if assessment rather than fast grading is the goal. Having two thirds of the students failing it begs the question whether the topic was taught rather than just spoken.

    ReplyDelete
    Replies
    1. Lets make this into a question and a proposed answer:
      Why did 2/3 of the students get \le 6.

      Your conjecture that it was not in scope is not the case- we covered PHP and problems like this (reread the post) quite a bit. Perhaps more convincing- of all of the reasons given (both as comment in this post and within the class) nobody gave the `it was on material we didn't cover' reason or anything close to it.

      My reason (and the theme of this post) was that the students memorized rather than understood.
      Some of the comments above give alternative reasons.

      Delete
    2. it don't look like they were saying the topic wasn't in the scope of the class

      looks like they are asking if you actually taught the material or just talked about it

      if students can memorize instead of learn can't professors talk instead of teach?

      Delete
  21. I was very happy with the problems on the second midterm, and in particular those problems which test deep understanding as opposed to memorization. However, what I wish we would've been able to do is provide this second midterm before the 'W' date, such that people could have faster feedback about whether it made sense for them to be in this particular offering of the course. Unfortunately, the challenge of having to collaborate with the 132 and 216 instructors to temporally distribute all of our midterms, since there is a non-trivial overlap between our registration and theirs, did not allow us to do this this semester.

    During grading, I did see things such as <= A \cdot n and, of course, way, way too many, problem 5's with 2^n instead of n \binom 3. So I would agree with the perils of memorization.

    ReplyDelete
  22. Sorry for reviving this thread months after the original blog post with this onerous rant. I recently came across this blog, and I had a few personal experiences to share.

    I am a rising sophomore at Georgia Tech, and I took a discrete math class my first semester. I had some experience with proofs in high school, thus I already had a sense of familiarity with many concepts taught in class, but I had never practiced proofs or attempted problems of such nature.

    I did not go to high school in the USA so I didn't really know what to expect out of college. Around Junior year in high school, I developed an aversion to rote memorization. I used to be amazing at it and get near perfect grades, but after being introduced to TCS I vehemently rejected it. My grades faltered but I didn't really care.

    Discrete Math is one of the most fun classes I have taken till now. My professor is a big advocate of not giving a shit about grades. In fact, we had zero discussion about test related stuff in class. Any questions related to tests were to be directed to the TA. He used to get annoyed if someone asked questions like "How hard is the mid-term going to be?", "Is this topic important for the midterm?"
    I loved his ideology of solely focusing on the material.

    I observed a few things after interacting with my peers at Georgia Tech. Almost all of the students I interacted with just wanted an A in every class they took. Although this might be a perfectly reasonable ideology, it promotes studying for tests over understanding the material. Weirdly enough, for every class that I took modulo Discrete Math, the exams tested how good you are at memorization. The exam questions were a union of previous exam questions, homework questions, recitation quizzes, and of course, lecture notes. A small example:

    For my Intro Algorithms & DS class, the questions weren't like "Design an algorithm to ...." or "Prove why heap operations work in log(n)", but they were like "Draw how the heap looks like after we added these numbers" etc. You could get an A without understanding why anything worked. There was next to zero discussion about invariants/proofs in class for every algorithm and data structure we learned. Exam questions weren't analytical, but more like a "how well have you practiced for this midterm" test.

    In a way, you could say that the system itself values memorization over understanding. I remember my discrete math professor doing a deep dive into the lattice of stable matchings. I was fascinated by the concept and the beautiful structure it induced, but almost everyone around me was gasping at how complex it was. I remember people being ecstatic when our TA announced that it wasn't going to be on the midterm.

    Amazingly, a marginally harder version of the question discussed on this blog appeared on my first midterm as well; the major difference being that we hadn't seen anything remotely similar on any homework. I remember struggling with it for 15 minutes before getting the right insight, followed by the joy you get when you solve a unique problem from scratch.

    (Continued below)

    ReplyDelete
  23. (Part 2 of the above rant)

    The memorization versus understanding argument is something I've thought about (and discussed) too many times. I once discussed this with my good friend and this is the argument he gave:

    - Theory CS is not for everyone. Understanding non-trivial proofs can take a lot of time and be frustrating for the student who got As all his/her life.

    - Most people like other fields within CS, such as development. High GPA is important to get your foot-in-the-door. If you were forced to study something you don't really like, you might also resort to rote memorization.

    - Georgia Tech wants to prepare students to get high paying jobs in the industry. Thus it makes sense to focus on skills one would require to secure such jobs and do well in them. No one will ask you to prove why AVL tree insertions are O(log n) - they'll ask you to code it.

    I believe that this makes sense from a pragmatic point of view. Interests and goals differ from person to person. I enjoy delving deep into the material and understanding concepts more than getting straight As, although it may not be the most pragmatic thing to do in my situation. I can totally see why people prefer the other, less elegant approach as well. I feel like ultimately there isn't one correct way - it's just a function of how you look at education and what you want to get out of college.

    I didn't think I would write so much. This is something I've thought about a lot so it feels good to write it down, once and for all. Thank you for reading!

    ReplyDelete
  24. I love the post. Thanks for sharing these thoughts. Very inspiring!
    https://blog.mindvalley.com/how-to-memorize

    ReplyDelete