Thursday, September 07, 2017

Statistics on my dead cat policy- is there a correlation?

When I teach a small (at most 40) students I often have the dead-cat policy for late HW:

HW is due on Tuesday. But there may be things that come up that don't quite merit a doctors note, for example your cat dying, but are legit for an extension. Rather than have me judge every case you ALL have an extension until Thursday, no questions asked. Realize of course that the hw is MORALLY due Tuesday. So if on Thursday you ask, for an extension I will deny it on the grounds that I already gave you one. So you are advised to not abuse the policy. For example, if you forget to bring your HW in on thursday I will not only NOT give the extension, but I will laugh at you.

(I thought I had blogged on this policy before but couldn't find the post.)

Policy PRO: Much less hassling with late HW and doctors notes and stuff

Policy CON: The students tend to THINK of Thursday as the due date.

Policy PRO: Every student did every HW.

Caveat: The students themselves tell me that they DO start the HW on Monday night, but if they can't quite finish it they have a few more days. This is OKAY by me.

I have always thought that there is NO correlation between the students who tend to hand in the HW on Thursday and those that do well in the class.  In the spring I had my TA keep track of this and do statistics on it.

The class was Formal Lang Theory (Reg Langs, P and NP, Computability. I also put in some communication complexity. I didn't do Context free grammars.)  There were 43 students in the class. We define a students morality (M) as the the number of HW they hand in on Tuesday. There were 9 HW.

3 students had M=0

12 students had M=1

9 students had M=2

5 students had M=3

4 students had M=4

4 students had M=5

1 student had M=6

1 student had M=7

2 students had M=8

2 students had M=9

We graphed grade vs morality (see  here)

The Pearson Correlation Coefficient is 0.51. So some linear

The p-value is 0.0003 which means the prob that there is NO correlation is very low.

My opinion:

1) The 5 students with M at least 7 all did very well in the course.This seems significant.

2) Aside from that there is not much correlation.

3) If I tell my next semesters class  ``people who handed the HW in on tuesday did well in the class so you should do the same'' that would not be quite right- do the good students hand thing in on time, or does handing things in on time make you a good student? I suspect the former.

4) Am I surprised that so many students had such low M scores? Not even a little.










11 comments:

  1. Ha, we use the slightly less morbid term "barfing cats excuse" with Beeminder. Eg, http://forum.beeminder.com/t/weasels-and-barfing-cats/3229

    And here's what I've used for a late homework policy: http://messymatters.com/deadlines/
    It has a similar idea for avoiding making judgment calls about what excuses are legit.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. (Try again with less typos- thats why I removed)

      1) Barfing cat! Wow- I'm surprised my examples is so close to someone else's example. More on that later.

      2) LIKE your hw policy. Has the advantage of also teaching the students some math to understand the penalty function.

      3) The stereotypical excuse used to be `my dog ate my HW' (When it really happened to Bart Simpsons he said ``I didn't know dogs could really do that'') This excuse seems to have gone out of fashion in the electronic age.

      Delete
  2. My late penalty is a continuous piecewise linear function of (submission_time - due_time): https://www.dropbox.com/s/ru0qxfns0lfueb6/late-penalty-120-plot.png

    The two pieces are a slowly-increasing late penalty for the first 12 hours, followed by a fast-increasing late penalty for the next 12 hours. For example,

    3 hours late: 2.5% off,
    6 hours late: 5% off,
    9 hours late: 7.5% off,
    12 hours late: 10% off,
    15 hours late: 32.5% off,
    18 hours late: 55% off,
    21 hours late: 77.5% off,
    24 hours late: 100% off.

    If I knew more game theory, I could prove this works, but right now it's just intuition. The continuity of the penalty function is necessary: I think any sharp cutoffs (e.g., 25% off for each full day late) introduce incentive to complain about just barely missing a point of discontinuity.

    In a class of 72 students, I had no requests for late submissions the entire term, and one student THANKED me for the policy. (!!!)

    ReplyDelete
    Replies
    1. Beautiful! Yours is very much like the one I describe in messymatters.com/deadlines

      And, yeah, a big advantage of both our schemes is eliminating the discontinuities so, as I put it in that post, there's no quibbling about what fluke technical difficulty delayed submission at some critical moment.

      Btw, my original formula didn't get to 100% penalty until a week out but you can parameterize it like so:

      Penalty for being t seconds late = 1-(t/g)^4 where g is how many seconds until the penalty should hit 100%.

      Also that exponent can be tweaked to give something very much like your function but continuously differentiable. Visually it seems to match yours pretty well with an exponent of 2.6 or so. Call it e for roundness' sake.

      https://www.dropbox.com/s/thok9ukhfa7y2qr/late-function.png?raw=1

      PS: One more advantage of my smoother function (and higher exponent) is it replicates Bill's original idea of an automatic grace period. There's the nominal (moral, as Bill calls it) deadline but you can go a couple days and still get >99% credit.

      Delete
  3. Hi dreeves:

    It seems we independently converged on the idea of a continuous late penalty!

    I actually considered something smooth like yours at first, but decided it would be easier to explain two linear pieces to the students. However, I think they should have similar incentives, especially because of the property that the slope is more gradual at first and increases at time passes.

    --Dave

    ReplyDelete
  4. I have used the following policy: assignment is due on Friday 8PM (don't waste students' weekend doing assignment), if it is late by t their mark will be multiplied by 1-(t/2 days).

    Works very well, never got any request for an extension.

    ReplyDelete
  5. I've always used the following policy:

    No late homework. The deadline is the deadline.

    But I'll drop the 3 lowest assignments. If something happens that means missing more than 3 homework assignments, come talk to me.

    ReplyDelete
    Replies
    1. I had your policy in the past, but found that, reliably as clockwork, it caused about a third of the A students to skip the last few assignments, who would then (also reliably as clockwork) bomb the related questions on the final.

      And, it seemed not to stem the flow of requests for late homework, as many students mentally translated the policy to "I'm supposed to choose three homeworks to skip, and if something bad happens I can ask for leniency on a fourth homework".

      I agree it's an equally reasonable policy (it fact yours is far MORE lenient than my current policy), but somehow it gets interpreted much differently by my students and so doesn't have a equal effect on their incentives.

      Delete
  6. RE: 3 -- I suspect students *driven by the intention* to actually hand HWs in on Tuesdays will out-perform themselves in an alternative world where the same students have no such explicit drive. Does this mean I disagree with your assumption about correlations? Who cares -- figure out how to appropriately measure this phenomenon in future courses!

    ReplyDelete
  7. As here homework marks aren't counted to the course grade, I go for: late homeworks will be marked last, the closer it comes to class the less comments I will give and the less the student will get out of the marking.

    ReplyDelete