Wednesday, February 27, 2008

When good recommender systems go bad!

Lance brought up the question of amazon giving bad or odd here. The oddest pointer to an item I might want is as follows.

I bought a novelty CD by a group called Throwing Toasters (it was very good). I then was asked by amazone Do you want to buy a toaster? and got a pointer to home applicances.

Why did it make that association? Asking various computer people I got three answers. The only thing they had in common was that everyone was sure they were right.
  1. When many people buy an item the amazon software can do a reasonable data mining and make intelligent guesses about what groups of items go together. However, since this item was probably not bought by many people, it does a random word search algorithm instead.
  2. Of the few few people who bought this item before, some of them did buy toasters as well! So if only 4 other people bought this item, but 1 of them also bought a toaster, then WOW- 1/4 of all people who bought this item bought toasters, so its worth inquiring if the current purchaser wants one.
  3. Its a programming bug.
I don't know which one it is, but I'd like to know.


  1. what are the odds that someone bought a CD from that obscure band and a toaster?

  2. If it's a content-based recommender (which I think it is, in part), it probably has a feature for words matching in the item names. I'm inclined to think that data sparsity or reason #3 is the culprit.

    I've gotten similar bizarre recommendations, like sexy little red riding hood costumes for Halloween -- I guess because I read fantasy books. Unfortunately, the costume didn't fit because it was made for women, but they have a good return policy, so..

  3. Obviously, it recommended the toaster because you're gay.

  4. 4. A design bug

    It's a combination of 1 and 4

  5. Amazon might also be using information about how people have navigated their site in the past.

    Perhaps people who were searching for "toasters" ended up simply visiting the Throwing Toasters page and then eventually bought a toaster.

    Alternatively, this could be how a significant number of people who actually bought the Throwing Toasters CD happen to find out about the album. A bunch of people probably search amazon for toasters, perhaps a few made an impulse CD purchase en route.

  6. I remember when I did a search on Google for ‘quarks’. The first entry in the search results (in the commercial section) was another google search under the title “Would you like to buy quarks?” This indicates that that there are several different types of search being done and returned as a unified result set. They have since tuned their searches so these types of results aren’t returned anymore.

    Several different types of search that can be done for a company like amazon are:

    1. Products that were bought by other people who looked at or bought the same product. (requires no maintenance other than tuning the search algorithm)
    2, Products in the same category (requires exhaustively cataloging each product)
    3, Key word search (requires a lot of tuning to make sure that bogus or strange results aren’t returned)

    I believe that #3 above is the most likely culprit in suggesting a toaster for someone who ordered a CD whose only relationship to a toaster is a word in the title or a song.

  7. Hi, I work on the Recommendations team at Amazon. Can you be more specific about a) what exact CD it was, and b) where on the site you saw the recommendation (and even better, what the heading of that section was?).

    My guess is that you typed in a search query "Toasters", and later you saw a widget on the homepage that made recommendations solely based on your search history.