I could do several posts on rankings but let us focus on rankings of Ph.D. programs in computer science. One cannot give a linear ordering of departments: One department might have a strong theory group and another strength in systems. Even within theory, one department could have strong algorithms while another group has strong complexity. Even then one's graduate experience depends more on their individual advisor than the department as a whole.
Nevertheless Americans like statistics and ordering things including CS departments. There are two major rankings of Ph.D. programs: The US News and World Report ranking last updated in 2002 and uses a methodology of giving questionnaires to department chairs and directors of graduate study. The NRC ranking looks at a slew of different statistics but hasn't been updated since 1993. I hear rumors that the NRC will do a new ranking soon.
The US News ranking captures perception of strength instead of strength itself, often based one's opinion of a department from a few years back. On the other hand, the purpose of rankings are perception as we wish to be perceived as a strong department. We all complain about the rankings but they affect us greatly, in recruiting students (Americans especially use the rankings to choose a Ph.D. program) and recruiting faculty. When hiring faculty we often rightly or wrongly give extra weight to Ph.D.s from higher-ranked departments. Deans and provosts use the rankings to allocate resources as higher rankings lead to more prestige for the university as well.
For example, if a mid-level department wishes to have the strongest quality faculty they should hire mostly in one area, as people like to join groups with people they know, respect and can work with. But this approach won't help much in the rankings so most departments try for a broad faculty with likely lower overall quality but a better ranking.
At least forty departments have a stated goal of being a top-ten department. The pigeonhole principle guarantees many won't end up happy.