(UPDATED 1/29/07 to feature a less asstastic pic.)
Retro games make use of settings, mechanics, or assumptions that are now considered outdated by the rpg mainstream, but these same assumptions once defined that mainstream. Does a game have classes and/or levels for no damn reason? That's a retro design element. Do you have a group of PCs going on ill-defined adventures with little or no motivation? That's another hallmark of retro design. These games may be old, or intentionally designed with an old school vibe.
Stupid games do not take themselves seriously. PCs can die horrible deaths and other grim stuff, but that horrible death might involve being eaten by a horde of zombie chickens. Big dice charts, such as wandering monsters or random mutations, are often associated with this style of game. Most comedy games fall into this category, as do all the good versions of Gamma World. The new editions of Paranoia and WFRP look pretty Stupid to me as well.
Pretentious games know that they are better than Retro or Stupid games. They're not just games, they're cutting edge explorations of blah, blah, blah. I'm sure you can all list an armload of Pretentious games: Vampire, most of the rest of the World of Darkness, Nobilis, various Forge-inspired designs. I'd also put HERO and GURPS into this category as examples of mechanically Pretentious games. You can have setting-based Pretension as well: Blue [Rose, Planet], Tekumel, Harn, etc.
In addition to more a more useful categorization schema, my theory has another leg up on Edwards's: incoherent play is better. You don't need laser-like focus on Stupidity, Pretention, or Retro-osity. Retro play is fun. Stupid play is fun. But Retro Stupid play is totally awesome. Case in point: Encounter Critical. Similarly, Vampire is plain ol' Pretentious, but add some Stupid to the mix and you've got Exalted or Street Fighter: The Storytelling Game. And how about Retro Pretension? I've got 2 words for you: Classic Traveller.
The big hole in my theory is that black area in the chart where Retro Stupid Pretention is found. That ought to be the most totally rad place in the universe, but right at this moment the only game I can think of that definitely goes there is World of Synnibarr. But that game is at its best when drifted to Retro Stupid play. Which is really easy to do. The trick is to not be Raven c.s. McCracken. I suppose one could argue that Dungeons & Dragons fits the requirements to be Retro Stupidly Pretentious. But every RPG theory fails the moment D&D is taken into consideration, so I probably ought to quit while I'm ahead.