This phenomenon as a whole appears to be mostly a consequence of D&D's broken leveling system. Other game systems award character advancement for accomplishing goals, thus rewarding risk avoidance. But D&D awards character advancement for fighting, and therefore bypassing these kinds of risks is counterproductive.
With respect this is completely wrong, D&D awards Xp for overcoming challenges.
Disarming traps or escaping from an ambush are challenges.
You raise a point that did not escape my attention. And it would be valid if WoTC truly meant it when they said that.
While the written text of the various source books does occasionally give some lip service to the notion that "overcoming" a challenge does not necessarily mean hitting it over the head with a blunt object, the kinds of "challenges" that are listed with explicit guidance on experience rewards are limited to combat encounters, and traps.
While this or that individual GM may take it upon himself to award experience for "overcoming" a cave troll by tricking it instead of killing it, he is going against an entire gaming culture and tradition when he does so, and thus he is unlikely to award as much as the canonical value for killing it. Even if he does, what if an even more clever party manages to bypass the cave troll altogether by figuring out the route through the maze instead? What then? Why does the troll even factor into the equation? Surely the fact that he is there is only peripheral to what the party is trying to do.
(Not to mention that the other great reward in the game, magical swag, can be missed by bypassing irrelevant fights.)
By having explicit, detailed rewards for dumb, hack-and-slash play, and a vague, hand-waving sentence or two about anything else players might do instead, D&D ensures that dumb, hack-and-slash play is what you get.
This is not because players are dumb. On the contrary, it is because they are smart. They figure out quickly what is incentivised, and they do that thing. Not only that, they optimize ways to doing it, until pretty soon no one plays a master of disguise, a contortionist cat burglar, or a zealous, self-flagellating church inquisitor. They're all too busy building a Cleric3/Nonsense Chanter2/Berserker1/Choirboy Sodomizer7, because it'll allow them to persist the Abstract Game Mechanic Rape spell five levels sooner.
It's not their fault. It's the game designer's fault. You always get whatever behaviour you reward.
It even starts to influence the designers as well. Of course they're going to make books full of slightly better ways to gank things that sit around guarding piles of treasure. Because that's what their players have been conditioned to look for. Because their GMs have been conditioned to fill the world with nothing but meatsacks that lie around waiting to be slain.
D&D isn't like living an adventure, or even being in a story. It's a simulation of playing World of Warcraft. There is very little fundamental difference between the four-encounters-per-day, 14-encounters-per-level routine of D&D, and being told to kill 37 Lesser Throbbing Wombats by some stationary whackjob with an exclamation point over his head.
The problem with this kind of treadmill is not only that it's boring. It's that it is a special, insidious kind of boring, a boring that compels people to keep doing it, not because they're having any fun, but because they have been cleverly given the feeling that the fun is about to start any moment now. And they remain in that state forever, clicking away on merlocks, or rolling d20s to behead their 745th orc, not because they are enjoying themselves, but because maybe the next level they'll feel like a badass, or the next area will be interesting, or the next magical item will provide satisfaction. But it won't.
They are a rat, pressing a lever. And then they get online, and listen to other rats pontificate upon the merits of various lever-pressing techniques.
This is not to say that D&D can't be fun. It can be, and often is. But that which is fun in it has nothing to do with it being D&D as opposed to any other roleplaying game. D&D doesn't add anything to the equation, apart from broken mechanics, metagaming, and boredom.
What Sunic is advocating makes sense mathematically, but it leaves out one critical fact, a fact that exists outside of game mechanics. This is the fact that a fight, or anything else, that has been IP proofed is
boring. Boring because the outcome is not in doubt. Boring because there is no risk. Boring by design.
Sure, it's set up to win, not to have fun... but why should I have to make that choice? Trying to devise minimal-risk ways of killing people, and then building them, is
my job. I get paid to do it, because it's not always fun.
I don't get paid to play RPGs. So there's no reason why we shouldn't incentivize fun things. In fact, that's the whole point. If players were rewarded for making the game exciting, then games would be exciting.