Certainly imagination plays a role in D&D. Even if you're not a Real Roleplayer and live for the wargame aspect of D&D, you're not actually fighting or even planning out fights, all the action is strictly imaginary (excluding dice-throwing after a bad call). The rules do serve to limit imagination, and rightly so. See, not everyone imagines the same thing at the same time, and nearly everyone is selfish to some degree or another (not a bad thing, just a survival trait). However, when you combine mismatched imaginations with selfishness in a collaborative project to imagine struggle and conflict, you eventually wind up in a little kid shouting match along the lines of "I shot you!" "No you didn't, you missed." "But I'm the best shooter in the world!" "I have super speed and can dodge arrows and bullets!" "I never miss, you're dead!" "No, I ran over to you and kicked you a hundred times before you could finish drawing your weapon, you're dead!"
I don't believe fluff should serve as a limiter, however. If it has no bearing on the mechanical outcome of a conflict (not always combat, the struggle between a puzzle's designer and the guy that needs to solve it to move on is conflict too), it's not necessary. Sure, it can serve to enhance the experience, but it can also degrade it. If the fluff is interfering with the players having a good time, it's having a negative effect, not a positive one, and it needs to butt out.