D&D tried horror element rules. They are... underwhelming.
Cthulu tries horror elements. They aren't really any better. Sanity checks? Tell that to some of the poor sods in Iraq.... Honestly a mechanism trying to force the character into a genre is just poor. Either the horror will work at the table, or it won't. Now if every player could decide for HIMSELF when to roll a sanity check, THAT's a mechanism that would represent the quality of the gameplay.
I DO have a basic issue with needing rules and mechanics for everything: You need rules and mechanics for stuff where you want to direct a certain AND/OR a random outcome on a situation.
Take character flaws: You might roll on your Avarice flaw when you wish to determine whether you character can spare tuppence for the begger or not. Or Greed to decide whether the character will betray his friends for money. But FORCING those checks just means you are taking control of the character away from the player, and forcing him to play in a certain way. That, IMHO, is a bad mechanism. As is shaping a character around his flaws, instead of shaping the flaws around the character. Still, these kinds of rolls can enhance the roleplaying experience if the players are willing to run with them, as long as they remain organic and flexible.
Some mechanics behave contrarily to that, by forcing players to have their characters act in certain ways. These mechanics are bad, IMHO, because they do not enhance play, they limit it. They also probably won't feel like they organically integrate into the gameworld, rather, they might attempt to force the gameworld into a certain direction. Like the sanity checks. Face it, tentacled monsters (or whatever) will never be truly horrific, unless you MAKE them horrific, as a GM. If you do, then that's your horror. A botched sanity check won't make me stop laughing at the critter if its just ridiculous.
I recently provoked the GM who started running Cthulu for us by setting up my character as a classical shoot-em-up hero. I rolled good stats, took mostly combat skills, got a fat gun and had him set up nicely as a 20s prohibition gangster. Instead of doing stupid rituals and exorcisms, he just wanted to burn down the house with the monster. When that wouldn't work (due to... well... rain) he just talked, completely in character, about blowing it up with dynamite. At that point the GM just basically packed up in a huff, because "Cthulu just doesn't work that way, and you CAN'T run about talking about blowing shit up and shooting stuff." Hell, it does, if you want it to. The character just can't KNOW that the monster isn't hurt by either fire nor dynamite. The GM however then wanted us to then reroll characters who basically couldn't have any combat capabilities, nor own any weapons.
So I agree with Josh that a game that purports to be certain genre, but doesn't have a mechanism for its main schtick is probably a poor game - or at least it won't work as what it tries to be. However, a poor mechanism is no better than none at all, and quite possibly that much more frustrating to handle. You can still do most things without rules and have fun, you probably won't have fun doing them with bad rules.