Well, the way they handle this kind of thing over on DnDOG is to have an open list of kudoers. So when you kudo someone, you say why, and people can see who changed their kudo status. So you can see that PI is sitting there leaning on the -1 button or whatever, and if it's important to you, you can PM them and ask why.
Personally, I'm not really interested in it any way you slice it. I tend to judge people based on their ability to spell and construct a coherent sentence, rather than postcount or game-fu. I'm sure I have an internal game-fu for people, everyone does, but it's usually pretty toned-down.