Author Topic: The BG Rating Scale is a bit odd  (Read 19707 times)

0 Members and 1 Guest are viewing this topic.

Cam_Banks

  • Curious George
  • ****
  • Posts: 325
    • Margaret Weis Productions
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #40 on: January 18, 2009, 07:53:29 PM »
You not seeing the possibility is weak claim on it's existence or rather non existence.

Right, but I'm not worried about the truth of that. I don't see the point in it, since it's not what I'm looking for. There very well may be a point to it, but it's irrelevant for my purposes.

Quote

I am far more interested in hearing about games you like than games that are "good." Obviously, we can like games that are "bad" by various metrics, and since that's apparently more relevant to enjoying things, let's go with that. I might try something you like, or I might not. I'm sure there are games that are "good" that you don't like, correct? Or is being "good" automatically going to mean you'll like it?

Cheers,
Cam
Managing Editor & Community Manager | Margaret Weis Productions

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #41 on: January 18, 2009, 08:05:50 PM »
I am far more interested in hearing about games you like than games that are "good." Obviously, we can like games that are "bad" by various metrics, and since that's apparently more relevant to enjoying things, let's go with that. I might try something you like, or I might not. I'm sure there are games that are "good" that you don't like, correct? Or is being "good" automatically going to mean you'll like it?

Have you listened to the reviews?

Our first review Cthulutech is a "bad" game I liked.  Aces and eights is a "good" game that we don't like. 

Ennies Nominees - Best Podcast 2009

Cam_Banks

  • Curious George
  • ****
  • Posts: 325
    • Margaret Weis Productions
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #42 on: January 18, 2009, 08:20:22 PM »
Have you listened to the reviews?

Our first review Cthulutech is a "bad" game I liked.  Aces and eights is a "good" game that we don't like. 

And you know what's more interesting to me is why you like Cthulhutech and why you don't like Aces and Eights. I really could care less if you think either of them is a "good" or "bad" game otherwise. Make sense?

Cheers,
Cam
Managing Editor & Community Manager | Margaret Weis Productions

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #43 on: January 19, 2009, 01:59:28 AM »
Have you listened to the reviews?

Our first review Cthulutech is a "bad" game I liked.  Aces and eights is a "good" game that we don't like. 

And you know what's more interesting to me is why you like Cthulhutech and why you don't like Aces and Eights. I really could care less if you think either of them is a "good" or "bad" game otherwise. Make sense?

While that may be interesting it is not a review, it is a list of things I like and dislike.  I like cthulhu and I dislike cowboys.
Ennies Nominees - Best Podcast 2009

Cam_Banks

  • Curious George
  • ****
  • Posts: 325
    • Margaret Weis Productions
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #44 on: January 19, 2009, 12:20:40 PM »
While that may be interesting it is not a review, it is a list of things I like and dislike.  I like cthulhu and I dislike cowboys.

Of course it's a review. We trust you to have some understanding of what you're talking about, or at least that's what you should be aiming for, but likewise I don't think anybody expects or believes you to have the power to remove your own biases formed through years of good and bad game experiences and reading shelfloads of books and somehow deliver a purely objective and fact-driven review. Tell us what you liked about the game and what you didn't like about the game based on your experiences and knowledge of games, and it's a review.

Now, if you just said "this game sucks, I hate this game" or "this game rules, you should go play it now" then we'd probably want something a little more substantive.

Cheers,
Cam
Managing Editor & Community Manager | Margaret Weis Productions

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #45 on: January 19, 2009, 12:42:56 PM »
While that may be interesting it is not a review, it is a list of things I like and dislike.  I like cthulhu and I dislike cowboys.

Of course it's a review. We trust you to have some understanding of what you're talking about, or at least that's what you should be aiming for, but likewise I don't think anybody expects or believes you to have the power to remove your own biases formed through years of good and bad game experiences and reading shelfloads of books and somehow deliver a purely objective and fact-driven review. Tell us what you liked about the game and what you didn't like about the game based on your experiences and knowledge of games, and it's a review.

Now, if you just said "this game sucks, I hate this game" or "this game rules, you should go play it now" then we'd probably want something a little more substantive.

Cheers,
Cam

Nah.  I'll just keep giving out objective unbiased reviews.  People can believe them or not.  I would rather be right and entertaining with no one "believing" me rather than just pointless and entertaining and everyone "believing" in that.  I've been down this road many times now.  If you just keep pushing the truth people will start to catch on.  The truth is one of the most powerful tools in the world.
Ennies Nominees - Best Podcast 2009

Meg

  • Message Board Extraordinaire
  • Brilliant Gameologist
  • Man in Gorilla Suit
  • *
  • Posts: 2069
  • Are you rapier than me?
    • Brilliant Gameologists
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #46 on: January 19, 2009, 12:59:51 PM »
In our Live episode I came to a realization-- we aren't really putting out reviews, we're putting out critiques.  I akin our style much more to a film or restaurant critic than to a review.

Cam and Talen- your viewpoints are appreciated and noted.  There are pieces of the feedback we'll take into consideration and pieces that we'll pass on because it doesn't support our goal and mission.

We will continue to use a numerical system- but my hope is that it's balanced with qualitative analysis which is the actual episode.  If we were going purely quantitative we'd just have a post with a game and a number attached.  But having both I'm hoping we're hitting both sides of the audience.

However, we agree that a straight "3 out of 8 lightbulbs" is far too subjective.

On the point of subjectiveness- I really get in a knot when I hear the argument Cam is making.  It's prevalent and it's so counterproductive in my view.  Total pet peeve but us saying "we're trying to be more objective with the review process" and the first reaction is "how can any review be 100% objective??  Of course my opinion plays into it!  Rar Rar Rar!  Huffle and Puffle!"  This was the whole ENnies issue and it frustrated me to no end.

More objective does not equal 100% objective?  Why go with the extremes?  I'd like to be more fit than I am now.  If I say that is your first reaction going to be "Why would you want to be a body builder?  Those women are hideous!  Why would you do that to yourself?"  More does not imply absolute.

3 out of 8 lightbulbs is too subjective.  Recommending a game for purchase is slightly more objective.  

And to think of these as a critique- of course there are objective pieces of a review.  A resturant reviewer has several criteria that they use in every restaurant to compare somewhat equally.  Except, instead of reviewing cleanliness, service, and value, we have peritextual elements, writing quality, and well, value.  The pieces more subjective in a restaurant would be food quality and environment.  We have instead "does the game meet its goal" and a review of mechanics and gameplay.  That's where our opinions come in stronger.

I'm still married to our rating scale.  I think the only adjustment that needs to be made is that of scale.  

1= the minimum amount of books you would buy in a year (that isn't zero).
3= the low average amount of books you would buy in a year
10= the most books you've bought in a year.

So if you, on average, purchase 5 books, our 1 is still a 1, our 3 is more like a 2 and our 10 equals a 5.  Or, you could look at it without adjusting for scale- and instead think, "I may only check out books that rate a 1 or a 3 because I'm not going to buy 10 books."  Our 10's are books that may be good to have, but they won't sustain game play, so only get them as long as you have other games to play throughout the year.
All of my updates are on twitter! 

This is my angry voice.  Text written in red, by me, is  an official moderator "suggestion"

Want to meet me or the other Gameologists?  Check out where we'll be on the Conventions, Meetups and Events board!

Cam_Banks

  • Curious George
  • ****
  • Posts: 325
    • Margaret Weis Productions
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #47 on: January 19, 2009, 01:37:46 PM »
Meg - Thanks. I think your explanation of things makes a lot more sense to me, especially when you alter the premise to be one of critique rather than review. You make a good point about the objectivity question. I suppose I am reacting more to Josh's delivery (framed as it is in terms of his being an objective expert in game theory and so forth) than the content, but that's a trigger. As I said earlier, I object to the "you are being deluded" implication (or outright statement) that comes with people who express enjoyment in a game that is described as being bad or worthless. I don't see any merit in pursuing that line of analysis; it just makes people defensive, angry, or stop listening altogether.

Cheers,
Cam
Managing Editor & Community Manager | Margaret Weis Productions

MilwaukeeJoe

  • Barbary Macaque at the Rock of Gibraltar
  • ***
  • Posts: 154
    • Rappan Athuk Play-By-Email
Re: The BG Rating Scale is a bit odd
« Reply #48 on: January 19, 2009, 02:07:26 PM »
I like the rating scale just the way it is. I'm not confused by it, enjoy it's uniqueness, and heck if I care if any of the BG crew wants to heavily flavor their critiques with personal opinion or not. I listen, and then I make up my own mind (and usually after listening to different reviews/critiques as well).

I'm still married to our rating scale.   

I knew it! You're married. Next you'll accidentally reveal that you have 8 kids. Mmhmm. :D

MilwaukeeJoe

Meg

  • Message Board Extraordinaire
  • Brilliant Gameologist
  • Man in Gorilla Suit
  • *
  • Posts: 2069
  • Are you rapier than me?
    • Brilliant Gameologists
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #49 on: January 19, 2009, 02:18:55 PM »

I'm still married to our rating scale.   

I knew it! You're married. Next you'll accidentally reveal that you have 8 kids. Mmhmm. :D

MilwaukeeJoe

Haha!  Geeze, how the hell would I have time for any of the shit I partake in if I had 8 kids???? 

I have 2 dogs and 2 cats- they feel like a husband and 8 kids sometimes.

Look at me being all sexist though- I live in MA.  I could have legally either a husband or wife.  Not that I would, but it's a possibility. 
All of my updates are on twitter! 

This is my angry voice.  Text written in red, by me, is  an official moderator "suggestion"

Want to meet me or the other Gameologists?  Check out where we'll be on the Conventions, Meetups and Events board!

Talen Lee

  • Bi-Curious George
  • ****
  • Posts: 447
  • Forum Ninja
Re: The BG Rating Scale is a bit odd
« Reply #50 on: January 19, 2009, 06:48:43 PM »
We will continue to use a numerical system- but my hope is that it's balanced with qualitative analysis which is the actual episode.  
If you can and do, then my entire argument is withdrawn.

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #51 on: January 20, 2009, 01:56:41 AM »
Meg - Thanks. I think your explanation of things makes a lot more sense to me, especially when you alter the premise to be one of critique rather than review. You make a good point about the objectivity question. I suppose I am reacting more to Josh's delivery (framed as it is in terms of his being an objective expert in game theory and so forth) than the content, but that's a trigger. As I said earlier, I object to the "you are being deluded" implication (or outright statement) that comes with people who express enjoyment in a game that is described as being bad or worthless. I don't see any merit in pursuing that line of analysis; it just makes people defensive, angry, or stop listening altogether.

I have pointed many times over that everyone enjoys trashy stuff now and again.  Pretending that romance novels are Shakespeare is delusion, enjoying romance novels as a fun read is living in reality. 

Ennies Nominees - Best Podcast 2009

Wordman

  • Ring-Tailed Lemur
  • **
  • Posts: 70
    • Asteroid
Re: The BG Rating Scale is a bit odd
« Reply #52 on: January 20, 2009, 01:45:21 PM »
« Last Edit: February 27, 2009, 03:09:01 AM by Wordman »
Ctrl ]

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #53 on: January 21, 2009, 12:11:45 AM »
We will continue to use a numerical system- but my hope is that it's balanced with qualitative analysis which is the actual episode... we're trying to be more objective with the review process... So if you, on average, purchase 5 books, our 1 is still a 1, our 3 is more like a 2 and our 10 equals a 5.

I'll continue to ignore the numeric bit regardless, so take what follows with a grain of salt. (I will be commenting on both the scale and "objectivity", but may it be a little rambling.)

A useful technique for a review critique to bridge the gap between the impossibility of "100% objective" and "more objective" is to recognize, and call attention, your own bias when it is relevant.

For example, you did a good job of this during your Aces & Eights review. You clearly admired the game but, equally clearly, wouldn't be playing it because it wasn't really your bag. And it is here that the current definitions for your numeric system fail. They fail because they contain the word "you" (meaning "you the listener"). I don't remember what number you gave Aces & Eights, but it seems pretty clear to me that there is no possible way that all the "you the listeners" are using even close to the same scale for it. Those that really enjoy crunchy, simulation type games will dig it, other people pretty much won't. And you said as much during the review.

But, at that point, your scale became a hindrance.

I think you'd be better off defining the scale in terms of your own bias, make your own biases clear during the review, and then trust that listeners will develop a feel for that bias and judge accordingly. That is, set the scale to mean "if we were going to buy one book" (meaning "we, the Brilliant Gameologists").

Whether you change the definition or not, listeners will be judging your bias against their own anyway. I know I do. Seems like it would be more useful to just acknowledge that up front.

Or we could just not consider our bias.  Given that we know our bias, we can remove it. 

Given the nature of RPGs it is not practical to go with a biased approach.  Again you are claiming that things like writing cannot be appraised objectively, yet they can and are in schools across the nation. 
Ennies Nominees - Best Podcast 2009

Shoggoth

  • That monkey with the orange ass cheeks
  • ****
  • Posts: 246
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #54 on: January 21, 2009, 12:27:55 AM »
Or we could just not consider our bias.  Given that we know our bias, we can remove it. 

Given the nature of RPGs it is not practical to go with a biased approach.  Again you are claiming that things like writing cannot be appraised objectively, yet they can and are in schools across the nation. 

I'm really interested in seeing these critiques.  If they're really the rigorous analytical papers that you're describing, then they'll be unprecedented in the hobby.

Still came that eldritch, mocking cry - "Tekeli-li! Tekeli-li!" and at last we remembered that the demoniac Shoggoths...had no voice save the imitated accents of their bygone masters.

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #55 on: January 21, 2009, 12:38:36 AM »
Or we could just not consider our bias.  Given that we know our bias, we can remove it. 

Given the nature of RPGs it is not practical to go with a biased approach.  Again you are claiming that things like writing cannot be appraised objectively, yet they can and are in schools across the nation. 

I'm really interested in seeing these critiques.  If they're really the rigorous analytical papers that you're describing, then they'll be unprecedented in the hobby.

We've already done many things never before done in this hobby.  For us a day where we do something revolutionary and groundbreaking is called a weekday.  We save the really special stuff for the weekend.
Ennies Nominees - Best Podcast 2009

Talen Lee

  • Bi-Curious George
  • ****
  • Posts: 447
  • Forum Ninja
Re: The BG Rating Scale is a bit odd
« Reply #56 on: January 21, 2009, 01:26:37 AM »
Or we could just not consider our bias.  Given that we know our bias, we can remove it. 

Given the nature of RPGs it is not practical to go with a biased approach.  Again you are claiming that things like writing cannot be appraised objectively, yet they can and are in schools across the nation. 

I'm really interested in seeing these critiques.  If they're really the rigorous analytical papers that you're describing, then they'll be unprecedented in the hobby.

We've already done many things never before done in this hobby.  For us a day where we do something revolutionary and groundbreaking is called a weekday.  We save the really special stuff for the weekend.
Unprecedented is not the same as useful.

Wordman

  • Ring-Tailed Lemur
  • **
  • Posts: 70
    • Asteroid
Re: The BG Rating Scale is a bit odd
« Reply #57 on: January 21, 2009, 02:19:09 AM »
« Last Edit: February 27, 2009, 03:11:00 AM by Wordman »
Ctrl ]

Josh

  • Brilliant Gameologist
  • Grape ape
  • *
  • Posts: 1835
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #58 on: January 21, 2009, 02:20:38 AM »
Or we could just not consider our bias.  Given that we know our bias, we can remove it. 

Given the nature of RPGs it is not practical to go with a biased approach.  Again you are claiming that things like writing cannot be appraised objectively, yet they can and are in schools across the nation. 

I'm really interested in seeing these critiques.  If they're really the rigorous analytical papers that you're describing, then they'll be unprecedented in the hobby.

We've already done many things never before done in this hobby.  For us a day where we do something revolutionary and groundbreaking is called a weekday.  We save the really special stuff for the weekend.
Unprecedented is not the same as useful.
Correct.  Usefulness is an additional quality it possesses.  Though usefulness is one of it's key unprecedented aspects, the first useful review(critique).  

The real question is "why all the fighting about it?"  Everyone wants to fight about objectivity.  

This is not new, I just have never figured this one out.  Say you are going to be objective and people fight.  A shiny sham-wow to the person who can tell me why.  
Ennies Nominees - Best Podcast 2009

Cam_Banks

  • Curious George
  • ****
  • Posts: 325
    • Margaret Weis Productions
    • Email
Re: The BG Rating Scale is a bit odd
« Reply #59 on: January 21, 2009, 03:32:59 AM »
This is not new, I just have never figured this one out.  Say you are going to be objective and people fight.  A shiny sham-wow to the person who can tell me why.

Because they do not believe you when you say it.

Cheers,
Cam
Managing Editor & Community Manager | Margaret Weis Productions