Alright wonderful. People are interested!
As far as people being busy, I'm going to be busy too with my summer school starting on this Tuesday. However, there's no need for me to be in charge , we can make a list of tests to do and then resolve them together.
We need to start out with some base rules on how we're going to do our test. There are several factors that affect the accuracy of a playtest
1. # of Characters in Party
2. Diversity of Characters
3. Levels to Playtest (ex. 1,5,8,13,18)
4. Diversity of Encounters
5. Number of Successive Encounters
6. Diversity of Terrain
7. Available Magic Items
8. # of times to conduct a battle
It seems to me that to do an effective playtest we need to remove some of the factors. If we tested 3 different types of terrain for 3 types of encounters for 3 different character combinations for 5 different levels 3 times (for a more accurate test) that would be 405 tries which is a bit much.
My idea is to use the following choices
1. 1, 2, 4 or 5 characters
The problem with more than 1 character is that it's hard to statistically justify that one character is better than another against monsters.
The problem with 1 character is the game has a plethora of supportive powers which would become unhelpful in a playtest.
2. All parties are the same character ( 4 Fighters, 4 Wizards, etc.) or All parties are the same character with 1 control character (3 Paladins 1 Cleric, 3 Rangers 1 Cleric, etc.)
3. Do tests for Level 1, 5, 11. Start with lower levels and then proceed to higher levels. Other level tests can be conducted if people are still interested
4. 3 different types of encounters
5. 3 successive encounters with a 5 minute rest in-between (to simulate daily power loss)
6. Open Field or Open Field with 2, 2x2 rocks for cover and rogue hiding
7. Minimal, one level-appropriate enhancement weapon and one enhancement armor item with no daily or encounter powers.
8. Conduct each encounter 3 times, maybe a critical, miss, etc. tainted the results
With this way in place we'd test a party for 3 battles 3 times each. This would be 9 battles per party per level to test against another one. Tell me what the rest of you all think.