09-15-2012, 09:33 AM,
|
|
Poor Yorek
Sergeant Major
|
Posts: 607
Threads: 51
Joined: Jun 2012
|
|
RE: Designer ranking his own scenarios?
One issue to consider, perhaps, when evaluating even a designers "play" and/or AAR, is that, in my opinion, the variance to be expected based on PG combat tables is "large" compared to the number of die rolls in even a big scenario. Simply put, a designer could play the same scenario dozens of times with the same strategy/tactics and approach and yet attain quite different results (that being said, some scenarios, even with wide variance, will end usually in the same result - albeit perhaps arrive there differently).
I'm suggesting therefore, that though a designer's play might be weighted as of more value than any single given player at large, it ought not to be weighed too heavily, that is, should not "sway" one's judgement in and of itself. Turning that around, the designer need and ought not to consider their tendered play/analysis as necessarily "canonical."
I think this is one of the greatest values of PG-HQ's AAR section as we can observe the variance for a given scenario play out over time as folks submit Actions.
|
|
09-15-2012, 02:54 PM,
|
|
zovs
Recruit
|
Posts: 42
Threads: 3
Joined: Aug 2012
|
|
RE: Designer ranking his own scenarios?
I agree with Shad on this.
At work we grade our selves and then our team lead grades us and then our peers and then we see the feedback from all three and talk about it. It's interesting to see what the individual rates himself as, his peers and his boss.
It works.
|
|
09-15-2012, 09:36 PM,
|
|
plloyd1010
First Sergeant
|
Posts: 3,474
Threads: 353
Joined: Jun 2012
|
|
RE: Designer ranking his own scenarios?
Yes, Drew, we do. I and my boss, and his boss think it is stupid. It also isn't really germane, because the audience is different. Alan & Vince get it. None of us are suggesting Jay, or any other designer, wouldn't give an honest evaluation of their work. Amongst the sea of self promoters, and with design expectations, it really is best if designers stay out of the ratings game.
BtW, that thesis, did you really think your grading the paper was about your writing & research?
|
|
09-15-2012, 10:23 PM,
|
|
zovs
Recruit
|
Posts: 42
Threads: 3
Joined: Aug 2012
|
|
RE: Designer ranking his own scenarios?
Quote: it really is best if designers stay out of the ratings game.
I have to disagree with this opinion.
Designers are also game players so there is no reason they need to stay out of a rating of a game.
Where it can get bad and negative is at say Game Squad where Designer # 1 hates Designer # 2 and they both viciously attack each other at every chance, that gets tiresome and that may only be seen on ASL places, but stating that a Designer should stay out of a ratings game is at best just silly, in my opinion.
Heck look at some of the old Generals, Moves and S&T magazines, designers did just that. It was not until F&M came out where "purists" wanted to see the ratings for players come on more.
Designers Ratings are like all things, their opinions of something, it's not gospel, and I may even disagree with them, or the opposite be so intreged that I go out and buy said product to see if its that bad/good as Mr. Soandso says it is.
|
|
09-16-2012, 12:15 AM,
|
|
Shad
General of the Army
|
Posts: 2,247
Threads: 293
Joined: May 2012
|
|
RE: Designer ranking his own scenarios?
(09-15-2012, 09:36 PM)plloyd1010 Wrote: Yes, Drew, we do. I and my boss, and his boss think it is stupid.
As an old HR guy, I suppose I hold personal evaluations (& 360 degree evaluations) in a higher regard than most.
Quote:Amongst the sea of self promoters, and with design expectations, it really is best if designers stay out of the ratings game.
To me the crux of the issue is whether it is obvious that Rater/Reviewer A is actually the person behind Game B. If there is a disconnect or some deception there then we have a problem. But I maintain that to expect or even demand silence does nothing productive.
Quote:BtW, that thesis, did you really think your grading the paper was about your writing & research?
The professor graded us wholly independently of our own evaluations. The point was to learn to critically evaluate our own performance. As you older generations are so found of pointing out, us post-1980 kids grew up in the everyone-gets-a-trophy period and are often ill-equipped for the not-niceties of The Real World.
...came for the cardboard, stayed for the camaraderie...
|
|
09-16-2012, 09:55 AM,
(This post was last modified: 09-16-2012, 09:56 AM by Matt W.)
|
|
Matt W
Lieutenant Colonel
|
Posts: 1,037
Threads: 22
Joined: Jun 2012
|
|
RE: Designer ranking his own scenarios?
I'm a bit late to the party I'm afraid but I don't feel designers rating their own work to be a problem. This is a factor of three things:
First, this is a small but active community where I have trust in the people who put in the ratings.
Second, there is the potential for substantial variation from play to play for each scenario. I have found myself really enjoying low rated scenarios and bored by highly rated scenarios but I find that the AARs are usually very helpful.
Last, we simply don't have enough plays yet to consider many of the ratings statistically valid so any input on a scenario can be useful.
I hope that someday we can have sufficient ratings where statistical validity will matter but with something shy of 4,000 plays and 2,000 AARs for a universe of around 1,700 scenarios we aren't anywhere near that point. I understand that this could bias the sample but with such a small sample at this point I see this as a theoretical problem, not a practical one.
No "minor" country left behind...
|
|
|