SSDF margin of error

Designed for posting all types of tournaments and Games (e.g. Man vs. Machine, Computer vs. Computer and basement matches.)

Moderators: Harvey Williamson, Watchman

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Monsieur Plastique
Senior Member
Posts: 1014
Joined: Thu Jul 03, 2008 9:53 am
Location: On top of a hill in eastern Australia

SSDF margin of error

Post by Monsieur Plastique »

Hi,

Does the SSDF use ELOStat to come up with the rating lists, or do they use, for example, their own proprietary software?

The reason I am curious is that when I run my own pgn files (private computer versus computer games) through ELOStat, I get much lower margins of error (both plus and minus) than I see with the SSDF results. Although I am not certain how these margins of error are actually calculated, I imagine that apart from higher numbers of games reducing the error, the closer the opponents are in strength to a given program, the more reliable the rating might be and thus the margins of error are reduced (certainly in computer chess).

But I am wondering if the formulas used by the SSDF are tweaked in some way to account for the results being for computer versus computer games as opposed to human versus human games, given the consistently larger margins of error?

For example, I rated one of my machines at around 1914 ELO as per the current SSDF rating list, but it has only played 86 tournament games. The average level of opposition was slightly lower than this rating. Therefore, the margin of error should be quite substantial, but ELOstat tells me it is only minus 69 points. I think it should be higher than that - more like around minus 95 points or even more.

So if the SSDF adopts a different formula, I would really appreciate knowing what formula is applied if possible so that I can do the same when calculating my own private ratings.

Also, in a similar vein, does the SSDF have some sort of policy or procedure as regards the relative strength of opponents allowed in order to achieve a published rating? For example, back in the very old days, the USCF would run preliminary rating tests to obtain a very rough rating estimate. Based upon that estimate, the official rating process was only carried out against opponents within a certain limited range above and below that rating.

I do see that on average the strength of opposition is kept reasonably close with the SSDF rating list, however this does not tell me what the actual maximum / minimum range of opponents was - only the average.

Don't get me wrong, I am not trying to be critical. Infact the SSDF work has my utmost respect and it is the only ratings list that I refer to when trying to rate my own machines. And it is exactly for that reason that I would like to get a better understanding of the process so that I can privately rate my unrated machines to the same standards.

Thank you
Chess is like painting the Mona Lisa whilst walking through a minefield.
Christophermers
Member
Posts: 11
Joined: Fri Mar 20, 2015 12:02 pm

SSDF margin of error

Post by Christophermers »

Between us speaking, I would ask the help for users of this forum.
Post Reply