CCRL update (9th October 2009)

[Graham Banks]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040.live/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are updated separately to each other.
Also please note the live link will no longer necessarily give you the most updated lists. The links given in each update report will be the ones to use.

The links to the various rating lists can be found just beneath the default Best Versions list.
For example there is a 32-bit Single CPU list.

Our 40 moves in 40 minutes repeating and 40 moves in 4 minutes repeating are both adjusted to the AMD64 X2 4600+ (2.4GHz).

Currently active testers are:
Graham Banks, Ray Banks (FRC only), Shaun Brewer, Aser Huerga, Kirill Kryukov, Dom Leste, Wassim Saeed, Andreas Schwartmann, Charles Smith and Gabor Szots.

Be aware that in the early stages of testing, an engine's rating can often fluctuate a lot.
It is strongly advised to look at the many other rating lists available in order to get a more accurate overall picture of an engine's rating relative to others.


40/40 Notes

There are currently over 208,000 games in our 40/40 database.


4CPU 64-bit Engines

Although Rybka 3 is still number one, its lead could yet be cut a little by Deep Shredder 12, which is currently neck and neck with Naum 4 in the early stages of testing.
Stockfish 1.5.1 (also in the early stages of testing) looks as though it might move into fourth spot ahead of Deep Fritz 11. This would be a marvellous achievement by those working to improve this free open source engine.
Zappa Mexico II has fallen off the pace, but still has an edge over Thinker 5.4c Inert and Deep Sjeng WC2008.
Hiarcs 12.1, Toga II 1.4.1SE and Bright 0.4a comprise the next group and are very close in strength. However, we're already talking over 200 elo adrift of Rybka 3.
Other well tested latest versions (in order of strength) are Loop 13.6, Deep Junior 10, Crafty 23.0, Scorpio 2.1 and Pharaon 3.5.1.
The strong Toga derivatives, Grapefruit 1.0, the Cyclone xTremes and TheMadPrune 1.1.25 have not been tested in this category.


The relative ratings of the 2CPU engines that have been well tested are pretty much the same as their 4CPU counterparts.


Single CPU Engines

Rybka 3 is almost 100 elo stronger than second placed Naum 4, which now has strong competition from Shredder 12 for second spot.
Thinker 5.4c Inert's hold on fourth spot looks likely to be threatened by both Stockfish 1.5.1 (which is still in the early stages of testing) and Fritz 12 (untested as yet).
Zappa Mexico II seems destined to be on its own a little further back in sixth place.
Deep Sjeng WC2008 and Grapefruit 1.0 come in next, narrowly ahead of Toga II 1.4.1SE, the Cyclone xTreme settings, Onno 1.1.1, Hiarcs 12.1 and TheMadPrune 1.1.25. Again, we're already talking more than 200 elo behind Rybka 3.
Fruit 2.3.1, Loop 13.6 and Bright 0.4a are the next group of engines, ahead of Ktulu 9, Spike 1.2 Turin and Junior 10. The new version of Twisted Logic looks likely to wedge itself inbetween these two groups.
Other reasonably recent commercial engines, SmarThink 1.10, Chess Tiger 2007.1 and the top Chessmaster 11 settings, are in a group that includes Frenzee Feb08, Booot 4.15.0 and Delfi 5.4.


Free Single CPU Engines

Rybka 2.3.2a is the the top free engine, but watch out in the coming weeks for Stockfish 1.5.1 which looks likely to move ahead of Thinker 5.4c Inert into second spot.
Grapefruit 1.0, Toga II 1.4.1SE and the Cyclone xTremes are a little further back.
TheMadPrune 1.1.25 bridges the gap between the previous group and the next, which includes Fruit 2.3.1 and Bright 0.4a. It is predicted that Twisted Logic 20090922 is likely to be in the mix here after more testing.
Spike 1.2 Turin is further back still, but with a clear edge over Naum 2.0, Frenzee Feb08, Booot 4.15.0 and Delfi 5.4.

CCRL tests a wide range of free engines, ranging right down to the 2000 elo level. The intention is to get well over 200 games for each of these engines.
Tournaments involving these engines can be followed in our public forum.


Blitz Notes

There are over 475,000 games in the 40/4 database and it is well worth a visit.
Shaun, Gabor, Kirill and Aser put a lot of work into this list, testing engines in a very well organised and systematic manner. Well worth a look!
Updates here are less regular at present due to Shaun being extremely busy with other matters. However, an update is coming soon.


FRC Notes

There are currently almost 60,000 games in the database and it is almost completely up to date, with only a couple of latest versions not tested yet.
Ray has recently tested one of the Cyclone xTreme settings, and will be testing Stockfish 1.5.1 plus Tornado 3.21a as his time allows over the coming weeks. Due to financial constraints, he doesn't own Shredder 12, so it might miss out on FRC testing unless one of the other testers is prepared to help out.

Thanks to the much appreciated efforts of Matthias Gemuh, ChessGUI is now able to be used to test all FRC engines, both Winboard/UCI plus Shredder/Arena specific.

Rybka 3 has a big lead over Naum 4, which in turn has a similarly big lead over the evenly matched Deep Sjeng WC2008 and Shredder 11. This pecking order is of course likely to change once Stockfish 1.5.1 and Shredder 12 (?) get tested.
Hiarcs 12.1 and Cyclone xTreme Fear are further back.

For FRC the best list to look at is the pure list.
http://www.computerchess.org.uk/ccrl/404FRC/


Stats/Presentation Notes

The LOS (likelihood of superiority) stats to the right hand side of each rating list tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

All games are available for download by engine, by month or by ECO code not from the live link though).
ELO ratings are now saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.

For any testers interested in joining our group, please read our homepage before applying.