Spacious Mind rating test reloaded

[DaMaBu]

That's very consistent with my casual experience with Easy 0 on the King Performance. I play some games where it doesn't make any obvious (to me) mistakes and gradually grinds me down, and I feel like it's fairly strong. Other times it makes a silly mistake and I can gradually grind it down, and I feel like it's fairly weak. Overall, I seem to win against that level around 60% of the time.

[Dave C]

Thank you Eric for the extensive work you have shared. I remember testing most of my computers after Spacious Mind introduced and shared with us his testing program. Lots of fun, and I still reference the results recorded on the long list (Excel) he created of just about every chess computer ever made.
KR,
Dave C

[spacious_mind]

Hi Everyone

It has been 5 years since the last time I posted or visited a Forum. I hope you are all doing well. I had been very busy in recent years as a result of a promotion where I ended up being the head of my business area for North America for the Spanish company I worked for. This meant that it was basically nonstop work dealing with Corporate HQ in Spain and a fairly large team that I was responsible for as well as lots of liaison with other divisions in South America. It meant that it was really all work and no time for play. In fact, the last time I had played a chess computer (not counting the last couple of weeks) was also about 5 years ago.

Anyway 2 years ago the Company sold their North American interest and I then had to spend the next 15 months in conversion work, converting our systems and processes to theirs. About a year ago after completing the conversions and not wanting to start all over again with a new company (especially since I am now at the retirement age, I retired from work. In the last 12 months I then had my house remodeled with new windows, floors, kitchen, new sunroom and painting etc. Which I had all planned to do so that I can retire in peace and hopefully enjoy life.

Regarding the email that I used for my website etc., same applied, I had barely received emails on it and it was cluttered daily with spam which I guess I collected from all the chess stuff I downloaded in the past. So, in the end I had stopped using it even though I still have it (but forgotten it's password).

Anyway, I do feel bad that I had not been in touch, but I am sure you all know how "I will do it tomorrow", turns into next week, month year and so on.

About 2-3 weeks ago I did decide to just look in on Hiarcs and Schachcomputer.Info to see if they still exist and was quite sad to see how the posts here at Hiarcs have become much fewer than they used to be. I guess it might be the sign of our times and all of us getting older and younger people seem to have other interests.

But, by chance Tibono had just posted about my rating spreadsheet that I had almost forgotten about. This sparked my interest (now that I do have leisure time) and I remembered I had an idea a few years ago about using a formula to punish weaker moves into the negative in the evaluations that should work on weak moves and not affect good ones at all.

I spent the last couple of weeks revisiting the ratings list and redoing the evaluations from scratch using Stockfish 16.1 which had just come out. This changed quite a few of the evaluations where previously some good moves are no longer that good and vice versa.

Now that I completed the new ratings, I am quite pleased with the results.



The rating lists are now maxed at 3800 elo which should keep them relevant for a few years. I went with most moves doing a full evaluation of every available move, 40 ply deep, which means that even Stockfish 16-1 at 30 seconds per move scored 3483 with room to spare.

The weaker programs as in the old test continue to do very well in test Game 1 but you can see them suffering as expected in some other test games which overall reduced their rating to quite close to their game ratings. All the programs show their strengths and weaknesses quite well.

I am in the process of developing another 5 rating spreadsheets which should increase the rating to 10 from 5 and hopefully this will even more balance out the average ratings of the chess programs.

Anyway, after all these years I had a lot of fun the past couple of weeks relearning and hopefully adding and improving these rating lists.

Here is the download link: https://www.spacious-mind.com/forum_rep ... evised.zip

By the way any old tests that you may have done in the past you can just copy and paste them into the move fields of the new sheets, and it will recalculate them to the new rating. Just remember to paste the moves as "text"

Tibono, I am curious to know your thoughts after you get a chance to paste your recent tests into these new tests.

Best regards
Nick

[Steve B]

Very Glad to see you again
Alive and well!

I feared the worst when you stopped posting and didnt answer PM's Or Emails

Welcome Back Regards
Steve

[oriental]

Hi Nick,

Very happy to read you here again, and thanks for all the work done in the chess computer community.

Maybe the average chess computer fan is older than it used to be, but younger people have joined the hobby too.

Grateful regards,

Laurent

[Tibono2]

Tibono, I am curious to know your thoughts after you get a chance to paste your recent tests into these new tests.

Hello Nick,
also very happy to read you again, and much impressed with your commitment to your job.
Of course I shall replicate my recent tests with your new worksheets, and share some feedback!
Warm regards,
Eric

[Bryan Whitby]

Hi Nick
Good to know that you are still in the "Land of the Living"

[Tibono2]

Hi Nick,

let's start with a few small technical fixes:

Code: Select all

Game 1 
G01M19W lacks moves:
19.Qb4
19.Qb6
19.Qc5

Game 2 
G09M07B drop list too short, lacks access to:
7. ... Qxf3
7. ... Rb8
G09M08B drop list too short, lacks access to:
8. ... Rb8
8. ... Rc8
8. ... Rd8

Game 4
G01M15W drop list too long, enables access to empty line
G01M25W drop list too short, lacks access to:
25.Rxa5

Game 5
G02M17B drop list too short, lacks access to:
17. ... Rhg8
G02M21B drop list too short, lacks access to:
21. ... Rh7

I use LibreOffice Calc application, not MS Excel, thus maybe the below comment is just about a compatibility issue my side:
I noticed your search formula (to retrieve move's score) ends with 1 as last argument; as a consequence any input (even unlisted ones) would retrieve a value from the data list.
Using copy/paste to feed the moves doesn't trigger the drop-list control, and can lead to un-noticed issues. I would rather use false() as the last search argument, meaning a "not found" move would result in #NA value; and so require fixing the input. If I am correct, 0 can be an alternative to false().
Just an additional consistency control, not a mandatory one.

Kind regards,
Eric

[spacious_mind]

Hi Tibono

Thanks, for spotting missing move options. I have tried to check after the analysis is complete that all the move possibilities are accounted for. But to do this quickly I have 3 laptops running at the same time so it's possible I did not catch them all when I cut off the extended moves. I use Arena as it's easier than moving my Chessbase 12 CD which needs to be in a drive.

I noticed with my Arena's that once you get to 41 ply it tends to get stuck and if you remove use the 41 ply then you have missing moves that are pasted from Arena or if Arena is currently evaluating a ply and you paste, you then have to make sure that you go back one ply and take those moves otherwise moves in that Arena was calculating are not on its last ply list when pasted.

So, if you spot any more then let me know and I will go and fix them.

Also, thanks everyone for the kind words.

Best regards
Nick

[spacious_mind]

Hi Eric

Yes the error was in the Dropdown list, I had forgotten to update it with changes.

I have uploaded the corrected sheets to the same link.

https://www.spacious-mind.com/forum_rep ... evised.zip

Regards
Nick

[kamoj]

Hello Nick,
I'm very glad too, that you are back.

I got my interest for chess computers back in the Covid-times, and then you had already "checked-out".
But I have visited your site countless of times.

So, I'm happy you are back and can receive my thanks for all you have done!
/Lars

[paulwise3]

Hi Nick,

Good to hear from you again.
And welcome to the club of retirees who are still short of time!

Best regards,
Paul

[spacious_mind]

Hello Nick,
I'm very glad too, that you are back.

I got my interest for chess computers back in the Covid-times, and then you had already "checked-out".
But I have visited your site countless of times.

So, I'm happy you are back and can receive my thanks for all you have done!
/Lars

Steve, Laurent, Eric, Bryan, Lars & Paul.

Thanks again all for welcoming me back! It is appreciated.

Lars,

My website was another bone of contention for me. Shortly before I stopped posting Yahoo sold their Webhosting services to Tubify. I then started receiving a bunch of emails on how to transfer all my work to the new company and it all sounded so complicated and time consuming that I gave up on it. In addition, I already had the problem and still have the problem that the authoring software that I used and got really good with went out of business and was no longer supported and I tried to change to some other authoring software but with none of them could I ever replicate the style and look of my site, which uses a lot of tables and those in other software were just garbage and I did not feel like having to recreate hundreds of pages without the date in the tables.

Anyway, I continued paying the monthly hosting subscriptions and was surprised that the site remained. I guess one of their tech gurus transferred it completely. Only thing that does no longer work on the site are the pgn game examples I used to have at the bottom of the pages where there is now just a green line.

Maybe when I am through with creating more tests, I will take my time and compete the Website and remove those "coming soon" by adding the content.

I see from your correspondence with Steve, that you had a lot of experience with chess computers back in the 80s/90s, that's great to know.

Best regards
Nick

[Tibono2]

Hello Nick,

I am in the process of providing you with my recent tests and I just spotted another required fix: test4, G01M45B, the dropdown list of moves looks incorrect (most probably built from a wrong position).
Should be:

Code: Select all

45. ... Ba4
45. ... Bb3
45. ... Bc2
45. ... Be2
45. ... Bxf3
45. ... Kg7
45. ... Kg8
45. ... Kh8

I let you double check.
Best,
Eric

[spacious_mind]

Hello Nick,

I am in the process of providing you with my recent tests and I just spotted another required fix: test4, G01M45B, the dropdown list of moves looks incorrect (most probably built from a wrong position).
Should be:

Code: Select all

45. ... Ba4
45. ... Bb3
45. ... Bc2
45. ... Be2
45. ... Bxf3
45. ... Kg7
45. ... Kg8
45. ... Kh8

I let you double check.
Best,
Eric

Hi Eric
Ok I reuploaded the correction. Just download the zip file again. Move 45 changed from the past and every move now has same score of 30 based on the new evaluations from SF.

Best regards
Nick