An Analysis of Chess Program playing style over the years.

This forum is for general discussions and questions, including Collectors Corner and anything to do with Computer chess.

Moderators: Harvey Williamson, Steve B, Watchman

Forum rules
This textbox is used to restore diagrams posted with the fen tag before the upgrade.
Post Reply
User avatar
scandien
Member
Posts: 206
Joined: Mon Sep 12, 2011 1:15 pm
Contact:

An Analysis of Chess Program playing style over the years.

Post by scandien »

Hello,

I have just finished a study on the style of playing of Chess machines.

Detailled informations can be found here :

http://lechiquier-orseen.pagesperso-ora ... le_en.html

My impressions:

If at the beginning machines were too low in strategy to be able to reach the level of an expert player, the programs managed to create programs which managed finally to emulate a kind of planning and of strategic vision.

From now on, the new programs play in a more well-balanced way, and either real weaknesses.
It is easier now to be able to make correspond the style of game(set,play) of machines to certain champions of past, because previously machines had a style of game(set,play) by too defensive.

Best regards

Nicolas
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Re: An Analysis of Chess Program playing style over the year

Post by spacious_mind »

scandien wrote:Hello,

I have just finished a study on the style of playing of Chess machines.

Detailled informations can be found here :

http://lechiquier-orseen.pagesperso-ora ... le_en.html

My impressions:

If at the beginning machines were too low in strategy to be able to reach the level of an expert player, the programs managed to create programs which managed finally to emulate a kind of planning and of strategic vision.

From now on, the new programs play in a more well-balanced way, and either real weaknesses.
It is easier now to be able to make correspond the style of game(set,play) of machines to certain champions of past, because previously machines had a style of game(set,play) by too defensive.

Best regards

Nicolas
Hi Niolcas

That is cool. To map out a chess program and then map a GM's style and match it in your graph. Great idea. I would never have thought Fischer's style matches a Russian program :) Smarthink 0.17 is one of my all time favorites. Would love to see it inside a Millennium 300 MHz box! :)

Best regards
Nick
User avatar
Fernando
Admiral of the Fleet
Posts: 3059
Joined: Tue Jul 31, 2007 4:35 pm
Location: Santiago de Chile

Post by Fernando »

I wonder if tactics and strategy merge into each other beyond certain threshold of analysis.
When engines went at most to 8 or 9 plies, they need certain general rules we call strategy to subsidize the lack of specific search. It is exactly what we, humans, do. But machines of old times had not that general rules. Programs were not incorporating enough rules of thumbs for that because there was another limit, memory space and speed.
Now when some machines goes easily at 30 plies in middle game, general rules are not needed. Simply the engine SEE specifically what is going to happen and so does not need a rule of thumbs or the so called strategy.

Strategy is in sense the glamorous name given to guessing and/or knowledge, which it is, also, a substitute to really KNOW.

Fern
Festina Lente
User avatar
scandien
Member
Posts: 206
Joined: Mon Sep 12, 2011 1:15 pm
Contact:

Post by scandien »

hello fernando,

all the program/machines run the Khmelnitsky test. And even for top level we can really see that some engine are using positional knowledge to select their move, whileother are really tactitian , ( this is this skill there are using the most) . Their profile really differ from each other.

Obviously if you consider Stockfish or Komodo ,their overal level is very high and the test only shows (in my view) engine's preference ( and i cannot consider this ass a weakness).


this is just a try to have a "scientist" approach of the engine style .

best regards

Nicolas
User avatar
Fernando
Admiral of the Fleet
Posts: 3059
Joined: Tue Jul 31, 2007 4:35 pm
Location: Santiago de Chile

Post by Fernando »

scandien wrote:hello fernando,

all the program/machines run the Khmelnitsky test. And even for top level we can really see that some engine are using positional knowledge to select their move, whileother are really tactitian , ( this is this skill there are using the most) . Their profile really differ from each other.

Obviously if you consider Stockfish or Komodo ,their overal level is very high and the test only shows (in my view) engine's preference ( and i cannot consider this ass a weakness).


this is just a try to have a "scientist" approach of the engine style .

best regards

Sure they still use it. Why not. It is of support anyway. Just saying that deeper and deeper search push beyond and beyond the need to recur to rules of thumbs as strategy is at the bottom.
Playing Sargon V regards
Fern

Nicolas
Festina Lente
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Post by spacious_mind »

Hi Nicolas,

OK I played around a bit with your test and tested AM68060 Sargon 3. I also plotted it and compared it with programs you tested that scored almost as high. Here is a graph:

Image

And here is another graph of some more you tested:

Image

You can see that the programs in the first graph are much more balanced than the programs in the second graph.

Best regards
Nick
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Post by spacious_mind »

Hi Nicolas,

Here is another chart where I am try to show a bit of evolution with Stockfish as best and Rebell 5.0 included as weakest tested by you.

Image

I am a little surprised that Stockfish did not do better at pening, Middle Game and Strategy, but otherwise you can see almost a perfect circle from Stockfish as it should be.

Also your list does not show which Stockfish, Houdini and Komodo was used.

Best regards
Nick
User avatar
scandien
Member
Posts: 206
Joined: Mon Sep 12, 2011 1:15 pm
Contact:

Post by scandien »

the interestiong data are tuned/balanced data. The brut data are adjust to average 50. .

WHen you see a weakness in this test, working ion this weakness is the best way to improve your game.

So a perfect circle indicate a player with no weakness ( if he has a very high Rating , you can tell he is the perfect player)

Chess Champion will have no real weakness , but will prefer to play some move rather than other (tal prefer sacrifice while fischer won't )

the version :

Stockfish 6
Komodo 3
Houdini 1.5


best regards

Nicolas
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Post by spacious_mind »

Hi Nicholas,

If you take the graph below:

Image

What is throwing me a bit is how Sargon 3 performs defensive better than Genius and Genius performs attack better than Sargon 3.

Can you can explain the criterias you used as I am not sure if these criterias should be reversed?

For example:

Image

I would consider Test game 4 a passive defensive game and the above reflects in my mind Sargon 3's difficulty in this type of game. Whereas games 1, 2 and 3 I would consider to be more attacking and Sargon 3 scores well in these.

So I am a little unsure what to make of the results when comparing charts.

Best regards
Nick
User avatar
scandien
Member
Posts: 206
Joined: Mon Sep 12, 2011 1:15 pm
Contact:

Post by scandien »

hello

First i would like to know if in your graph is using brut or tunned data ? looking at the data i think those are brut datas.

then :

the brut data indicate the real level of the player/machine under test. The original goal of this test was to identify the weakness and the strength of the player. In khmelnitsky book , the master indicates some ways of improvement . The author considers that one should work on his weakness for better improvement.


I am using the test diffently for the machine (obviously the machine cannot use this test to work on weakness or to improve). My feeling is that the tests are showing on which skill the playing strength machine rely. So i have tuned the value to get a mean of 50. This can give me a graph which can be compared between all machines.

I suppose that if a machine/program is better in attack than in defense, it would be seen as a fierce attacker (launching attack even if it has to weaken is own position).
If a machine is better in defense and counter attack but have a low level in attack ( long term skill) the it would probably be seen as passive ( awaiting you to launch an attack and then take advantage of your faults).


looking at
http://lechiquier-orseen.pagesperso-ora ... test.xhtml
and searching for Chess Genius you can see that Chess Genius program is good in each category , and it has great skill to launch attack ( ie it is able to detect your weakness even if you are not really attacking it) . It is like when you play versus a Chess Master like Karpov : you can fell confident n your position, and then you see that you will lose because Chess Genius get the best.

If you look at the roma data then you can see that the roma is far from been a good attacker. It's main skill is defense, counter attack and threat detection. Playing versus the Roma ( the engine are relative) requires to be patient, because the roma will play a good positional game, but will wait yours attacks, and rely on his defensive skill to defeat you! If you don't launch any attack and play quietly, probably you will get a draw. This is has playing versus Smyslov or Pretrossian.


Now another point Sargon is better in tactics than in strategy ( like many dedicated machine) , and it is very good in calculation which mean that it is quite better in complex position. I didn't check other game , but i remember that the first one was a tactical game ( am i right ?).

i have run the test fr Designer Mach IV (Master 2325) too and this show a quite defensive style (a tactical player better in defense than attack).

Finally I have the feeling that as the processor power increase the strategic skill ( which need deep calculation) increase too. So it could be interesting to test Sargon 3 with a "lower" emulated processor and see what happens.

best regards

Nicolas
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Post by spacious_mind »

Hi Nicholas,

Thanks for your response. Yes the chart is what you would call "brut".

I think I am going to have to buy the book myself to be able to understand the ideas behind it better.

Yes, test game 1 is tactical.

Image

So what throws me is that for example King 2.2 Defensive scores highest in Game 1 and lowest in Game 4. All the old Lang's score bad in game 4.

King 2.2 Defensive, the word defensive is not exactly what I would use on Sargon 3 for example. But then again my thoughts might be all wrong.

It almost makes me feel that I would have to put one or more criteria behind each and every move played to more accurately assess a test game and seperate white play and black play into these decisions that you are using move by move:

Endgame, Middle Game, Theoric Endgame, Opening, Calculation, Strategy, Tactic, Threat, Attack, Counter-Attack, Defense, Sacrifice

Best regards
Nick
royb
Member
Posts: 20
Joined: Sun May 15, 2016 4:30 pm

Post by royb »

Fernando wrote:I wonder if tactics and strategy merge into each other beyond certain threshold of analysis.
When engines went at most to 8 or 9 plies, they need certain general rules we call strategy to subsidize the lack of specific search. It is exactly what we, humans, do. But machines of old times had not that general rules. Programs were not incorporating enough rules of thumbs for that because there was another limit, memory space and speed.
Now when some machines goes easily at 30 plies in middle game, general rules are not needed. Simply the engine SEE specifically what is going to happen and so does not need a rule of thumbs or the so called strategy.

Strategy is in sense the glamorous name given to guessing and/or knowledge, which it is, also, a substitute to really KNOW.

Fern
Fern, you and I agree. I've said this for decades -- what we call strategy is really a set of guidelines/rules that humans created to help steer us in a better direction due to our lack of ability to see deeply enough into every position.
kosterix
Member
Posts: 81
Joined: Mon Aug 07, 2017 11:35 am

Post by kosterix »

I'm not sure we are really allowed to comment on godlike machines play styles.
Strive for avoiding anything flat that emits light. Love the unhackable chess computer with real pieces and a tactile feel. I do not collect, I live.
Post Reply