Why is it so hard for comps to play like people?

This forum is for general discussions and questions, including Collectors Corner and anything to do with Computer chess.

Moderators: Harvey Williamson, Steve B, Watchman

Forum rules
This textbox is used to restore diagrams posted with the fen tag before the upgrade.
Reinfeld
Member
Posts: 486
Joined: Thu Feb 17, 2011 3:54 am
Location: Tacoma, WA

Why is it so hard for comps to play like people?

Post by Reinfeld »

Friends,

This question plagues me lately. I've been spending time playing real humans at a local club, and having a great time of it - but I'm struck anew by the failure of programmers to find that sweet spot that makes a program play like a human.

By my own reckoning, my practical live rating hovers in the 1500-1600 zone, with occasional leaps upward on a good day, especially after I've forced myself to study endgames and tactical patterns. In club play, this means I wipe out the dabblers, play even with a couple of solid veterans, and get smashed by the guys who know their tactics well.

Even the better players have blind spots. I recently scored a sneaky victory over a 2000-rated player. I won the exchange in the middlegame, and held on like grim death through an endgame as he gradually reduced my advantage.

How does this relate to dedicateds and programs that can't play like people? Bear with me - I'm getting there.

Real players have habits. They have opening repertoires they will play every time if allowed. They are correspondingly weaker in systems they do not know. They prefer some phases of the game more than others. They make bad plans. They overlook counterpunches. They flinch when they see moves they don't expect.

That's not what comps do. Tune them down, and the most likely scenario is simply blundering a piece early on, typically a Knight blowing itself to bits for a KBP, followed by a dull grind and suddenly smart defensive play. Another annoying tendency of programmed personalities: burn up all the time, then play instant and accurate moves when the clock runs down.

These are the patterns I've seen when playing Chessmaster's "personalities," the best attempt to build human-style play in my experience - though it's not terribly satisfactory. I notice those programmed personalities will reflect tendencies ("favors knights," or "likes Sicilian," for example). Yet even that is not so great.

When it comes to dedicateds, the tuning opportunities are more limited. You can select aggressive, or passive, or simply limit the horizon. Those options don't capture the human effect, however. They don't get to the zones I'm talking about, i.e., bad plans, psychological breakdowns, playing more weakly after a setback, etc.

Has anyone found answers in this area? I know the conventional wisdom gives Kittinger programs credit for human style, and I agree to a point - you get pleasingly unsound sacrifices, for instance. But what have the rest of you done (if you've tried at all) to persuade machines to play like people?

- R.
"You have, let us say, a promising politician, a rising artist that you wish to destroy. Dagger or bomb are archaic and unreliable - but teach him, inoculate him with chess."
– H.G. Wells
User avatar
BenRedic
Member
Posts: 163
Joined: Mon Jan 09, 2017 11:29 am
Location: Norway

Re: Why is it so hard for comps to play like people?

Post by BenRedic »

This is all an interesting question.

I have one approach that I have been looking a little bit at recently. A couple of years ago a student (Matthew Lai) tried to apply machine learning to computer chess. The result was a program that could basically teach itself master level chess within three days. Lai would go on to be headhunted by Google for the DeepMind project, where he became involved with the AlphaGo program which went on to beat the worlds top go player. See https://motherboard.vice.com/en_us/arti ... atthew-lai

Now, Giraffe is not the strongest chess engine in the world. But that's not the point. The point is it takes a different approach to learning chess. One that we can mimic. The engine, the source and the training documentation is all freely available. So I downloaded all the necessary files and set up an environment in which I could train my very own Giraffe. But instead of letting it train for the longest time possible for the strongest possible play, I grabbed snapshots for different steps of this training. I then set these up against each other, and against other chess engines. The results are, I think, are interesting.

After the first round of training, Giraffe has a style that is very similar to a rank beginner: It has no idea how to develop pieces, so it pushes pawns, brings out the queen, knights to the edge of the board, even starts moving the king around. Usually does not castle. The only saving grace is that it's clearly stronger tactically than a rank beginner, but the style and piece development certainly screams "n00b!"

An example:

Code: Select all

[White "Giraffe 20161023 64-bit [t1]"]
[Black "Zotron 4.4.6"]
[Result "0-1"]
[ECO "D00"]
[Opening "Queen's Pawn Game"]
[TimeControl "40/180:40/180:40/180"]
[Termination "normal"]
[PlyCount "58"]

1.d4 d5 2.Qd3 e6 3.Nh3 Nf6 4.Nc3 Nc6 5.a3 e5 6.dxe5 Nxe5 7.Qg3 Qe7 8.b4 d4 9.Na4 Ne4 10.Qf4 Ng6 11.Qf3 Rb8 12.g3 b5 13.Nb2 Bb7 14.Qd3 Nd6 15.f3 Ne5 16.Qb3 Nxf3+ 17.Kd1 Ne4 18.Na4 Rd8 19.g4 Nc3+ 20.Nxc3 dxc3+ 21.Bd2 Qe3 22.Kc1 Nxd2 23.Qxf7+ Kxf7 24.Ng5+ Qxg5 25.e3 Qxe3 26.Be2 Bxh1 27.g5 Nb3+ 28.Kb1 Qc1+ 29.Ka2 Qxc2# 0-1
Let it train itself for a while, and the picture changes.

Code: Select all

[White "Giraffe 20161023 64-bit [t25]"]
[Black "Waxman 2010"]
[Result "1-0"]
[ECO "A22"]
[Opening "English"]
[TimeControl "40/180:40/180:40/180"]
[Termination "normal"]
[PlyCount "72"]

1.c4 Nf6 2.Nc3 e5 3.e3 Bb4 4.Nge2 d6 5.a3 Bxc3 6.Nxc3 Bg4 7.f3 Bd7 8.Bd3 O-O 9.O-O Be6 10.Be2 Nbd7 11.f4 exf4 12.d4 Nb6 13.d5 Bd7 14.Rxf4 Qe7 15.a4 a5 16.e4 Rfe8 17.Qf1 Qe5 18.Be3 Nh5 19.Rh4 Nf6 20.Qf2 Bc8 21.Rf4 Nbd7 22.Rf1 Qe7 23.Qg3 Kh8 24.Rh4 b6 25.h3 Rb8 26.Bd4 Ne5 27.Nb5 Ng6 28.Rxf6 gxf6 29.Qg5 Kg8 30.Bxf6 Qf8 31.Bh5 Rb7 32.Nd4 Bd7 33.Nf5 Bxf5 34.exf5 Re5 35.Bxg6 Re1+ 36.Kf2 fxg6 
    {Resigns}
1-0
Now it develops properly, before mounting a kingside attack against the kingside.

I am thinking that maybe combining the approach I have taken of limiting the machine learning with limiting the tactical abilities by reducing the search capabilities could produce something like the chess player you are looking for.

Some more links:
- The report Matthew Lai wrote about his engine https://arxiv.org/abs/1509.01549
- Giraffe source code and information about building and traning: https://bitbucket.org/waterreaction/giraffe

I can also recommend taking a look at the blog of Brendan Norman at http://chessncognac.com/. He has a lot of articles about tuning engines to play like humans.[/url]
Thank you for an interesting game.
User avatar
Bryan Whitby
Senior Member
Posts: 1001
Joined: Wed Feb 18, 2009 9:57 pm
Location: England

Post by Bryan Whitby »

Thank you Ben!
Great reading in the links.
Bryan
Volodymyr
Member
Posts: 141
Joined: Sat Apr 08, 2017 1:03 pm
Location: Ukraine,Radyvyliv

Post by Volodymyr »

Hi Reinfeld
Everything is there, but you need to try to play.
In free access more than 1000 engines.Against which you played?

http://www.computerchess.org.uk/ccrl/404/
I'm sure you'll find some interesting engines for your level.

New chess engine Sabrina,around 1500.I liked it, very easy.
http://www.linformatica.com/

My game number 2,Blitz 15 min.In the first game, I yawned the figure, and lost.

[Event "Computer chess game"]
[Site "MYCOMP"]
[Date "2017.06.03"]
[White "Player"]
[Black "Sabrina_3_0_21"]
[Result "*"]
[PlyCount "97"]
[TimeControl "900"]

1. e4 d5 2. exd5 Nf6 3. Nf3 Qxd5 4. Nc3 Qd8 5. d4 c6 6. a4 Bg4 7. Be2 Nbd7 8.
h3 Bxf3 9. Bxf3 Qa5 10. O-O O-O-O 11. Qe2 e6 12. Bf4 Bb4 13. Qc4 Nb6 14. Qb3
Bxc3 15. bxc3 Qf5 16. Bh2 Qa5 17. Rfd1 Nfd5 18. c4 Ne7 19. Bf4 Nd7 20. Rab1 b6
21. c5 Qa6 22. Re1 bxc5 23. Be2 Qb6 24. Qc4 Ne5 25. Bxe5 Qa5 26. dxc5 Nd5 27.
Bd6 Nb6 28. Qa6+ Qxa6 29. Bxa6+ Kd7 30. cxb6 axb6 31. Red1 Ra8 32. Be5+ Ke7 33.
Bb7 Ra5 34. Bxg7 Rg8 35. Bd4 c5 36. Be5 Rxa4 37. Bc6 Ra7 38. Rxb6 Rg5 39. f4
Rh5 40. Rb8 f5 41. Re8+ Kf7 42. Rdd8 Rh4 43. Kh2 c4 44. Rf8+ Kg6 45. Be8+ Rf7
46. Bxf7+ Kh6 47. Rg8 Rxh3+ 48. Kxh3 c3 49. Bg7#

Human style?Long castling and with mistakes game.
Reinfeld
Member
Posts: 486
Joined: Thu Feb 17, 2011 3:54 am
Location: Tacoma, WA

Post by Reinfeld »

Thank you very much, Ben - I am digesting and thinking. But what does this mean (emphasis added)?
But instead of letting it train for the longest time possible for the strongest possible play, I grabbed snapshots for different steps of this training.
Could you expand on this notion?

- R.
"You have, let us say, a promising politician, a rising artist that you wish to destroy. Dagger or bomb are archaic and unreliable - but teach him, inoculate him with chess."
– H.G. Wells
X__Gandalf__X
Member
Posts: 155
Joined: Thu Jan 26, 2017 8:36 pm
Location: Tenerife

Post by X__Gandalf__X »

Reinfeld wrote:Thank you very much, Ben - I am digesting and thinking. But what does this mean (emphasis added)?
But instead of letting it train for the longest time possible for the strongest possible play, I grabbed snapshots for different steps of this training.
Could you expand on this notion?

- R.
Think of a 'Grabbed Snapshot' as being like a photograph - A fixed moment in time.
But instead of a photograph, it is the state of the chess program that is frozen and saved - at a fixed moment in time.
A series of 'Grabbed Snapshots' allows the owner to study the chess program at various stages in its learning progress, and go back to revisit them at any time - just like photographs.
JeffB
Member
Posts: 420
Joined: Sat Mar 07, 2015 6:36 pm
Location: Seattle, WA USA

Post by JeffB »

One possible way to get more personality into a ML engine could be to limit its training set to games played by lower level players instead of using games by GMs. This appproach could produce some interesting results by introducing imprecise moves and patterns into the evaluation function.
Cheers,

Jeff B.
User avatar
BenRedic
Member
Posts: 163
Joined: Mon Jan 09, 2017 11:29 am
Location: Norway

Post by BenRedic »

Reinfeld wrote:Thank you very much, Ben - I am digesting and thinking. But what does this mean (emphasis added)?
But instead of letting it train for the longest time possible for the strongest possible play, I grabbed snapshots for different steps of this training.
Could you expand on this notion?
Training Giraffe involves feeding it a large file with FEN positions, then it looks at this and tries to play with the positions using the knowledge it already acquired. Whatever it learns in this process is then stored into a new file, containing Giraffes chess knowledge at that point. It then iterates over this process; The new knowledge file is used to ponder over the FEN positions yet again, this time with (usually) improved results.

The "snapshots" simply mean that I keep the individual knowledge files from each of these steps, so I can choose how much training I want Giraffe to have before letting it loose over the board.

The "t1" and "t25" in the engine names in my previous posts meant that Giraffe was playing with 1 round of training and 25 rounds of training respectively.

If anyone is interested I can make the knowledge files I have made available. I have only gone up to 25 rounds so far, it takes quite a while to generate these files, at least on my old computer :-)
Thank you for an interesting game.
User avatar
mclane
Senior Member
Posts: 1600
Joined: Sun Jul 29, 2007 9:04 am
Location: Luenen, germany, US of europe
Contact:

Post by mclane »

The thing that humans and engines differs is that humans have a plan. They look on the board and try out an idea.

Chess programs do not have a plan.
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....
Reinfeld
Member
Posts: 486
Joined: Thu Feb 17, 2011 3:54 am
Location: Tacoma, WA

Post by Reinfeld »

mclane:
The thing that humans and engines differs is that humans have a plan. They look on the board and try out an idea.

Chess programs do not have a plan.
Absolutely agree this is the fundamental difference. It follows that Chess programs don't know how to make BAD plans.

- R.
"You have, let us say, a promising politician, a rising artist that you wish to destroy. Dagger or bomb are archaic and unreliable - but teach him, inoculate him with chess."
– H.G. Wells
User avatar
Steve B
Site Admin
Posts: 10140
Joined: Sun Jul 29, 2007 10:02 am
Location: New York City USofA
Contact:

Post by Steve B »

Reinfeld wrote: It follows that Chess programs don't know how to make BAD plans.

- R.
True
as was said about the Iran Nuclear Deal...

No Plan.. is better then a Bad Plan Regards
Steve
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Post by spacious_mind »

I don't know. Who says chess programs are not capable of bad and good plans :P

Bad Plan

[fen]r2qkbnr/1b3ppp/p2p4/1pp5/3NP3/1P6/1PP2PPP/RNBQR1K1 w kq - 0 11[/fen]

Other than crazy humans who would have thought of playing Ne6? in the above position. Its the sort of thing I might do if I were playing around with someone I knew I could beat easily at any time like playing against a 3 year old!

[fen]r2qkbnr/1b3ppp/p2pN3/1pp5/4P3/1P6/1PP2PPP/RNBQR1K1 w kq - 0 11[/fen]

There you go, terrible plan. But even in a bad plan, sometimes there is some compensation, like leaving the King awkwardly protected and castling not that great an option.

[fen]2r1k3/1q3rnp/p2p2p1/1b1Np1b1/1Pp1PP2/R1B3P1/1P2Q1KP/3R4 w - - 0 36[/fen]

There are some subtle things going on in this position even though white is still well behind. Rook on a3 maybe sneaking across to h3? f opening up. King stuck in the center.

Seems like a lot of planning going on here for white :)

[fen]2rbk3/1q3r1p/p2pn1p1/1b1N1P2/1Pp1P3/R1B5/1P1RQ1KP/8 w - - 0 39[/fen]

And there you go another sneaky move, meaning f5 that was just played.

[fen]2rbk3/1q3rn1/p2p2p1/1b1N4/1Pp1P1Q1/R1B5/1P1R2KP/8 w - - 0 41[/fen]

Now Black starts to panic. All it sees is the possibility of losing the g-pawn and moves it forward g5 to have it protected by the Bishop.

Notice also the Rook on a3 that everyone seems to have forgotten about. Very sneaky being blocked in by the Bishop on c3.

[fen]2rbk3/1q3rn1/p2p4/1b1N2p1/1Pp1P1Q1/R1B5/1P1R2KP/8 w - - 0 42[/fen]

There you go, pawn was advanced to g5? But, not a problem, white simply exchanges Bishop on c3 for Knight on g7!

[fen]2rbk3/1q4r1/p2p4/1b1N2p1/1Pp1P1Q1/7R/1P1R2KP/8 w - - 0 43[/fen]

And a few moves later, Black still has a piece advantage but the game is over!

Kf7 44. Rf2+ Kg8 45. Qe6+ Rf7 46. Qg6+ Rg7 47. Rf8+ Kxf8 48. Rh8+ Rg8 49. Rxg8# 1-0

I don't know, although it seemed that white started the game with a bad plan :) It certainly finished the game with pretty good plan in mind! Probably better than 99% of the worlds Chessbase unassisted humans . :P

Best regards
Nick
User avatar
scandien
Member
Posts: 206
Joined: Mon Sep 12, 2011 1:15 pm
Contact:

Post by scandien »

Hello

in my view machine are creating plan... the main difference with human skill to create a plan, is in the way used by th ehuman and the machine.. let's explain my view.

A human (with a good level) will planify like this :
  • 1-Analysis of the position ( weakness, strength ).
    2-choosing a position to obtain from this position, this dream position should be better of course.
    3-select few move that can be used to reach the position
    4-analyse the variation, and select the best one.
    5-if the dream position cannot be reach then select another one and go to 2

The plan will not run on all th egame and you will probably be force to imagine a new plan from time to time, you are going frm a dream position to another. This is more or less the silman method.


Now how the computer thinks ?
  • 1-It choose some move ( the is the goal of the selective search function, but brute force can be used too)
    2-all selected move are analysed , and the position reach will be treated as the initial position.
    3-when the machine reach its deepest level of analysis, the best move/variation is found ( to this depth of course) and the machine is able to play the best move (considering the opponent will play the best move too
If the machine is strong enough, the depth will be very great and the machine will play to reach his 'dream' position (in fact the best position caculated).
for a player facing a very strong machine it is exactly the same than playing with a grand master ( both the machine and the grand master will imaine/calculate a position far beyond our chess skills). Try to play versus Komodo or Stokfish ..

With weaker machine (expert to master level) the depth is not so great and the machine will ony play smal plan ( from 10 or 14 moves perhaps for the best) .. But playing a move , and being able to predict the resulting position 12 or 14 move later .. this is planify something, and this is creating a plan.

for weaker player and machine the plan really exist and both will play with tactical consideration only...

But in my view a weak machine is playing as a weak player , an grandmaster level machine is playing as a grandmaster player ...
the point is that the machine of expert level are not able to analyse correctly some position , and cannot be really considered as real Expert /Master if they reach such position.

best regards

Nicolas
User avatar
mclane
Senior Member
Posts: 1600
Joined: Sun Jul 29, 2007 9:04 am
Location: Luenen, germany, US of europe
Contact:

Post by mclane »

The main line increases the evaluation.

A plan is not the same.
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....
User avatar
spacious_mind
Senior Member
Posts: 3999
Joined: Wed Aug 01, 2007 10:20 pm
Location: Alabama
Contact:

Post by spacious_mind »

To believe that humans have plans and chess programs don't have a plan is I think not exactly true.

What is a plan?

Planning is preparing a sequence of action steps to achieve some specific goal. If you do it effectively, you can reduce much the necessary time and effort of achieving the goal.

Based on this above definition the modern program can plan much more effectively than any Grandmaster in History. Well it is obvious that it plans better as it beats them all handily nowadays even with a piece handicap.

Well, you can argue that they search deeper, but that kind of argument to assure some semblance of human superiority is not exactly true or is it? After all if the program has gaps in its program then it doesn't matter how deep it searches because it will lose. In short the programmer is still a little way away from developing all the necessary bits and pieces that the program has to PLAN for in its search analysis. But if he succeeds then then well the planning superiority is game over for all humans forever!

Scandien already showed some of the things that make up a good plan, whether it be a human plan or a computer plan.

Tactics
Counter Attack
King safety
Center control
Endgame knowledge
positional evaluation
Sacrifice
Defense

and so on...

So don't you think that regardless of if you are human or a computer you have to allow for the above in your plan (or calculations) before you decide on what you think is the best move? Look at the picture below.

[fen]r2qkbnr/1b3ppp/p2pN3/1pp5/4P3/1P6/1PP2PPP/RNBQR1K1 w kq - 0 11[/fen]

And then look at this picture below:

[fen]2r1k3/1q3rnp/p2p2p1/1b1Np1b1/1Pp1PP2/R1B3P1/1P2Q1KP/3R4 w - - 0 36[/fen]

Look at where the Knight is. Powerfully placed on d5, controlling the board. Look at the Rook a3! Why is it there? Well its obvious.. it keeps the Bishop honest and stuck on b5. The Black Queen is on that side too... why? because it worries about Rook on a3... other Rook maybe going to a1 and a potential break through if Black's Bishop or Queens where to be moved around especially with the strong Knight dominating on d5. So now the extra Bishop on b5 is out of the game = little advantage. But that is all subterfuge and smoke and mirrors. Because the f4 pawn is at the same time busting open the Kingside. And where are Black pieces? Well Bishop on b5, Queen and Rook on c8 are all uselessly out of the game. And now in addition the Black Bishop is also being pushed back into a lackluster defensive position.

Well, I don't know whether it be human or computer is seems to me that White has a pretty solid plan which any human would be proud of. Whereas black's plan just like many humans is catatonic as a result of being ahead on pieces and negatively defending that advantage instead of looking for his plan for glory.

I don't see a difference in the plan development whether it be human or computer. The plan was good for White and bad from Black.

Where the chess program is better than any human in planning is that a very good program considers all the knowledge that was programmed into it faithfully and accurately every time in its search depth evaluations, which in the end this makes it a much superior planner than a human nowadays can ever be.

In short the computer has become a superior planner :) All human plan efforts fall to the wayside as miserable failures as of the last 4 or 5 years and it gets worse by the day :)

We have become second fiddle behind a powerful calculator that knows how to plan :)

Best regards
Nick
Post Reply