HCE Pro 1.2.1 Import multiple files
Moderators: Watchman, Mark Uniacke, mrudolf
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
HCE Pro 1.2.1 Import multiple files
I tried something and got "strange" results.
Created a pgn file, then File | Import.
Seeing that I could select multiple files, I selected 1500 pgns.
After a long import, the file remains empty.
I waited some more time to make sure it was not saving.
What's wrong?
What I was expecting or what the program did?
Created a pgn file, then File | Import.
Seeing that I could select multiple files, I selected 1500 pgns.
After a long import, the file remains empty.
I waited some more time to make sure it was not saving.
What's wrong?
What I was expecting or what the program did?
Re: HCE Pro 1.2.1 Import multiple files
Hi,
i have tried with 11 pin files without problems.
Have you the same problem with 11 or 10 pin files ?
Do you have the same problem if you create a hce file and not a pgn file ?
i have tried with 11 pin files without problems.
Have you the same problem with 11 or 10 pin files ?
Do you have the same problem if you create a hce file and not a pgn file ?
Want to play a game ?
Im colfax64 on Lichess also
Im colfax64 on Lichess also
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
Re: HCE Pro 1.2.1 Import multiple files
Tried with 10 pgn files, imported into the pgn database without any problem.
Tried against an hce database with 1500 files, ends quickly saying 0 games imported.
Looks like some error condition escaped through error management.
Re: HCE Pro 1.2.1 Import multiple files
I have downloaded 50 pgn files from web and duplicate it x times to have 1600 pgn files.
I have created a new pgn file and the select all pgn files to import them.
All is fine and at the end I have a lot of games.
with HCEPro I found all games in one file.
I open the disk file (Finder for Mac) and the size file is 0.
Then I close HCE PRO and then HCE Pro save the pgn file and finally I have a 2GB size file saved.
Same for you ?
I have created a new pgn file and the select all pgn files to import them.
All is fine and at the end I have a lot of games.
with HCEPro I found all games in one file.
I open the disk file (Finder for Mac) and the size file is 0.
Then I close HCE PRO and then HCE Pro save the pgn file and finally I have a 2GB size file saved.
Same for you ?
Want to play a game ?
Im colfax64 on Lichess also
Im colfax64 on Lichess also
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
Re: HCE Pro 1.2.1 Import multiple files
Nope.Ruyter wrote: ↑Thu Aug 24, 2023 6:14 pm I have downloaded 50 pgn files from web and duplicate it x times to have 1600 pgn files.
I have created a new pgn file and the select all pgn files to import them.
All is fine and at the end I have a lot of games.
with HCEPro I found all games in one file.
I open the disk file (Finder for Mac) and the size file is 0.
Then I close HCE PRO and then HCE Pro save the pgn file and finally I have a 2GB size file saved.
Same for you ?
In my case it remained empty.
Re: HCE Pro 1.2.1 Import multiple files
is it possible you share your pgn file with some cloud or other platform ?
you use windows or osx ?
you use windows or osx ?
Want to play a game ?
Im colfax64 on Lichess also
Im colfax64 on Lichess also
- Mark Uniacke
- Hiarcs Author
- Posts: 1459
- Joined: Sun Jul 29, 2007 1:32 pm
- Location: United Kingdom
- Contact:
Re: HCE Pro 1.2.1 Import multiple files
I cannot reproduce this, it would be useful if you could narrow it down, could the HCE file be in a read-only location.
What are the sizes in games of the respective PGN files?
Importing 1500 PGN files is probably not that common, I wonder if one of the files has an issue, are they all valid PGNs?
Could you zip the PGNs and upload and then email us the download link so we can try to reproduce, if we can reproduce it can be fixed.
Can you send a Help->Report the report a problem info with config and log?
Have you tried importing 750 files to see if that works? Isolating a file or set of files where the issue exists that would be excellent.
What are the sizes in games of the respective PGN files?
Importing 1500 PGN files is probably not that common, I wonder if one of the files has an issue, are they all valid PGNs?
Could you zip the PGNs and upload and then email us the download link so we can try to reproduce, if we can reproduce it can be fixed.
Can you send a Help->Report the report a problem info with config and log?
Have you tried importing 750 files to see if that works? Isolating a file or set of files where the issue exists that would be excellent.
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
Re: HCE Pro 1.2.1 Import multiple files
Found what happened.GoneBallistics wrote: ↑Thu Aug 24, 2023 8:06 pmThe output file is empty.
The input files are twic 1-1500.
I'm going to less files (1000, 500, etc)
Made a few attempts loading blocks and went down to the single file causing the problem.
The first twics were txt files with games in the middle, so I cleaned up all what is not pgn code.
File twic61 didn't contain pgn code, apaprently, so the text remained.
If you try and import it you get a msgbox "Error: the database 'twic0061' has incorrect format."
Apparently with multiple files this notification is lost.
If you import the files with Chessbase they are imported without any problem.
If you want the file I can send it, but any text file should be fine to replicate the problem.
- Mark Uniacke
- Hiarcs Author
- Posts: 1459
- Joined: Sun Jul 29, 2007 1:32 pm
- Location: United Kingdom
- Contact:
Re: HCE Pro 1.2.1 Import multiple files
Thanks, that makes sense if the pgns were corrupt.
If you use hold the Alt key (or Cmd on Mac) and select the Help->Show application log... menu item, it will list the first offending file,
e.g. 00:45:22 Database: [Warning] the database 'C:/Users/admin/Documents/HIARCS Chess/Databases/corrupt.pgn' has incorrect format.
Given one file is incorrect and corrupt it would seem sensible not to try to import the other files regardless to avoid corrupting the user database.
If you use hold the Alt key (or Cmd on Mac) and select the Help->Show application log... menu item, it will list the first offending file,
e.g. 00:45:22 Database: [Warning] the database 'C:/Users/admin/Documents/HIARCS Chess/Databases/corrupt.pgn' has incorrect format.
Given one file is incorrect and corrupt it would seem sensible not to try to import the other files regardless to avoid corrupting the user database.
Re: HCE Pro 1.2.1 Import multiple files
Hi,
nice to learn the happy end.
nice to learn the happy end.
Want to play a game ?
Im colfax64 on Lichess also
Im colfax64 on Lichess also
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
Re: HCE Pro 1.2.1 Import multiple files
I learned something: I had no idea that Alt + Help gave those additional informations.Mark Uniacke wrote: ↑Thu Aug 24, 2023 11:51 pm Thanks, that makes sense if the pgns were corrupt.
If you use hold the Alt key (or Cmd on Mac) and select the Help->Show application log... menu item, it will list the first offending file,
e.g. 00:45:22 Database: [Warning] the database 'C:/Users/admin/Documents/HIARCS Chess/Databases/corrupt.pgn' has incorrect format.
Given one file is incorrect and corrupt it would seem sensible not to try to import the other files regardless to avoid corrupting the user database.
The problem I see I get the line number for the errors but there is no file name.
Apparently it works as expected only with a single file.
09:49:19 Progress: start operation 0 "" (100 steps)
09:49:19 PGN: [Warning] PGN parse error in game 1 (line 1): no header
09:49:19 PGN: [Warning] PGN parse error in game 9 (line 152): no header
09:49:19 PGN: [Warning] PGN parse error in game 11 (line 173): no header
I was importing 100 files.
Re: HCE Pro 1.2.1 Import multiple files
As far as I understand, there are a few issues:GoneBallistics wrote: ↑Fri Aug 25, 2023 7:55 am I learned something: I had no idea that Alt + Help gave those additional informations.
The problem I see I get the line number for the errors but there is no file name.
Apparently it works as expected only with a single file.
09:49:19 Progress: start operation 0 "" (100 steps)
09:49:19 PGN: [Warning] PGN parse error in game 1 (line 1): no header
09:49:19 PGN: [Warning] PGN parse error in game 9 (line 152): no header
09:49:19 PGN: [Warning] PGN parse error in game 11 (line 173): no header
I was importing 100 files.
1. One buggy import file cancels the import. Not sure if this is correct behaviour or whether we should import other files. You can argue both ways, so I believe HCE Pro behaviour is reasonable.
2. The message for incorrect format is easy to miss.
3. The log doesn't contain file names when importing multiple files. (BTW. these are just warnings, they do not stop import).
Is that correct?
-
- Member
- Posts: 43
- Joined: Wed Nov 02, 2022 7:18 pm
Re: HCE Pro 1.2.1 Import multiple files
Completely agree with 1, the behaviour is perfectly reasonable.mrudolf wrote: ↑Fri Aug 25, 2023 8:05 am As far as I understand, there are a few issues:
1. One buggy import file cancels the import. Not sure if this is correct behaviour or whether we should import other files. You can argue both ways, so I believe HCE Pro behaviour is reasonable.
2. The message for incorrect format is easy to miss.
3. The log doesn't contain file names when importing multiple files. (BTW. these are just warnings, they do not stop import).
Is that correct?
About 2, the message doesn't appear at all, or perhaps it's removed so quickly that I can't see it: in single file mode the msgbox is clearly visible.
ABout 3, the log is not much helpful.
Not a big problem, I'm going to join the file as I did before, and the check them: the problem arised becaus I wanted to try the multiple files import, which is a new function apparently.
The problem is I trusted too much the original files.
Re: HCE Pro 1.2.1 Import multiple files
I would add one more thing: please don't use PGN for large files unless you really really need it (so, mostly, for export to another program).
PGN is not a format suitable for large databases: it takes a lot of space and changing one move in a single game requires a complete rewrite.
Use native database and export to PGN if necessary. Your data will be safer and everything will be faster.