Most Winboard engines come with a default opening book that can be used with the program. Many, however also allow you to either edit the opening book or build a completely new Opening book.
There could be several reasons why you might like to build a new opening book. Firstly, you may be unhappy with the lack of variety of opening lines used by your favourite Chess engine and/or you wish to practice a specific line . Often the opening books are geared towards playing Computer opponents. Since the top Computers are far stronger tactically than even most Grandmasters, they are unlikely to fall for unsound gambits. This usually means that tactical highly unsound gambits are seldom used by the Chess engine, or are not in the default opening book at all. This of course is a problem if you want to practice such lines against the computer program.
For some programs like Crafty, you can adjust parameters [by setting book Random=1 and Width to higher values], that will allow the engine to play some other less popular or unsound move. While this is a good and easy way to increase the variety of openings placed by the engine, this presumes that the opening book itself has a large variety of lines! Often if you want to just focus on some unusual line like the Icelandic gambit, it's much better to just build the whole opening book from a whole database of games involving that line to ensure a knowledgeable opponent to practice against. This is superior to just forcing the engine to play the specific line.
Another related idea is to create a opening book from games played by a certain player say Fischer. Some Chess programs [notably Chessmaster] as well as Winboard compatible engines allow you to alter the playing style or parameters so it's conceivable that you might want to build a opening book of games to match that of the "personality".
On the other hand, if you are a Computer Chess Hobbyist and enjoy running Computer Chess tournaments [See my article, A Beginner's Guide to running Computer Chess tournament. ] , you might want to try to improve the opening book used by your favourite Chess engine.
Traditionally, for most of the amateur Chess engines, the author seldom has time, resources or expertise to develop a good opening book for their program. Unlike Commercial programs, where authors often employ opening book specialists to produce hand tuned opening books, amateur opening books by comparison are often generated automatically from a collection of games. The problem with such a approach of course is the danger of accidentally including some blunder or losing move in the database that will be blindly followed by the engine. There are some safeguards of course {see later), but in general auto-generated books [without any additional tuning] are inferior in quality to hand tuned ones.
If you are a very strong Chess player [Perhaps ELO 2000+], and have a lot of experience with a specific Chess engine you might be able to assist the author in building a better opening book, by pointing out opening holes in the default book, and or suggest entirely new opening lines that suit the style of the particular Chess program.
I will be covering the rest of the article from the point of view of someone trying to build a better opening book for Winboard engines. But of course, the steps needed to build the opening book will be the same whether you are building a opening book for practice training purposes or to help the Chess engine perform better. Certain sections regarding the collection of games will probably apply for people seeking to build opening books for any kind of Chess program. Users of the ChessBase line of products should refer to Electronic T-notes by Steve Lopez.
This is not probably not a complete list but they include Amy, Bringer,Cilian,Colchess, Crafty, Esc, Exchess, Faile, Freyr,Gaviota, Goliath, Green Light Chess , La Dame Blanche Nejmet, Phalanx,Pepito, Quark, Sjeng, SSEChess ,Tristram, Wildcat, Yace, Zchess.
While The Crazy Bishop and Gromit 2.0/3.0, have no opening book
generators , it seems that the pure PGN file itself is sufficient, if
you alter the tcbxinit/gromit.cui file.
Others like Aristarch use opening books that are text files. If you are
aware of any engines that have opening book builders, but are not in
the list, please drop me a email
.
Building a collection of games
If you wish to build a totally new opening book for Winboard engines, you will need a file/database of games in PGN format.[ Goliath's book builder is unique in that the file needs to be in the old CBF format] . Before you go out to collect the games typically from a online or offline database, you need to consider what type of games you are going to include.
You might be tempted to include only games by the very top players , for example only those with a rating above 2600.While that might seem a good idea to reduce the number of poor quality opening moves, this often leads to a opening book that lacks "breath" in coverage of openings. This is because at the higher levels, certain openings are just not played in top GM levels, and a program depending only on such games can often be out of book quickly.
Of course, the dilemma you face is that if you want the book to include a wide variety of openings, you need to include games played at the lower levels ,perhaps 2400 level. But unfortunately the possibility of including a number of bad moves increases here. There will always be a trade-off, but I think good bet , would be to include extremely high quality games from Correspondence Chess. Such games often cover less popular lines used in Over the board play like Evans Gambit, Two Knights defence etc. Also because players are allowed to consult their libraries, such games often produce high qualify opening moves and new technical innovations. See below for online sources of such games
Another possibility is that you supplement the games with some hand tuned lines from various opening books or manuals like Modern Chess Openings, Encyclopaedia of Chess Opening etc, although I advice you to be careful of any copyright infringements.
A interesting idea in my opinion is to include games played either by the program itself or by other Chess programs. If the original opening book is small, the former might give you a insight into the style of the chess program once out of book. In fact, you might be able to simulate book learning crudely, by constantly, using the games generated by the program to add lines or build a entirely new book include only moves [either by using a database program to filter those out or through the book generator options] that have done well. Including games by other programs might cover opening lines commonly used by Chess programs but could lead to some "computer-like" moves being added.
There are a lot of online sites where you can download, search for games, here are some.
If you have collected the games from various sources, it's time to combine them all together. If they are all in PGN format, you can just combine the two together easily since PGN files are mere text files.
copy filename1.pgn + filename2.pgn newfilename.pgn
If you prefer to use software you can use Winboard itself or free utilities like PGNplus, PGN manager etc. You can download some of them here.
If you have some game files in other commercial database formats you must eventually convert them to PGN for the book building process. Again you might need some converters from here.
Before subjecting the whole games file to the book generating/building process, you might want to filter out duplicate games, or some obvious blunders which are often indicated by very short games. While some opening book generators have safeguards against including bad moves by either filtering on number of times a move has being played or by results, it is far easier and *faster* in my opinion to actually do the editing and preparatory work first using a commercial or free database. You might want to try SCID a free to use Chess database. Chessbase light may be a good choice but it limits you to 8,000 games.Still it handles files in the CBH format which you need if you want to build Goliath books.
After filtering how many games should you have? There is no strict rule on the total number of games/moves that you need. If you need or want a big opening book, you will need a extremely big opening database of games, perhaps in excess of 27000 games. Remember, not all the games will be taken into the opening book, when the book is generated.
Extract is a good utility to use for the above work. One requirement that must absolutely be met is that the PGN file used for generation is "clean" . In other words it must be stripped of all annotations, variations and NAGs [Note: Some book builders like those belonging to Yace, Crafty, Zchess etc can actually use these symbols like ! or ? to control the frequently such a move will be played]. This can be done by extract as well.
Of course, some book builders are more "tolerant" than others, and probably can parse/read PGN databases that others will choke on. For example, when trying to build from 2600.pgn file available on Dann's ftp, I found that Sjeng was not able to complete the book building process due to the errors in the game file. Other book builders like Bringer, displayed error signs but were able to continue.
Most opening book generators require that you run the program in
command line mode and then typically run a command like
Bookcreate nameofpgnfile
Note, not all the opening book editors allow all these options. In the future, I may add a table, comparing the flexibility, efficiency in terms of size, speed of generation process etc of the various book generators when I have the time, or there is sufficient interest. Until then I have posted a short summary of my opinions on some of the book generators at the Winboard forum which you can read here
When you are ready, run the opening book generator, be prepared for a extremely long wait if you have used a very big file of games and/or have a slow computer. In particular, building a book for Bringer takes a relative longer time than others engines using the same number of PGN games.
Further Testing and conclusion
Once the generation process is complete, you might want to test out the opening book to see if there are any obvious holes in the opening book. I would recommend running blitz games against another Chess engine. You should be looking out for situations where the program is out of book early or exits from the opening book with a poor/losing score. The position might be objective losing, or the poor score might be a result of the engine "not understanding " or "not liking" the resulting position.
Such lines should be further investigated, and handled either by editing the opening book if that's possible or if it's not you may need to re-generate the whole book, which is extremely time consuming.
Some engines like Bringer, Exchess allow you to directly edit the opening book without recompiling, which is easiest for small changes. In particular, tuning the opening book within Bringer's GUI is especially easy as you can display the current position. By holding on the CTRL button, you can then rate each move from Very Good to Very bad which will affect how often the move will be played. For more details refer to The Bringer Guide translated and edited by by Jim Kearman.
The next easiest way to edit the opening book is employed by programs such as Crafty and ZChess etc. Those engines generally use two books, one bigger book, and a smaller more restrictive opening book. Both books will be used by the engine but the idea behind having two opening books, is that the smaller book allows you to easily modify and rebuild the opening book with small changes more frequently because of it's smaller size.
Once the opening book is ready, you might like to put up your opening book for others to download. If your opening book proves to be really superior to the default, perhaps it might even be adopted by the author as the default! Remember building a good opening book is a very difficult task , so practice makes perfect!
Until then, have fun building opening books.
Aaron Tay
18-03-2001