This page was initially part of my web site on setting up of Winboard engines. However, I
discovered that many visitors to my site, were actually looking
for information on setting up endgame tablebases. Given the fact
that my Winboard web site was already very long, I decided to
start a new web site using the information already available in
the original page as a base. I added information from the various
web sites and past posts at Rec.games.chess.computer
and the
Computer Chess Club.
I'm a layman who is largely unfamiliar with the technical
details of endgame tablebases. As such much of the technical
detail here is only available thanks to the kind permission of
experts such as E.A. Heinz,Guy Haworth, (both frequent contributors to
ICGA journals
on this topic) Anders Thulin (from
rec.games.chess.computer) , Robert Hyatt, Bruce Moreland and
especially Dieter Buerssner (all of those are authors of strong chess
programs that use endgame tablebases). I would be grateful if you can
email me if you spot any
errors.
I hope that you will find this web site useful. If so, please
help sign my guest
book.
Yours Sincerely,
Aaron Tay
01-04-2001
My PGP - DH/DSS 4096/1024 Key
Further information
As of April 2006, one of the best places to obtain further information
about
endgame tablebases is at the following forum
. I highly recommend it, if you have further questions,
particularly technical ones.
23-09-2003 More 6 piece tablebases loaded on Hyatt's FTP site.
Including 6 piece tablebases
with pawns.
23-05-2004 New viewer site by Convekta allowing access to 6 piece
tablebases added
25-07-2004 DVD Endgame Turbo by Chessbase contains a corrupted file
for the KRPPKR endgame.
08-10-2004 Added alternative site for online searching of Nalimov
tablebase results here.
01-12-2005 Add more sites for searching online Nalimov tablebase
results.
Added link to emule collection of tablebase sites.Added Scorpio uses
bitbases
and offers dll for use.
25-04-2006 Cleaned up sections on six piece tablebase, and download
sections.
[A.1] What is a endgame tablebase? How do they work? How much
stronger will it make my chess program play?
Chess endgame tablebases (EGTBs) are special databases that stores
all possible positions with a given material balance and their results.
Note about terminology
Why not call them endgame "database" instead of
"tablebases"? I have no idea, in fact Chessmaster's
version of them are called EGDB (EndGame DataBase)
, but regardless of what they are called they are similar in principle.
In this FAQ, I shall use the terms Endgame tablebases because these are
the names they are most widely known by.)
For example, there is one tablebase file (normally named KNNKP) that
covers positions
where one side has a King and 2 knights, while the other side has a
king and a pawn and yet
another tablebase file (KQK) that covers all positions where one side
has a King and Queen, while the other side has a lone
King.There are more endgame tablebase files of course, though naturally
the more pieces on the board, the bigger the Endgame tablebase file.
Each endgame tablebase file allows the user to check what the result
is with best play.
For example, the KNNKP tablebase,can allow you to input a specific
position with such
a material configuration, and find out what the result should be with best
play.
Nalimov endgame tablebases (one of several types of tablebases),for
example could tell you
if that position is a forced win and if so in how many moves, or a draw
with best play.
One thing to note is that,the raw endgame tablebases files are
highly compressed,and
require that you use some kind of graphic
browser software, before you can view the files.
.However, many Chess playing programs [See here
for a list] can use endgame tablebases automatically,which will
allow them to play that endgame perfectly.
Most chess engines don't even need to reach the position
covered in an endgame tablebases to use the tablebases. For
example, a few moves before such a position, the engine
calculates (but does not play yet) a series of exchanges that
leads directly to a position in the tablebase. The engine will
then look up (or probe) the tablebase and get the results for
that hypothetical position. This should help improve play of
course.
But how are tablebases constructed in the first place?Basically
endgame tablebases are database files of stored
endgame positions calculated using retrograde analysis.Here's
a
basic layman explanation on how tablebases are constructed
Here's another explanation by Bruce
Moreland, author of Ferret (who has generated endgame tablebases
of his own design).
The algorithm works via induction.
How these things works is that all possible positions are
examined, and positions where one side is mated *now* are marked
(apparently legal piece configuration, with one side to move, in
check, and unable to get out of check). These are called "mated
in 0" positions.
Once this is done, all positions with each side to move are
examined, and if any of them can reach a mated in 0
position, these are marked as "mate in 1".
Now, we look at everything again and search for positions
where all possible moves lead to "mate in 1"
positions. These are marked as mated in 2.
Now, we try to reach "mated in 2", these are "mate in 2".
Next is a bit different, because we look for cases where it is
impossible to avoid either "mate in 2" or
"mate in 1". These are "mated in 3".
And so on, until no progress is made. The rest of the positions
are proven to be draws.
It's a little trickier if it's possible to convert to a won
ending, if you are trying to make a "distance to mate" database.
You have to be careful of this idea of "no progress", because
there are gaps. There might be a mate in 105 in KBP vs KN, but
there might not be a mate in 95. This is because the mate in 105
might involve immediate conversion to a KBN vs KN, and yes, such
positions do exist.
How much stronger will it make my chess program play?
Endgame tablebases allow chess programs to play better in 2
situations. Firstly, when the endgame position covered by the tablebase
occurs on the board. This is most always a gain for the chess engine
(assuming you don't have a incomplete tablebase set).
Unfortunately, the current set of endgame tablebases (5 pieces or less)
cover mostly simple endgame positions that most chess engines can
handle either by a combination of search
or knowledge programmed in. So most engines will not gain much from
using tablebases this way (except
in rare positions).
A much greater gain is achieved by the fact that chess engines can
consult tablebases
as a kind of oracle to decide whether to exchange down into a endgame
covered by the tablebase.This allows
the range of positions to be covered to be much larger. A chess engine
can for example avoid exchanging down
to losing positions (even if they are very deep) or win by converting
into a absolutely won endgame.
And yet, people have estimated that most engines gain at most 25 elo
from using them, and many have found no significant difference in
playing strength. Experiments results by
Robert
Allgeuer and Les
Fernandez using Crafty and Yace
show no significant difference in playing strength.
In this article I listed 5
possible weaknesses of endgame tablebases
that might cause this. Of the 5, some are quite rare (problems #2,#3)
or can be avoided (problems #1) , except for the last problem which is
the resulting slow down from accessing the hard-disk.
Robert Allgeuer did a interesting experiment with Yace. And
basically he found that while endgame tablebases allowed engines to
reach and win
such positions, in games where such positions were not reached they
played weaker. By considering games
where positions covered by endgame tablebases was actually reached on
board ,Yace with tablebases played significantly better (about 115 Elo)
On the other hand, in games where such positions was not reached, Yace
with endgame tablebases
played significantly weaker (about 94 elo). The implication was that
in such games, using endgame tablebases was a liability. Robert writes "Apparently
statistically for each game they successfully maneuver into a won 5
piece position, they also lose another one during the preceding
endgame, so that overall this effect more or less cancels out the
advantages of tablebases."
The above results can be still extended by using different time
controls, egtb caches, engines (some like SOS claim to be heavily
dependent on tablebases because the author has stripped out all endgame
related knowledge covered in tablebases) , but
it seems that using the current set of tablebases (3-4-5 pieces), you
can't expect more then a minimal strength increase if any.
Still you might want to use tablebases for learning and analysis
purposes.
[A.2] What are the different endgame tablebases formats out
there? What are the differences?
There are various kinds of Tablebase formats widely in use,
including Ken
Thompson,Steven J. Edwards,Eugene Nalimov Tablebases and Chessmaster/De Koning EDGB (in the order
in which they first appeared). While these are popular,
some programmers have chosen
to use endgame tablebases of their own design, for example Bruce
Moreland's Ferret
In general though, they fall into 2 types of tablebases.
Distance to mate (DTM) and Distance to Conversion
(DTC)tablebases. Roughly in layman terms, the difference is as
follows.
Distance to mate (DTM) - DTM tablebase stores the
shortest possible mate (in ply) for each position. Eg Nalimov
tablebases
Distance to conversion (DTC) - DTC tablebases stores
the shortest number of ply between each position and a
"conversion". A conversion is either a pawn promotion or a piece
being captured or checkmate. Eg Thompson tablebases.
In addition, Distance to zeroing (DTZ) and the
superior Distance to rule (DTR)tablebases have being
proposed to overcome the fact that DTM and DTC do not recognise the
fifty move rule. Only in theory so far.
The tablebases also various in other ways Dr Robert Hyatt
[Author of Crafty] explains the differences between them as
follows [in a posting to rec.games.chess.comp 26/10/2000]
"Edwards (Tablebase): Distance to mate values stored.
The main problem with these is that they are larger than the
others.
Nalimov (Tablebase): Distance to mate values just like
Edwards, but Eugene's files are compressed, and they may be used
in the compressed form, with no penalty of any kind. Rather than
way over 30 gigabytes for all of the 3-4-5 piece files, you end
up with about 7.5 gigs.
Thompson (Tablebase): Distance to conversion (capture
which takes you to a smaller ending class). These are difficult
to use in their compressed form, from inside a chess engine. They
also provide different info than Eugene's database... i.e. it
tells you something, but it doesn't differentiate between losing
and drawing as the Nalimov files do.
Best choice: Nalimov. Nearly every chess engine supports
those... "
This was of course written before the release of the De Koning
EDGB format used by Chessmaster 9000 only in August 2002. Nalimov
still remains the most popular tablebase format since it's supported by
the largest number of engines.
About Chessmaster endgame tablebase / De Koning EDGB
De Koning EDGB -
Propertry format used by only Chessmaster 9000. Like the Nalimov
tablebase it is a Distance to mate tablebase. Unlike the Nalimov
tablebase though, it stores only wins from the white point of
view (similar to Thomson)."(It) searches only for wins for
White. All positions that are not proven to be won by White are
stored as unknown. These unknowns might be draws, or lost by
White, or (in large numbers) illegal." [From
FEG or Final endgame generator for Chessmaster 9000 ]
Note the original version of the endgame generator had a minor
flaw, and the EDGBs that shipped with the CDs also had this flaw.
See here for more
Other differences between the De Koning and Nalimov formats
are
Nalimov's format has KxyKz and KzKxy merged. Eg in KRBKQ
the wins for white (with RB) are stored as well as the losses for
white. In Thompson's and De Koning's white can only win or not
win. [From
FEG or Final endgame generator for Chessmaster 9000 ]
This change means that besides the usual pawn promotion case
problems there are other "dependencies" problems between EGTB
files (my comments)
Chessmaster 9000 currently does not probe the Tablebase in
search but according to email correspondence this is a decision
design and there is nothing in principle that stops this from
occurring.
De Koning EDGB are slightly smaller than compressed Nalimov
endgame files in general.The full De Koning EGDBs came up to
about 5.7 GB , the EGDB's can function with only either the WTM
or BTM files.[Note during generation, you need both files!] This
can reduce the space required to as little as 2.7 GB which is
extremely light on hard-disk space compared to the 7.5 GB (as
quoted by most people) required for Nalimov tablebaes.
The Chessmaster EDGB format generation process is also
faster
compared to Nalimov. (Full set of 5 piece tablebase take 5 days
on 1.2 GHZ Athlon) It also requires much less free RAM (less than
13MB to be precise). The generator also makes generating 6-man
with KWWWWB possible on a 32 bit computer possible , but that
requires about 300MB (which isn't a lot by todays standards) of
free RAM
4 piece KxyK + KxKy 23 MB
5 piece KxyzK 1.25 GB
5 piece KxyKz 3.47 GB
5 piece KxKyz 1.04 GB (The numbers designate the full set. For
simple lookup the half set will do (either wtm or btm). The half
set is somewhat smaller than half of the full set.)
Otherwise De Koning EDGB's are similar to Nalimov in terms
of
castling and 50 moves rule (see below).
More about Nalimov and Compressed Nalimov tablebases
Nalimov tablebases are nearly "perfect" since they take into
account en Passant. However, they don't take into account
castling, but this flaw is probably of interest only to Chess
problem Hobbyists.
In general, though almost all modern Chess programs [including
most Winboard programs] use Nalimov Tablebases partly because
they are non-propriety and partly because they are more
efficient. Also some of the 6 pieces Tablebases are now available
in the Nalimov format. The Nalimov tablebases comes in 2 forms,
uncompressed and compressed. The compressed ones end with the
extension "emd".
Most of the modern Chess programs( e.g. Crafty supports to the
use of the compressed Nalimov format from versions 16.5 onwards)
can use the tablebases in compressed form by uncompressing and
using them on the fly as needed. One exception I'm aware of is
a old version of Esc , a
Winboard
program released on 4 Feb 2001 which uses only the uncompressed
form. If you generated the Nalimov tablebases yourself (see Section [A.8] ), they will already in the
uncompressed form before applying datacomp.exe. datacomp.exe
can also
be used in reverse to get uncompressed Nalimov tablebases from
compressed ones.
Other uncommon types of Tablebases
Bitbases
There are other types of less common tablebases that work quite
differently from
those mentioned above. They generally go by the term "bitbases".
There is a short articleby
Roland Piffer, author of Patzer explaining what bitbases are.
Unfortunately, it's available only in German.
Here's a layman explanation attempted by yours truly.Unlike normal
tablebases that store the
full distance to mate or conversion, bitbases only stores information
about whether
a position is a forced win or loss. Sometimes, this alone is enough to
help a chess engine to discard lines quickly.
However bitbases (unlike typical TBs like Nalimov tablebases) does
not store the exact score (eg mate in x) and hence cannot be used to
work out the best
winning line directly. However because bitbases need only store whether
a position is a win or draw
as opposed to whether it is mate in x (where x can be anything from
1-128 or more), they are naturally smaller
and faster to access and can fit into the computer's RAM. They can be
used together with more traditional
Endgame Tablebases (Nalimov etc).
Yet another note about terminology
The term "bitbases" comes about because it requires only
one bit to store the results
for each position. Again, the naming might differ, e.g. knightdreamer
calls them
Endgame models (egm).
E.A Heinz was probably the first one to pioneer the concept.
They were integrated into his chess program "DarkThought" by late
1996 and proved their practical viability during the 1997 WMCC
in Paris where "DarkThought" employed them throughout the whole
tournament. In addition, he also published an article on it in ICCJA
(see below)
in 1999. Eventually bit-based endgame databases started to appear in
many other microcomputer chess programs (e.g. According to E.A
Heinz,Chrilly Donninger
integrated them into "Nimzo" based on the ICCA Journal article cited
below).
E.A. Heinz. "Knowledgeable encoding and querying of endgame databases."
ICCA Journal, Vol. 22, No. 2, pages 81-97, June 1999.
(published by the ICCA; preprint available from
http://lcs.supertech.lcs.mit.edu/~heinz/ps/know_edb.ps.gz
or
http://www.i-u.de/schools/heinz/ps/know_edb.ps.gz
)
Unlike normal endgame tablebases where there are only a few standard
formats used by almost all chess engines, so far no open standard
bitbase format exists and the few chess engine programmers who have
implemented this idea, use bitbases of his own design.
These are the ones that I'm aware of which use a form of bitbases.
1) Nimzo 8 uses proprietary "Nimzo tablebases" (probably a form of
bitbase too) that are
stored in RAM for faster access.
2) Patzer, which is available as part of the GambitSoft
Winboard Edition II package uses bitbases too.
3)
Yace versions (after
0.68) also use
bitbases.Yace uses bitbases
for 3+4 pieces only and this is stored in memory . This allows Yace to
access them in Quiescience search without much slowdown compared to
using Nalimov tablebases.
4)
Knightdreamer does something similar with
it's own Egmprobe.Like Yace's bitbases, there are currently only 4 men
tablebases and are small enough
to be loaded in RAM. More technical details, A decision tree grown
with an ID3-style algorithm is used to predict if a position is won,
lost or drawn. Eugene Nalimov's EGTBs are used as an oracle. An
accuracy above 99% is reached for most endgames. A table with the
exceptions is compressed with run length encoding and huffman codes.
The exception table is probed on the fly, without generating any
decompressed tables.
Knightdreamer uses this to "adjust scores" (increase scores if there is
a absolute win, divide the score
to get a score closer to zero if a draw is possible with best play) to
help aid scoring (used in pruning too) but does not consider any wins
found
by EGM as a absolute win. This is because occasionally knightdreamer is
unable to win absolutely won games, or may misplay technically drawn
positions. Most interestingly, EgmProbe is distributed as a DLL for
free
for non-commercial engines. To download all the egms for all 5 pieces
tablebase (except for 4-1) go to http://cap.connx.com/Contrib/Johan/KnightD33.zip(211
mb).
5) Delfi has
internal bitbase tablebases (stronger side wins/draws) for almost all
the 4 men
tablebases except N vs P and R vs B and KRPKR. For more information.
6) Scorpio
you uses bitbases up to 4 piece. He also offers a dll to access these
bitbases for those interested. Fruit Gambit and latista uses this too.
7)GAFS
Guido from G6 [ a group of 6 Italians Winboard engine
authors] has announced a new endgame tablebase format - GAFS .Guido writes
It is also possible to generate EGTBs with 2 bits per
positions (win,
loss, draw, illegal position).
There is also a very small object file to link to the chess program in
order to use 8 bits and/or 2 bits EGTBs and this is possible inside the
same run. The idea was that 2 bits tablebases could be sufficient
during
the search process before entering a tabulated ending, while inside the
ending only 8 bits EGTBs can give the correct move.
I don't use compression at least for now. To the persons that tried to
use my EGTBs I suggested to read the result on the disk file (without
loading the file in RAM), and save it in the hash table. If the number
of different positions to read, i.e. access to the file, are not very
high (no more than some thousands), the cost in time should be
acceptable, as a single access to the disk can cost some ms.
8) By 2005, there are now many more engines that use bitbases, many
of their
own design. Examples include Kiwi,
Slowchess,Glaurung
.
I have also heard of tablebases being generated for other
Chess variants like Losing Chess , but I have not be able to find
more information.
[04-01-02] Sjeng
11.2 now allows you to self generate 2 and some 3 tablebases
for suicide Chess.KKFchess
does the same for various variants as well as supporting normal Nalimov
tablebases.
If you have more information,web sites with more information
about endgame tablebases to link to, or clarifications that you
want to make, please
emailme
Note: For the rest of the FAQ, unless I state otherwise,
endgame tablebases refer to Nalimov tablebases.
[A.3] Where can I learn more about the endgame table
formats?
For Nalimov tablebases you need written permission from him.
You might want to look at probe.txt which is part of TBGen.zip. Which
can be normally be found in most places which allow EGTBs to be
downloaded)which
if I understand correctly, gives instructions on how to hook up
your Chess Engine to probe Nalimov's tablebases.Looking at open source
programs
might also help for example, "probe.c" file from BeoWulf or egtb.c file
from Crafty.This post might also
help .
Attention! For Chess programmers trying to learn more
about the format to figure out how to make their Chess programs
use Eugene Nalimov tablebases, : Dr Ernst A. Heinz who is
the author of the Chess program, Dark Thought and author of the
book " Scalable Search in Computer Chess " writes:
"My WWW pages at http://supertech.lcs.mit.edu/~heinz/contain
preprints of two interesting texts about the index schemes of
endgame databases for chess (incl. Eugene
Nalimov's).Specifically, you might want to take a look at:
* E.V. Nalimov, G.McC.Haworth, and E.A. Heinz.
"Space-efficient indexing of chess endgame tables." In ICGA
Journal, Vol. 23, No. 3, pages 148-162, September 2000.
(published by the ICCA; preprint available from http://supertech.lcs.mit.edu/~heinz/ps/NHH_ICGA.ps.gz
* E.A. Heinz. "Endgame databases and efficient index
schemes."In ICCA Journal, Vol. 22, No. 1, pages 22-32, March
1999. (published by the ICCA; preprint available from http://supertech.lcs.mit.edu/~heinz/ps/edb_index.ps.gz
[In a posting to rec.games.chess.comp, 03/13/2001]
You can download for free the
FEG or Final endgame generator for Chessmaster 9000 . It
comes with a nice document file that explains clearly in layman
terms what endgame tablebases are and summarises some of the
issues involved.
[A.4] What are the Chess programs that support endgame
tables? Which format do they support?
Here's a small list.
Program Name
Endgame Tablebase types supported
Fritz 6.0+
Nalimov
Junior 6.0+
Nalimov
HIARCS 7.32+
Nalimov
Nimzo8+
Nalimov, Nimzo Tablebases
Chess Tiger 14.0+*
Nalimov
Rebel Tiger 2.0*
Nalimov
Chessmaster 9000 and up
De Koning format (new)
Shredder 5.0+
Nalimov , Thompson
Chess Genius 6.5
Thompson
Patzer**,Yace Paderborn
Nalimov , Bitbases
Knightdreamer,Delfi
Bitbases
Crafty***
Nalimov
Gromit 2.2,Dragon [Pre-4.0],Bionic
Edwards
Most free Winboard engines****
Nalimov
SCID
Nalimov
Madeleine
Nalimov and GAFS
* The latest update 02/04/2001.Refer to Rebel Web site . Prior
versions did not have any endgame tablebase support.
** Available as part of the GambitSoft Winboard Edition II
package.
*** Old versions of Crafty used Edwards Tablebases.Crafty
15.21 onwards supports only Nalimov tablebases. Crafty 16.5
onwards also supports the use of compressed Eugene Nalimov
tablebases.
Additional note, when Chess engines are used in Chessbase GUI
or Chessmaster 9000, the GUI will use any tablebases available
once the position on the board matches that of the relevant
tablebase. But this does not mean that the Chess engine itself
"uses" the tablebases in it's search etc.
[A.5] Where can I get endgame tablebase?
Nalimov tablebase
If you own only Fritz 6 or any Chessbase engine, you will have
the 3-4 men Compressed nalimov tablebases, but you might want
more. You basically have 3 choices. You can buy
, download or generate
them. [From easiest to most difficult.]
Chessmaster EDGB or De Koning EDGB
If you own Chessmaster 9000, it will already come with all 4
piece tablebases together with some 5 piece tablebases,
particularly the KRPKP and KQPKQ and their respective
depedencies.The set that comes in the CD is as follows
Unfortunately, at time of writing 21 September 2002, there is
no place to download or buy De Keoning tablebases for Chessmaster
9000, so you will have to or generate them
yourself.
Thompson Tablebases
Thompson Tablebases was the first endgame tablebases to be widely
used by chess programs.
Older versions of chessbase , Fritz etc used to support them.
Unfortunately, they are not used much today, because they are largely
inferior to Nalimov's. (E.g They don't cover en-passant, not
all 5 piece tablebases available, they are larger, slower to generate
and use , and only one side to move was generated).
I don't know of any place where they are available for download (not
sure if that's legal), and while they used to be for sale, it's likely
they are not now.
Edwards Tablebases
There's a generator available somewhere on Dann Corbit's ftp. Also
3-4 piece tablebases available for download
on Dann's site.
Bitbases of various formats
Because bitbases are generally small and available for 3-4
tablebases (so as to fit in memory), most either come with the program
as separate files (Knightdreamer,Nimzo), are integrated internally
(Delfi), or can be generated by the program
itself in a couple of minutes or seconds (Yace).
[A.6] Where can I buy tablebases?
Buying Nalimov tablebases
You can buy The Chessbase
DVD Endgame Turbo 2 which is a 5 DVD set that has all of the
Eugene Nalimov 3,4,5 and some 6 piece tablebase set. Conveka
makers of Chess Assistant also sells all 3-4-5 Nalimov tablebases
in
either a DVD set and another set consisting of the above plus some 6
peice tablebases in
2 DVDs.
Buying Chessmaster EDGB or De Koning EDGB
As yet, this is not available.
Buying Thompson tablebases
You used to be able get Thompson Tablebase with certain software
packages
like the "World Champion package" from Millennium
System.Chessbaseused
to sell a 4 CD set of 5 men Thompson Tablebases both separately
and for use with Chessbase 7
I have no idea if Edwards
Tablebases are available for sale.
[A.7] Where can I download tablebases?
Downloading Nalimov tablebases
You could download them file by file for the 3,4,5 and some 6
men Nalimov compressed Tablebase from UAB ftp. This is the
original source and will always have the latest and most complete sets
available as soon as Eugene Nalimov sends them. But see comment on 6 piece tablebases. You will need a
fast internet connection if you want to download the 5 men
tablebases which are very large. , These files are already in
compressed emd format, and it doesn't help to further compress
them with pkzip etc.
[A.8] How do I generate tablebases on my computer?
Generating Nalimov tablebases
Lastly, you can generate Nalimov tablebases using the
tablebase generator tbgen.exe. The
same generator is also on the Fritz6 CD. But self generation of 5
men tablebases is not really feasible unless you have a very fast
computer with lots of RAM especially. You can find a guide by Mogens Larsen about
generating 4 piece tablebases. Antonio Senatore also made a
nicepost at the Winboard forumthat
offers some guidance if you really want to try to generate 5
pieces tablebases.
First
download the FEG or Final Endgame Generator. When you choose
to do a full install, all the tablebases will be copied to
c:\mydocuments regardless of the actual directory Chessmaster
9000 in.
Important note: If you use the full installation of
Chessmaster with EDGB and/or downloaded and generated other
tablebases using FEG version 3.2 (you can check the readme file
to check the version) , the tablebases you generated and some that came
with the CD are
flawed.In a nutshell
Endgames with mates no longer than 1 (eg KNNK) did
not contain the black-is-checkmated positions, or rather their
wrong value being "not_white_win". This affected also some
positions leading to black-is-checkmated by capture (eg KNNKR
with the mating move being NxR).
This bug will affect Seven of the 4 piece endgame tablebase and
Thirty five of the 5 piece endgame tablebase.
Of those that came with the Chessmaster CD, the following sets:
KNNK,KBKB,KBKN,KBKP,KNKR,KNKB,KNKN,KNNKP,KNKBB falls prey to this bug.
I have generated these file sets and put them into one file for your
download. You can them unzip them over the EDGB files
installed from the CD. The chessmaster
EDGB file sets are 12mb zipped and contain the Jix and Joo files
necessary to be used in chessmaster. For the 2 five tablebase
set KNKBB and KNNKP, I have made available only the btm files because
only those
are installed by the chessmaster CD.
Others that are flawed include
KNNKQ,KNNKR,KNNKB,KNNKN,KNPKQ,KNPKN,KNPKP,KBKQB,KBKQN,KBKQP,KBKRB,KBKRN,
KBKRP,KBKBB,KBKBN,KBKBP,KBKNN,KBKNP,KBKPP,KNKQR,KNKQB,KNKQN,KNKRR
,KNKRB,KNKRN,KNKRP,
KNKBN,KNKBP,KNKNN,KNKNP,KPKQR,KPKRR,KPKRB
To change the directory where chessmaster looks for tablebases
edit cm.ini (found in the chessmaster 9000 directory) with a text
editor. You need to change the path
ofEGDBPath=whatevernewapth where your tablebases
are.
A second bug was found on Dec 26, dubbed "the transparent pawn
bug".
"Affected files are the ones where both sides have at least a Pawn and
White has at least a sliding piece. As well as the ones relying on them
via promotion (KPPKP) or capture (6-man only) or both. Regarding
3,4,5-man these are KQPKP, KRPKP, KBPKP, KPPKP."
Note, that even though there are some 5 piece tablebases on
the CD to save space the 5 piece tablebases on the CD do not come
with the WTM files.This means you have to regenerate all the 5
piece tablebases if you want the complete set due to the
depedencies.
[A.9] How large are the tablebase files? Can I put then all into
one directory? Do I have
to use a complete set of 5 men tablebases?
Note the following refers to Nalimov tablebases. Users using
Chessmaster format should ignore this section. Chessmaster 9000
itself is not vulnerable to the incomplete
tablebase problem.
The more tablebase files you install, the stronger the program
will be. However, a full set of 3,4 and 5 Tablebase files takes
about 7.05 GB of Hard disk space! BTW many of the "complete"
endgame tablebase sets you can buy on CD are not really complete
and may exclude some of the less important tablebases for example
the so called 4+1 men tablebase sets [KXXXK] on the assumption
that those are trivial enough to solve. Most people download a
full set of 4 Piece databases and select only a few of the 5
pieces.
Note you can mix all the different endgame tablebase into one
directory if you wish. However if you do decide to split up the
files among various directories (there might be some advantage
in leaving a small subset of tablebases (clear wins like KXXXK in
a chess engine game as opposed to for use in analysis.See here), you might need to check
whether the Chess engine/interface you use allows you to point to
multiple paths.
Dieter Buerssner has informed me that in general all Chess
engines/interfaces allow the use of separate paths for nalimov
tablebases as long as they are separated by " commas, semicolons,
or (at non- Windows systems) by colons." I have tried testing
with a few engines, most seem to work. For example in Pepito , I
add EGTB_PATH C:\fourpiece;g:\fivepiece.
In addition, Dieter offers another interesting tip. " In
console mode, or with Alt-F1 (in Winboard) you can probably
change the path for many engines. For example for Yace, assuming
it had some tbldir in yace.ini, and I saw the analysis of one
position with TBs. Then I want to see it without TBs. I would
type tbldir nul "
For Crafty you need to set the paths (separated by ;)
in the command line (or Winboard.ini) and not in the Crafty.rc
file. (I'm not sure if this applies to newer versions
anymore)
However if you put all the tablebases in one directory as
opposed to splitting them up say by tablebase pieces, it can get
confusing to keep track of which tablebases you have and which
you don't. You can find some programs that help keep track of
this in Section [A.19].
Generally the 5 piece endgames with rooks are reached most
frequently and should be downloaded .Important ones include
KRPKP, KQPKQ and KPPKP. However, there are some pitfalls that you
should be aware of.
If you want to use the KRPKR tablebase [and assuming you have
all the 3 and 4 tablebases], make sure that you have the KQRKR
one as well. [You might consider adding KRRKR, KRBKR , KRNKN as
well but it's probably not necessary] This is to ensure that the
promotion cases are included.
If you lack say the KQRKR tablebase, some programs refuse to
queen the pawn even if that would lead to a win, because such a
move, would cause the program to drop from a position flagged as
"win", to a position that is uncertain [although with a high
score] since they lack the relevant endgame table.
In the position on the left, I have all the 4 men tablebases
but only use one 5 men tablebases- KRKRP. With White to play,
instead of queening the pawn with e8=Q, Comet will instead
play1 Re4+ ?
1 Re4+ appears superior to 1 e8=Q because from the program's
point of view 1 Re4+ leads to a position that is a mate in 16
[obtained from the KRPKP tablebase], while it is unable to search
deep enough to see that 1 e8=Q leads to a even
faster mate in 15 due to the lack of KQRKP tablebase! As such it
prefers a certain mate in 16 to a move that "merely" leads to a
high score. r7/2K1P3/8/4R3/2k5/8/8/8 w - - 0 1
A similar problem can result if you download only
the KQPKQ tablebase without KQQKQ , KQRKQ etc..
Some programs like Crafty and Yace are "smart" enough to avoid
this problem, but some like Fritz,Tiger,Amy,Comet,Gandalf or The
Crazy Bishop[all freely available Winboard engines except for
Fritz and Tiger of course] cannot handle this.
On the other hand, some positions are not problematic, if
there is a quick mate nearby after the pawn promotion that the
Chess engine can see.
There is a short mate in 5 after 1 f8=Q+ . As such Comet and
most Chess engines * can
quickly see this shorter mate and will prefer the pawn promotion
move to 1 Ke7 [found in the KRKP tablebase] which leads to a mate
in 9.
This is made easier by the availability of the 4 men
tablebases as the program need not calculate the position after 1
f8=Q+ Rxf8+ etc since it can use the KRKR etc tablebases. In
fact, the only other legal move [besides Rxf8+] after 1 f8=Q+ is
Kh7 which leads even to a quicker mate. All this is child's play
for Chess programs to calculate
Compare this with the above position, White has 15 other
replies after the pawn promotion which doesn't lead to a 4 men tb
position. 4K2k/5P2/8/6R1/8/5r2/8/8 w - - 0 1
If you are an author of a Chess program and wish to avoid
this problem, what you can do is this. If you find that in the
root position it is mate in X from probing the n men tablebases,
but after trying every legal move you don't find a mate in X-1
from the tablebase, it must be due to a incomplete tablebase
problem and you search without probing the tablebases.
Also take note that not all programs support all the
Tablebases. Yace for example, currently does not use the 4+1
tablebases [ King and 3 pieces/pawns on a side versus a alone
King] , because such positions are easily won most of the time
barring rare cases [Like double rook pawns, wrong coloured bishop
and King vers King].
You might also not want to download 6 man Nalimov endgame
tablebase (see also here)
because I think currently only a few Chess programs support them,
although it's usually a simple matter for the programmers to
activate them. These include Crafty, Sjeng , Quark , King of
Kings[all free Winboard Chess
engines ] and perhaps Rebel
Tiger . I have read that
the Fritz6 engine itself does not probe the 6 men tablebases, but
only uses them when they occur on the board.
* It appears that The Crazy Bishop might
just stop searching whenever it finds a forced mate and as such
it is "happy" with 1 Ke7 . Which explains why it has
problems here, as a mate in 5 is well within it's search
abilities.
[A.10] Questions about 6 men tablebases.Which are the most useful?
Generating Nalimov 6 piece tablebases requires the heavy duty
use of 64 bit computers and is currently out of reach of most
normal users.
As of Jan 2004, all the 3-3 and 4-2 pawnless Nalimov tablebases are
done (227 GB). The 4-2 with pawns
are being worked on, and some are available. None of the 3-3 with pawns
or the 5-1 nalimov tablebases are available yet.
As of Aug 2005, all of the 6 piece tablebases are complete except
for the 5-1 set but not all are available publicly.
There appears to be a lot of confusion about which engines can
access 6 piece tablebases.
This is due to several changes in the nature of the 6 piece tablebases.
First
there is the "normal"
6 piece tablebases generated before Dec 2002, it is suspected that some
of the newer commercial engines like Hiarcs
9 can access this. In Dec 2002, because of the size of the files, each
tablebase is broken into
2 parts, i,e "the multifile format". In September 2003, yet another
batch of tablebases was released with more changes
to the probe code.
Crafty V19.4+ (to be safe use versions above 20) , Wilhelm are
confirmed to work with them. Also confirmed working (ability to use in
search) according to http://kd.lab.nig.ac.jp/chess/tablebases-online/
are Chess Tiger 15.0, Crafty 19.19, Fruit 2.2.1, Gandalf 6.0, Rybka
1.1, Shredder 9. This refers to the larger multiple split set,
normal chess engines can probably handle the older and smaller 6 piece
tablebase sets.
Where to get them.
Historically the main source for 6 men Nalimov tablebases
has being at Bob
Hyatt's ftp . However, as of April 2006, they are still recovering
from a hard-disk crash,
and there are no 6 piece tablebase there. They intend to recover
though, so continue checking.
http://olympuschess.com/tb6/index.html
has a very limited number of 6 piece tablebase sets for download,
however, the best, most complete and most reliable
source for downloading 6 pieces Nalimov tablebases is still via emule
here!.
[Dec 14 2002] - More 6 men tablebases have being uploaded
on Hyatt's site. However because of the size of the files
(greater than 2 gb) they are broken into 2 parts (For example
krnkbn.2.nbw.emd) and it requires a different probing code and
currently [Dec 14 2002] no Chess engine can access it yet except
Crafty 19.2 which will be released soon.From the user point of
view, usage remains the same. As of March 2003, the new version of
Wilhem can access them too.
[September
2003] Yet more endgame tablebases produced with more changes
to the probing code. This current update includes some of the 4+2 with
pawns tablebases,
but very few programs can access it. One is wilhelm,
the other is Crafty 19.4. Any programs released before them (around Oct
03) are unlikely to be able to use such tablebases.
Here's an explanation by Eugene Nalimov.
New access code would shortly be released in Crafty
19.2. Changes were necessary because (1) you should use 64-bit
index to index some of 6-men TBs, (2) even after compression some
files were greater than 2Gb, and some OSes cannot handle such
files (or handle them in absolutely unportable way), so best
solution is to split those files into several smaller ones.
From chess program's point of view the only difference is that
INDEX type now should be unsigned 64-bit integer, not unsigned
32-bit integer. Something like 5 minutes of work.
Some unrelated problems are fixed in the code -- reported
memory leaks, SMP problems, Mac OS X locks problem that
manifested itself in access violation, very slow startup when
TBPATH pointed to the CD-ROM drive and there was no disk in that
drive, better integrity check for the TBs, etc.
[Eugene Nalimov, CCC posting Dec 13,2002]
The pawnless six men endgame tablebases are almost
complete (except for the trivial ones). This is
necessary of course to avoid the incomplete
endgame tablebase problem (although you can use them to see the
results without the best line of play), but it also makes them pretty
useless and may actually slow down the Chess engine if it probes
useless positions.
Of some interest is the KQQKQQ tablebase. It was one
of the first 6 piece tablebase generated back in 1999
especially, to aid in the Kasparov versus World game.
It is estimated a whole set of 6 piece Nalimov tablebases
(when ready) will take up at least 1 terrabye (1000 GB).
Unlike the nalimov tablebases it's possible to generate 6
piece Chessmaster tablebases using a normal 32 bit desktop and
512 ram.It has being reported that
Marc Bourzutschky, has managed to use Chessmaster 9000's FEG to
generate 6 piece tablebases including the KRPPKR in about a month
using 3 desktop computers (1.9GHZ and 2.4GHZ P4, and an an old
800MHZ P3). He has completed the above plus
,KQPKQP,KRPKBP,KRPKNP,KPPPKR,KPPPKB,KQPPKQ,KRPPKQ and supporting
tablebases, and the size is about 0.5 Terabytes.
In addition certain 7 piece tablebases were completed.See here.
[A.11] How can I find out which tablebase is corrupted? How
do I know the endgame tablebases are working?
If you download or generate your own tablebases, there is a
possibility that some of them become corrupted. Some chess playing
programs
do a check and warn you if any of the tablebases are corrupted, but
many don't. /p>
In practice though, checking if endgame tablebases are working
in general for a chess program is usually a easy affair. Just
setup a position, cover by the tablebases, and set the Chess
engine to analyze or move. If everything is done correctly, the
engine will move instantly, or come up with a instant evaluation
score that will not change. Usually, the thinking lines will
indicate that the score comes from the endgame tablebase. E.g
Crafty will display a <EGTB> in the principle
variation.
The above method is fine, except that it will take a long time to
check all the tablebase
files one by one. Here is what I recommend.
In general, when setting up tablebases, there are 2 possible errors.
The interface is configured incorrectly. Generally this occurs
when the path to the tablebase
is in error.
The tablebase files are corrupted.
The procedure above checks for the first problem. If you manage to
get one tablebase working
chances are the path is set correctly or the other tablebase files in
the same directly should work, if they are not corrupted.
Here are 3 ways to check for corrupted files.
The first solution is to make use of datacomp.exe
to check
the integrity of the file But you will need to do this using a
batch file for each file.
The second method involves checking the MD5 sum of each file
with the ones posted on the ftp site. See post by William Penn on
CCC on "How to use MD5 Sums to verify
tablebases".Note you can download Wilhelm a free
Chess program with tablebase functions that will do this
automatically
Lastly, run the endgame tablebases in the original Winboard
version of Crafty or Comet. Both will inform you which specific
tablebases are damaged.
[A.12] What is datacomp.exe? Where can I get it?
Datacomp.exe is a utility provided by Eugene Nalimov and it's
mostly used to compress Nalimov tablebases after you have
generated them. Datacomp.exe can be used in reverse as well to
uncompress and can be used to verify the integrity of existing
tablebases, which is useful to check for errors during
download.
You can get it at Dann Corbit's ftp site at http://cap.connx.com/chess-engines/new-approach/TBGEN.ZIP.The
files, tbgeni.exe, tbgenm.exe that you get from there are exactly
the same except that they have being complied using different
compilers. One or the other might be faster for you.
To use datacomp to compress , use the following format:
datacomp e:8192 knk.nbb
Note this allows you to generate only up to 5 piece tablebases. It
is not advisable
to try for 6 piece tablebases anyway.
[A.13] How do I get Crafty to work with Endgame
tablebases?
You need to edit the Crafty.rc file. For example, my
line reads tbpath=c:\chesseng\tb . You should edit the
line to point to the directory where your endgame tablebase files
are currently located. The procedure for setting up endgame
tablebases is similar for other Winboard engines that you can
download. See my other FAQ on Winboard
and Chess engines
[A.14] How can I get Fritz to use Endgame tablebases?
The advice below applies for users of Fritz 6. Users of newer
chessbase
software like Fritz 7 etc, have the additional option of setting up the
tablebase path within the interface.
First, copy all your tablebase files into one directory, say
C:\Nalimov
Next locate the file chssbase.ini [Not Chessbase.ini!] . It is
usually in your Windows directory. Part of the file is something
like this
[Tablebase]
Path=C:\Nalimov
CacheSizeKB=1024
Just edit the path to point to the directory where your
endgame tablebase files are currently located and remember to
save.This should work for all Chessbase engines including
Junior,Nimzo,Hiracs and the new Chessbase engines Shredder and
Tiger. For further information, refer to Steve Lopez's T-notes
on"Tablebases
and Fritz endgame Tablebase", 2 July 2000 and "Using
Endgame CDs in Chessbase 7", 21 March 1999
Please note that if the endgame tablebases that you use are
corrupted in some way (perhaps through a incomplete download)
Fritz/Junior etc will refuse to run. To find out which tablebase
is corrupted see Section A.11
[A.15] What is the difference between tablebases download
from Dr Hyatt's ftp site and those on the Chessbase endgame turbo
CD? Can they be used together?
Generally the tablebases that come in the Chessbase Endgame
turbo are in the format kxxkxxnbb.emd compared to that
downloaded, which come in the form kxxkxx.nbb.emd .Otherwise I
think they should be exactly the same. I have generally found
that Fritz can use either form, but you may want to add or remove
the dot, if it refuses to work.
Also kxxkxx_nbb.emd should work as well.
In general Nalimov compressed tablebases bought from other
commercial companies are also interchangeable with the downloaded
ones.
[A.16] What are the files ending with .tbs? What about those
ending with nbb and nbw? Do I need both?
Tbs stands for tablebase summary and give stats on the number
of positions
which are won, losses etc. They are not necessary if you are not
interested in such statistics.
Earlier versions of Nalimov's code need the .nbw and .nbb of
symmetrical endgames. Later this was changed (when support of
6-men TBs was added). So, engines using the older code will need
both, and newer code will only need one. The list of the
symmetrical TBs in the code are knkn, kbkb, krkr, kqkq, knnknn,
kbbkbb, krrkrr, kqqkqq but not kpkp
There is new Eugene Nalimov code that allows you to use only
one nbw file in all cases except when there are symmetrical
positions for example in KRKR positions. This may or may not work
depending on the Chess engine and probing code used. For access
to 6 men tablebase positions, the new Nalimov code but be used,
however for symmetric positions like knkn, kbkb, krkr, kqkq,
knnknn, kbbkbb, krrkrr, kqqkqq (but it seems not Kpkp) you need
both nbb and nbw.
To be safe though it's best to have both (or download nbw
first and see if it works), as currently it's not certain how
many Chess engines [besides Crafty] use the new Eugene code, and
you can't really save much space anyway for the symmetric 4 men
tablebases because they are pretty small anyway.
This is unlike the case of Edwards tablebases or
Chessmaster's EGDB , where
you need only either the file ending with tbb [positions with
black to move for Edwards] or tbw [positions with white to move]. But
even
for those, it's much faster to have both since with only one set, the
engine has to do a 2 ply search
instead of 1.
[A.17] Help, the endgame tablebases are not working
properly!
There are some possible reasons
You specified the wrong pathway for the endgame
tablebases.
File is corrupted or in the wrong form , they should be in the
form kxxkxx.nbw.emd or kxxkxx.nbb.emd [for compressed Nalimov
tablebases]. See Section [A.11]
One of the files either kxxkxx.nbw.emd or kxxkxx.nbb.emd is
missing. You need both of them.
You are using the wrong type of endgame table bases or the
Chess program does not support the use of endgame tablebases.
To check whether, the endgame tablebase is corrupted (perhaps
during download) , you can check the file sizes with the ones
available online or use datacomp.exe (which is part of the tbgen)
to check. The command you need is datacomp t
kxxkxx.emdwhere kxxkxx is the file you want to check. This
can be very tedious if you want to check all the endgame
tablebase files, so it might be a good idea to use a batch
file.
If the Program for some reason refuses to make a obvious
winning move [usually queening] in a endgame, it is possible, we
have a case of incomplete tablebases. See Section
A.9 .
No. Nalimov tablebases in the emd format are already
compressed, and there is little if any gain from further zipping
them up.Same goes for the Chessmaster EDGB format.
Let's do them in reverse order. The following websites allow you to
search the
results of a given endgame
position, if you don't have the tablebase files.Most sites below
require Java or at least javascript.
http://www.k4it.de/index.php?topic=egtb
- Probably the most complete online site
that allows
you to query Nalimov tablebases. Covers all 5 piece, all
3-3 (without pawns),all 4-2, Some 3-3 (with pawns).
Shredder online database
-Another very complete one on par with the last link. However it
requires Java.
http://nobi.inf.ethz.ch/games/chess/ -Endgame Chess Browser
[Down?] - A useful site that lists possible moves and the results
[either mate or conversion to another endgame] that will occur on
playing the candidate move.Covers all 5 men tablebases. Might be
of interest to Chess problem hobbyist.In my view, this is best of
all the sites listed here.
"I find Scid very useful for browsing Nalimov
tablebases. If a position is winning, it shows which moves mate
the fastest, if it is losing it shows which moves survive
longest, and if it is drawn it shows which moves keep the
position drawn and which moves (if any) lose. Of course, being
Scid's author I may be biased... " [Posting to rec.games.chess.computer on
12-01-2001]
SCID 3.3 and up also keep tracks of which tablebases you have and
stores statistics about each tablebase.
Wihelm- A free
Chess program but more importantly it comes with powerful
tablabase functions that allow you to automatically check the
integrity of tablebases using MD5 checksums as well as other
functions similar to the "endgame oracle"in shredder 6, (see
below) The website is in German, but the program is in
English/German and there is a good english
PDF help file which explains the various tablebase
functions.
Another alternative including using Fritz, and Chessbase 7 or
8, which has a special engine to view Thompson tablebases.
Freezer
- A commercial tool that uses Nalimov tablebases to help endgame
analysis. By "freezing" actually restricting certain features in the
position (pawn blocked, king cannot move beyond 4th rank etc), coupled
with Nalimov tablebases, it helps clarify certain aspects of endgames
like fortresses.
GAFS
- a program to generate and interrogate tablebases and a library,
under the form of an object file, with an example for using these
tablebases in a chess program. As cpu time for generation of
EGTBs is very high, the program contains an intermediate saving
system to avoid loss of the entire work if for any reason the
computer goes down or computation must be interrupted; so it is
possible to run the program during the night, stop calculation in
the morning and restart again in the successive night.
De Koning EDGB -
Propertry format used by only Chessmaster 9000. Like the Nalimov
tablebase it is a Distance to mate tablebase. Unlike the Nalimov
tablebase though, it stores only wins from the white point of
view (similar to Thomson)."(It) searches only for wins for
White. All positions that are not proven to be won by White are
stored as unknown. These unknowns might be draws, or lost by
White, or (in large numbers) illegal." [From
FEG or Final endgame generator for Chessmaster 9000 ]
