Influence of learn Value: max + (all the way to right)
Learning Strength: max - (all the way to right)
Minimum games: 0
Up to move: 100
These are the settings for a hand tuned book. I recommend that you take the time to learn how to fix the losing lines (as they are reputed in online play) and reinforce the winning ones.
You can of course turn on the book learn function, but this only stops the book from playing a line that it has consistently lost in, it does not find a solution to the problem. In general it is my opinion that a good book with the learn function turned on will only become more drawlish.
With out a doubt the most useful aspect of book making, to the average engine player is hand tinning. Most of the commercial opening books (the books that come with engines and GUIs) can be tweaked into competitive shape by this technique.
What exactly is meant by the term hand tuning. Well to me its more than just turning some book moves red and others green. It is coming to the point in the book line where things started going wrong and doing deep analysis in infinite time mode to find a better line. Sometime one must go several moves back from the point where the positional value changed towards the negative to get back on track
BOOK HAND TUNNING I
Ok, you have a tournament book that is focused and plays a narrow variety of opening moves to a good depth. Now you need to hand tune it and keep it current. But what exactly are you tuning it to? The answer is; primarily your entire playing system. Which includes your hardware and its capabilities, your engine and its configuration settings, as well as the ever changing lines being played by your opponents.
There is no such thing as a perfect book that has all the answers to all the lines. A good opening book is a thing that is in constant flux. While in the human chess world an opening repertoire may serve a player for many years with only minor changes, it is not so in engine chess. Computers have a way of finding their way around seemingly impassable winning lines with frightening elegances and speed.
To get started; go on line and play 50-60 games at your desired time control. A hand tuned book is most certainly sensitive to time control changes. (by that I mean use one book for 3-0 play one for 16-0 and another for 60-0 + times) You may use the same book but with different tuning. Do not fiddle around with your engine configuration settings to much. (If you are using Houdini 2.0c the default settings are the strongest)
Now take the games that you have just played and go off line and into the engine GUI. Open up the My Internet Machine Games database file and do a bit of weeding. At first just go ahead and delete all the draws and games that go fewer than 35-40 moves. Now physically look at the games that you won. Games that you won from opponents that had inferior hardware to yours can be deleted. This should leave you with the games that you lost and a few that you won.
Take these games and get to work, the idea is to alter the opening book manually not only to avoid the loses but to find a winning line form before the point your evaluation number went negative. Make sure that you have the opening book that you wish to alter loaded and go to the book display. Right click the mouse on the book display window and activate allow move adding. Now any moves that you make manually while the engine is in infinite analysis mode will be added to the book. (as green play in tour moves)
If the book line went wrong before the engine engaged it may be necessary to mark some book moves red and try a deferent way. (book moves are turn red by right clicking on the moves and choosing the do not play in tournament option)
When the engine analysis mode is used (in most engines) it reverts to a brute force search. Where as in normal play all kinds of pruning tricks are used to reduce the number of moves the engine must look at. My point is that when going over played games in analyses mode you can often find strong moves that Rybka discarded in early pruning as being unproductive. It take some time and patients to get the search depth to an effective level. (time dependent on your hardware of course) When you are analyzing a game that you have lost, that had an average search depth of 16 ply you will probably need to go 19-20 ply in a brute force search starting several moves before the detectable swing in positional value to find a good refuting line.
BOOK HAND TUNNING II
Although the opening book only has control of the first 50 moves. (100 half moves) they are most important of the game. There is no reason why (given a bit of practice) that you can not prevent your engine from losing in the first half of the game. No mater what hardware you have it is not difficult to duplicate the depth of the fastest CPUs in offline analyses. (it just takes a bit more time)
When reviewing your games played online give the most attention of course to the games that you lost, but do not forget to look at the draws and wins as well. Many times using infinite analyses it is possible to find a line that breaks the draw. In games that you have won against strong opponents lengthen the book lines. To do this use the manual add move function in the book window of the GUI. This method of adding moves to the book gives the best control but do not forget that it adds all moves as green play moves. So be sure to go back and remove the green of the move in the losing side of the line. (this is done by right clicking on the move and hitting main move a second time)
Spend more time on finding winning lines as white, as black you may have to be satisfied with getting the draw in some lines. Here are the facts about the percentages of White vs. Black wins.
W 50% - B 50% In human club play:
W 52% - B 48% In master play:
W 56% - B 44% In GM play
W 62% - B 38% In engine play
I think this data says something about the relative strength of computers over humans. But my point is that in engine chess, the opening impetus is more pronounced for white. As black you should not expect to come out of opening book with a positional advantage very often. -.30 -.50 going into the middle game as Black will still give you a fighting chance.
Trying to tell someone how to update and tune books is some what a like trying to tell someone how to type on a keyboard. You can tell them where to put the fingers and what some of the edit keys do, but the only way one learns is by doing. Along the way many mistakes will be made. (always keep a archived copy of the books that you are tuning, so if something goes very wrong you can start over) And believe me when I say that the book learning function will not produce the same result as the basic methods that I have talked about, nor will it give the same satisfaction and connection to engine chess.
OPENING BOOK DEVELOPMENT I
All right then, so you want to make a opening book from scratch. A one of a kind book that is your own. A book that in fact can be proprietary. Let me see if I can start you on the right track.
Although there are many ways to go about this let me describe the method that I have found to be the most successful. It Involves a process of development over time.
The first thing you will need is a few comprehensive databases of engine games played under rated conditions and times. (it is generally accepted that we do not use human games in engine books) I recommend using the available game bases from PlayChess server. They can be found in several places on the internet. Try to get the most current ones (all 2011 games and all Jan, 2012) are available. These are quite large databases; but can just be manipulated by the Fritz 9, 10, 11,12,13 GUIs.
It will probably be to your advantage to look around on the net for the most current game bases.
Ok, you now have large game bases of current engine games. Tens of thousands of them. All mixed together; good games, bad games, winning lines and losing ones. After combining all bases into one master base the first thing you will need to do is a bit of filtering.
Bring your master base up on the Fritz GUI and go to filter games. Lets get rid of all games with less than 35 moves. Set filter for 1-35 moves, select all, delete, remove deleted games from base. Do the same with the draws, set filter to bring up all draws then select all, delete, remove deleted games.
Now open up an empty data base and name it New Book Main Base (or something like Blitz Book Games) This will be one of two game bases that you will continue to add games to over a period of time.
Go back to the master game base and this time we will do something radical. We are going to take only the games played by the top rated players. Set filter to 2650 to 3100 Elo and check both and engage. This will return 40,000 to 45,000 games from the highest rated players. (for a smaller, but more directed base try 2700-3100 Elo) Hit select all and copy. Go to your newly created game base and paste these games there.
The next thing you will need is a date base of game to be learned. These games should come from the main game base that you just created. My method for choosing the games to be learned by the book is less than scientific but extremely practical. Pick 4 or 5 of the top players of your choice. for example:
Make an empty game base and name it New book Learn (or Blitz book learn, whatever you decide to name your book) Filter the main book base for Alien wins as white; copy and paste to the new learn book base. Do the same for Alien wins as black, and so on down the list. What you are doing is taking only these top players winning games and adding the to your book learn file. This will give your fledgling book the playing characteristics of these players. (basically stealing their best lines)
You are now ready to turn these new game bases into what I call a seed book. This book will be the starting point for your personal book to be developed from. In the Fritz GUI open a new empty book and name it. Import all games from the main book base that you have created. (to the maximum depth 100 moves) Now go to Learn from Database and click learn white and black wins (do not choose learn loses) and select your new learn file. (made from the top players wins)
You now have a seed book.
Go on line and play the book using these book settings.
Book settings: (Seed Book)
Use Book: on
Tournament Book: on
Varity of Play: max - (all the way to left)
Influence of learn Value: max + (all the way to right)
Learning Strength: max - (all the way to right)
Minimum games: 0
Up to move: 100
Save all games that you play with the new book as these are the games that, played with your own hardware will be instrumental in its development.
OPENING BOOKS DEVELOPMENT II
Before I go any farther let me address this issue of stealing lines. (I used this term in the last post) It is a fact that some players out there seem to think that if they personally worked out a line or a refutation of one; that they somehow own this series of moves. I find this thinking hilariously funny. It brings to my mind Bobby Fischer jumping up from the table and saying you can’t play that move it is mine, you stole it from my game in the interzonals last year… I call this concept of stealing lines. Fischer thinking. “Look Bobby, once you play the line in open play it belongs to all of us.” This is how, in esoteric disciplines, we collectively learn and advance human knowledge. It is by the possess of peer review that the lines you submit in open play are accepted or rejected. So there is no real stealing of anything only finding and using good ideas. We are all standing on the shoulders of geniuses to be where we are.
Now that you have made your seed book and played some games with it, you are ready to start its development by adding games. Adding games to a book (both your own and kibitz ones) is much different than hand tuning. I think it would be accurate to say that adding games increases the width of the book where as, tuning deals with depth and direction.
It is always best to add games to the book that are played by the book that you are adding them to. This technique has a tendency to reinforce and expand winning lines. However if you find good opening lines while kibitzing do not hesitate to include them into your book.
Lets look at the method of adding the games. First never add games directly to an existing book. Add the games to the database that the book was made from and then make a fresh book from the updated game bases. Importing games into an existing book is probably the most conmen mistake that is made in book making.
Make sure that if you add a game to the learn game base you also add it the main game base.
I recommend that you add all games that you won in online play to the main game base and a selected few that beat you solidly within the first 50 moves. Be more selective as to the games you add to the learn base. Look carefully at your played games; making sure that advantage in the game came from the opening or early in the middle game. (DRAWS optional but I think better to use gaems draws, and wins and losses)
Continue to add the games that you play with the seed book back into the book using this technique. Every day review your games, update the game bases (both main base and learn) and remake the book for the next time at online play. This process will slowly but surly expand the books practical repertoire and adjust it to your particular system engine combination. You will begin to notice games that your engine struggled to win now being won easily and good winning lines that were used against you now being played by your book.
At least 2000 games or so should be added to the seed book before you start the hand tuning process. Once you start hand tuning you can no longer add games as red-green marks would be lost.
Here are some of the best advise that we can found on the net
To Create a .ctg Book by Mark Mason
1. Start with a database of games that you want to use to make your book.
2. Create a new book in Fritz by selecting FILE/NEW/OPENINGS BOOK - call it anything you like. 3. Open your new book by selecting FILE/OPEN/OPENINGS BOOK and selecting the new book you created earlier. 4. Now import the games from your database by selecting EDIT/OPENINGS BOOK/IMPORT GAMES and navigating to your database. 5. There is a dialogue box which you need to tick against. "Games" is the number of games you want to import. Default is all games are selected so leave this. "Length" is how deep the lines will be in the book. Choosing "ECO-relative" plus a number will make the main line variations longer than the oddball side variations. If you pick "Absolute length" and a number, it will make all the variations the length you chose, regardless of whether the variation is a main line or side line - so book will be deeper. "Include variations" is important if some of the games from your database contain commentary in the form of sub-variations, but can be left un-ticked. 6. When you're finished with these settings, click "OK". You'll see a progress bar appear, keeping you posted as to how close Fritz has come to completing the, you'll see a menu of moves where the empty book had been and a small window saying "x new positions" (this tells you how many positions total are in the new tree). Click OK, and you're finished. To manually edit the book if you have the little booklet that came with Fritz 9 then in sections 5.6 and all of section 11 give details. Also there is a manual on the Fritz 9 DVD . Pages 49 to 54 give some instructions to have a look at. Meanwhile here is a brief pointer: Go into Fritz and do File/Open/Openings Book and select the book you want to edit. Then select the Openings Book tab on right hand side of screen. Basically you can either a) add a new move to the book, To do this right click somewhere in that book area (not on a move just in empty space) and tick Allow Move Adding. then when you play back through a game and want to add new line, just make the move on the actual board and then select new main line from the options. This will then have added the move to the book. b) change the 'weighting' of an existing move making it more/less likely to be chosen. This is shown in the Prob / % column. To change it right click on the actual move itself and do change weight. The value is from -125 (very unlikely to be picked) to +125 (very likely to be picked). You can change these manually to any value. Actually this is what the automatic book learning does based on whether games are won or lost - but this is how you do it manually. c) or mark moves either red (means they won't be played ever) or green (mean they will be picked, with a probablility given by the % column). To make a move red right click on it and select Don't play in tournament To make a move green right click on it and select Main Move. You might want to make a move red if you find that at some point in the book line you had a negative evaluation and therefore you can mark that move red so it won't be played again. There is a lot more to it than this but this should give you enough to start creating and editing your book. I hope this helps you.