alt text

Go Review Partner documentation

Software purpose

Go Review Partner (“GRP”) is a tool to help analyse and review your games of go (weiqi, baduk) using bots.

GRP is not yet another SGF editor, it won’t allow you to view or modify (adding variations or comments) an existing SGF file.

Instead, it takes one existing SGF and “feed it” to a bot (currently, GnuGo , Leela , Ray AQ and Leela Zero are supported) then has the bot give its “opinion” on the best moves to play for each game position.
GRP proposes a nice interface to review all those data.

Game review work-flow

Basically, it’s a two step process:

Analysis of SGF by a bot

Simply speaking, this consists in picking the SGF file from inside the GRP software, selecting a bot for the analysis, then have it perform the analysis.

You also have the possibility to select a specific range of move to perform the analysis, or only one color.

Beware, the analysis can take quite a bit of time. The higher your level, the higher you want the bot to be stronger to give valuable insight on your game.
So if you want the bot to spend 20 seconds to review each of the game moves, and if it was a 250 moves game, the be ready to wait for more than 2 hours.

Also, the analysis will give a lot of work to your computer during this time, so it might become really unresponsive, to the point you might not be able to use it!

So go ahead, grab a cup of tea/coffee, have a nap, play a few games on-line, do some tsumego, whatever… then come back to your computer when the analysis is completed.

So what will your computer be doing during all this time? It depends of the bot you selected for the analysis, but generally speaking, it will stop at each moves of your game, then ask itself what move it would play for that position.
In fact, it will probably come up with more than one move, and for each of those moves, it will also have a follow up sequence as it imagines the game would proceed from that point.
Usually, additional informations (like win rate) are provided for all those variations.

During the analysis, all that information is gathered and consolidated by GRP and used to produce a new SGF file, that is renamed as RSGF ( Reviewed SGF ).
So if the original game file was name my_game.sgf , then the resulting RSGF file will be named my_game.rsgf

See the Running analysis section for more information.

Review of that RSGF file using GRP interface

So when the analysis is completed, you end up with and RSGF file.

In fact, this is a valid SGF file, but believe me, you don’t want to open it with your usual SGF editor.
The reason for that is that the analysis will have produced a very big game tree: imagine that for each of the 250 moves of your game, 5 variations have been added (each variation being ~10 moves deep), then it will be very painful to review it using a standard SGF editor.

Still want to open it with your usual SGF editor? Sure! No big deal, rename the file from .rsgf to .sgf and have a look for yourself.

When too much information is available, the way to present it and navigate through it is critical. And so GRP comes with an interface adapted to those RSGF files.

See the Game review section for more information.

Installation and running

In fact, it is not really possible to install GRP so far, it’s a stand-alone application that come without installer.
So there are two ways to run GRP:

Running from the sources

Running GRP from the sources is not as complicated as it seems. There is not “compilation” needed, and in fact, this is the recommended way to run it if your are a bit comfortable with computers.

Installing python and TkInter

GRP is programmed using python 2, so you will need python 2 to run on your computer.
GRP also uses the TkInter library to display its graphical interface, so make sure to have that installed as well:

Downloading and running from the sources

At this point, you will want to download the sources for GRP. They are hosted on github at https://github.com/pnprog/goreviewpartner , look for the green button “Clone or download” and then “Download ZIP”. That zip should be named “goreviewpartner-master”. Unzip it wherever you think is appropriate.

Finally, you have to run GRP:

GRP is now hopefully working, you will have to set it up to use one or more bots. See the Settings section for more information.

Running from a bundled download

This is the recommended way if you are using a Microsoft operating system, and don’t want to download bots and set up GRP yourself.
You will find download links from the project main page at https://github.com/pnprog/goreviewpartner .

Download the ZIP archives, unzip it wherever you think is appropriate, then run GoReviewPartner.exe

Main panel

Here is what you will see when launching GRP:

alt text

Run a SGF file analysis will lead you to select a SGF file on your computer and run an analysis on this file.
See the Running analysis section for more information.

In a very similar way, Download a SGF file for analysis will lead you to enter the URL for a SGF file, then download it from the network on your computer and run an analysis on this file. This comes handy for modern Go servers like OGS that provide direct download links to the games. It avoids you the intermediate step to save the file first, and then open it from inside GRP.
See the Running analysis section for more information.

Run a live analysis Allow you to record a game or play against a bot, while, in the background, having the analysis proceed. This is a way to save on the analysis time, so that when the game is over, part of the analysis is already completed, and the review can start immediately.
See the Running live analysis section for more information.

Open a RSFG file for review will lead you to select a RSGF file from your computer and open it for review. See the Game review section for more information.

Convert RSGF file to SGF file will lead you to select a RSGF file to convert it to a SGF file.
See the Convert RSGF file to SGF file section for more information.

Settings is where you define the bots to use for analysis, some of their working parameters, and a few other settings for GRP. If you are running GRP from the sources (not from a bundled download) then setting GRP is a necessary step prior to any game analysis and review.

If GRP did not find any bot set in its configuration, the analysis will be disable. The main panel will then look like this:

alt text

This mean you should have a look at the setting panel. See the Settings section for more information.

Settings

Here is what you will see when opening the Settings panel of GRP:

alt text

It is currently divided in 6 sections:

You can navigate between the different sections using the menu on the left.

Go Review Partner settings

Below are the settings available for this section, together with their meanings:

Language

This parameter defines GRP interface language. Default is English, and so far French and German are available as alternative translations.
Both French and German translations are not 100% completed, so you will occasionally find some part of the interface in English when the corresponding translation is messing.

If you are interested in helping translate GRP to other languages, please let me know. GRP uses a Gettext like type of internationalization system, so it does not need programming skills to add support for a new language to GRP.

Maximum number of variations to record during analysis

Usually a bot will propose up to 10 variations for each moves. But some bots (like Leela Zero) will happily provide you with data on more than 300 variations per move ! To avoid the RSGF files size to become too big, this setting put a limit to the number of variations that can be recorded into the RSGF file. Because each variation is labelled with a letter during the review, it’s probably not necessary to record more than 26 variations.

Only keep variations when game move and bot move differ

This option tell GRP to discard all variations for one position when the bot best move and the game actual move are not the same for that position. This is useful to reduce the size of the RSGF file, especially if one want to review the RSGF file using a standard SGF editor.

Save bot command line into RSGF

This option allows to save the command line into the RSGF file as a reference. Command line sometime can contain useful informations like play-outs limit, number of threads used or weights file.

Stop the analysis if the bot resigns

During the analysis, when one side win rate probability drops too low, the bot will sometime propose to resign. This option allows stopping automatically the analysis at that point to save time. This feature won’t affect the Live Analysis.

Fuzzy Stones

This parameter defines how accurately the stones are positioned on the goban intersection. A value of 0 means perfect placement. You probably do not want to go for values higher than 0.3

Below is what it looks for different values:

fuzzy = 0 fuzzy = 0.2 fuzzy = 0.5 fuzzy = 1
alt text alt text alt text alt text

Real Game sequence deepness

This parameter defines how many moves from the actual game you want to be previewed on the left goban during review. You can compare the results of having this value set at 5 or set at 20 on the two following pictures:

Real game sequence deepness = 5

Real game sequence deepness = 5

Real game sequence deepness = 20

Real game sequence deepness = 20

Goban/screen ratio

This parameter define the size of the goban displayed, as the ratio between the goban size (in pixel) and the screen height in (pixel).

So a value of 0.5 means the goban width and height equal half of the screen height (or width, whichever is the smallest one).

It is also possible to resize the windows with the mouse, this ratio then defines the size at launching.

Maximum number of variations to display during review

Usually a bot will propose up to 10 variations for each moves. Sometime more. You may want to limit the number of variations to display during the review, and only keep the one with the highest win rate. This is what this setting is there for.

Blue/red coloring of the variations

This setting proposes three ways to color the variations proposed by the bot:

As GnuGo does not provide win rate for his variations, they are always represented in black.

Labels for variations

To possibility available:

You can compare the difference on the two images below:

Letters as label

Rates as label

Invert mouse wheel

The mouse wheel can be used to display the moves of the variations one by one. This option allow to inverse the mouse wheel.

Bot settings

The following 5 sections are used to set all bots. All bots share a number of settings in common. Here they are:

Bot common settings

Firstly, for each bot, GRP proposes to set 2 profiles:

Both slow profile and fast profile propose to set a command and some parameters for the bot.

There is also a button Test that will try to run the bot based on the command and parameters you provided, and redirect the output to a terminal like windows to help you ensure the bot is working fine, and help debugging the issues if any. The top part of the terminal represents the standard output of the program, the lower part represents the error output.

For example, when launching Leela on my computer:

GTP terminal

Pressing the Send GTP command can be used to send a GTP command, in this case, the genmove black command that ask the bot to play one move as black. Here is the result (black plays at C4) on my computer:

GTP terminal

Use this testing feature to check the bot is working as usual.

Then, once the bot has been set properly, there is the possibility to indicate where each profile can be used. Choose Slow profile , Fast profile , Both profile or None depending of the situation:

Bellow are the descriptions and specifics options of the 5 bots available in GRP.

AQ settings

AQ ( https://github.com/ymgaq/AQ ) is a very recent bot, by Yu Yamaguchi, distributed under MIT License. AQ is very strong already and is regarded as the first open source bot having beat a professional player on commodity hardware. Moreover, AQ seems to be used a lot by Japanese pro players for training on their own Go sever, where AQ has a 95% win rate against them.

Versions 2.1.1 (with or without GPU) is supported.

AQ has been trained on 19x19 boards with komi at 6.5, no handicap and japanese rule set, so AQ should be you first choice to review games with those conditions if your computer is strong enough.

The settings of AQ is mostly done in the aq_config.txt file, but different configuration files can be used for slow profile and fast profile using the –config parameter. For instance: –config=aq_config_slow.txt
The location of the configuration file can be provided as absolute path, or path relative to the AQ executable.

alt text

GnuGo settings

GnuGo ( https://www.gnu.org/software/gnugo/ ) is a free program that plays the game Go.

Its strength is nowhere near that of Ray, Leela, Leela Zero or AQ, but its analysis is still relevant if your strength is about 11kyu or weaker.

Beside, GnuGo runs well and fast on old computers (Ray, Leela and AQ won’t run on 32-bit computers), can play on 9x9, 13x13 and 19x19 boards, with all sort of handicap and all possible komi.

GnuGo is not using Monte Carlo methods (MTCS) so its behaviour may feels much more human in those situations:

Some of the parameters you may want to add are:

GnuGo specific settings are:

alt text

Leela settings

Leela ( https://www.sjeng.org/leela.html ) is an strong bot, by Gian-Carlo Pascutto. Leela saw a huge increase in strength following Google/Deepming publication in Nature, and in 2016 became the strongest Go bot free to download and use.

It fact, Leela is what prompted me to start GoReviewPartner.

Leela is dan level on all sort of hardware, with or without GPU. Leela can play with on 9x9, 13x13 and 19x19 boards, with komi values of 7.5, 6.5 and 0, with and without handicap. So if you are a kyu player with a modest computer or laptop, then you should consider using Leela.

Version 0.11 is supported.

Some of the parameters you may want to add are:

Leela specific settings are:

alt text

Ray settings

Ray ( https://github.com/zakki/ray ) is a strong bot, by Yuki Kobayashi, distributed under BSD License. Ray is still at its early stage of development, but is very strong already.

If your computer is equipped with a powerful Nvidia GPU that can run CUDA, then you can consider using Ray.

Version 4.32 is supported.

Some of the parameters you may want to add are:

alt text

Leela Zero settings

Leela Zero ( https://github.com/gcp/leela-zero ) is a new project initiated by Gian-Carlo Pascutto, which aims at reimplementing the system described in the Alpha Go Zero paper “Mastering the Game of Go without Human Knowledge”. It is a collaborative project, under the GPL 3.0 licence.

This bot is in permanent process of learning to play by playing against itself. It is now the stronger bot available for free, even beating pro players. You are welcome to contribute to this project by sharing your computer power.

Leela Zero is training on 19x19 board , using komi at 7.5, no handicap and Chinese rule set, so Leela Zero should be you first choice to review games with those conditions if your computer is strong enough. Note that at the time of writing this, Leela Zero still sometime plays a not working ladder, which can immediately ruin a game, although it is becoming very rare those days.

Some of the parameters you may want to add are:

Leela Zero specific settings are:

alt text

Running analysis

Running the analysis is a very straightforward process:

Selection of SGF is made from the main panel, either from a SGF on your computer (press the Open a SGF file for analysis button) of either from a on-line SGF (press the Download a SGF file for analysis button). See the Main panel section for more information.

If you choose to download the SGF file directly from the network, you will be asked to write or paste the URL into that pop-up, then press the Get button:

alt text

GRP will then let you select the analysis parameters:

alt text

Bot to use for the analysis will display the bots and profiles currently configured (if no command was provided for a bot, it won’t be listed).

Select a variation to be analysed let you choose which line of play to follow for the analysis. Typically, a game record SGF only contains one variation, but if a move was undo-ed during the game, this may have created a second branch. Pay attention to the number of moves for each branch to decide what branch to analyse. Other branches will be removed from the RSGF tree.

Select moves to be analysed let you chose the range of moves to analyse. You can analyse all of them, or only a subset of them. One or more move intervals can be indicated (ex: 10-20, 40,50-51,63,67 ).

Select colors to be analysed let you chose which player’s move to be analysed. It is advised to analyse both Black and White, as this is necessary to produce comparison graphs.

Confirm the value of komi let you double check and adjust the komi if necessary. By default, GRP will select the komi value from the SGF file. However, sometime this information is missing in the SGF file (GRP will report it as 0), sometime, the value is reported incorrectly (ie: 750 instead of 7.5) and that would confuse the bot. So correct that value if necessary.

Stop the analysis if the bot resigns allows stopping automatically the analysis when the bot would resign.

When this is done, just press Start button to start the analysis:

The indicated remaining time is an approximation only. After the analysis of the first move is completed, GRP will propose to directly open the RSGF file in review mode:

alt text

alt text

You can save the review for later if you want. The RSGF file will have been saved right next to your original SGF file, with the same name, except for the file extension.

Game review

To review an RSGF file, open it from the main panel Open a RSGF file for review button. This will open a double board interface:

alt text

Interface overview

Variations available

On both boards, you will noticed letters (A, B, C…). Those are the different variations available for this game position. So the left board only contains one (that of the actual game). The variations on the right board are the variation provided by the bot, with A being the best variation (the one the bot would chose), then B, C… second and third best variation, and so on.

alt text

If no variation is available, this could be because:

The right board moves may have different colors:

Note that if the letter color is black (inside a red or blue circle) this indicates that this move was also the game actual move.

Displaying sequences

Move you mouse over one of the variation, and the complete sequence will be displayed with each move numbered.

alt text

Additional information for that line of play will be indicated in the comment box bellow the board.

If you want to follow the sequence move my move, then you can use your mouse scrolling wheel, it will display the moves one by one. Your keyboard Up and Down keys can be used as well.

Territories

GnuGo provides a territory map, and Leela provides an influence map. Both can be visualized by pressing the territories button. This button does not appear when those data are not available.

alt text

Captured stones are marked using a dot from the opposite color.

Graphs

Using the data provided at each move allows to draw some interesting charts.
Not all bots provide all types of graph. Leela is the one offering the most graphs.

Win rate, Value Network win rate and Monte Carlo win rate

Those graph consists in the successive values of win rates, from black perspective.

alt text

Click on one of the grey bar will bring to that position in the review panel. A yellow bar indicates the current position in the review.

When no bar are available, it means no data were provided, possibly because:

Delta graphs

By comparing the win rate (or Value Network win rate, Monte Carlo win rate) at one move (when the bot best move would be played) with the win rate of the following move (the case when the actual game move was played), one can draw a comparison chart for each color.

This is a chart that indicates by how much the bot believes it could have played better than the human player, or eventually by how much the human player move was better than its own move.

alt text

On this graph:

So red means margin for improvement, and green means better the bot, from the bot point of view.

On the comparison graph above, the very last move win rate, when added the red quantity, now exceeds 50%, meaning that from the bot point of view, this move was a blunder that cost black the game.

Here is the corresponding white comparison graph:

alt text

Click on one of the grey bar will bring to that position in the review panel.

When no bar are available, it means no data were provided, possibly because:

Opening a move for manual analysis

During the game review, you might want to try out your own variation. This is made possible by using the Open position button.

This will open the game position in a new pop-up, where you are free to manually try other lines of play. The Undo button allows to cancel the last move played.

alt text

If a profile has be indicated for a bot (see the Settings section for more information.), you can ask it to play with you along the variation, and then check its answers to your different variations.

alt text

If a bot is not in the list, it could be because:

The different options to control the bot are:

quick evaluation

While the bot is thinking for a move, the interface will be locked, the goban line appear in red. There is no way to abort the bot while it is thinking, you have to wait for it to come up with its move.

In self play mode, to stop the bot, press the Abort button and wait.

self play

Live Analysis

A live analysis makes it possible to record a game or play against a bot, while, in the background, having the analysis proceed. This is a way to save on the analysis time, so that when the game is over, part of the analysis is already completed, and the review can start immediately.

It also has the advantage to make the result of the analysis directly available (given the bot performing the analysis is fast enough to catch up with the game speed) which is interesting during a live review.

Live analysis setting

Here is the setting panel for a live analysis, and the meaning for each settings:

Live analysis setting

Bot to use for analysis let you choice what bot will perform the analysis. It is probably better to select a slow profile bot.

Black player and White player let you choice who is playing for black and white. You basically have three options:

Board size , Komi , Handicap stones are the usual game settings. Handicap stones have free placement.

SGF file name allows you to select where the RSGF files will be saved.

Select colors to be analysed allow to restrict the analysis to only black or white moves.

So here are different way to use the live analysis mode:

When a bot is selected as black or white, that is not the same as the bot used for analysis, or when black and white are played by different bots, then an additional option appears No overlap thinking time that indicates to GRP to make sure the bots are no thinking at the same time, so that your computer processing power (CPU, GPU, RAM) does not becomes a shared resource.

No overlap thinking time

Depending of the situation, this option will be or won’t be useful:

Generally speaking, it is best to use play-outs limits for your bots. You can also set the process of the bot performing the analysis to run at low priority, so that it won’t make your opponent slower.

Press the Start button to start the Live analysis. You will be asked to first place the handicap stones first, if any.

Playing

Here is the live analysis panels, the first one is Human VS Bot, the second is Bot VS Bot:

Live analysis

Live analysis

Game section provides some basic information on the current game. If it is a Bot VS Bot game, there will be a Pause / Resume button to pause the game. If it is a Human VS Human game, or Human VS Bot game, there will be an Undo button available as well as a Pass button.

Analysis section provides some basic information on the background analysis. There is an Open position button available at any time to try out variations. And when the analysis for the 2 first moves is completed, a Start review button will appear that starts the usual review panel for this game. Of course, only the moves already analysed will be available, but the review panel will update itself automatically as the analysis progresses.

Convert RSGF file to SGF file

RSGF is a SGF file that contain non standard properties. Such properties are used to save win rate, value network win rate, number of play-outs and so on. A standard SGF editor can open a RSGF file but won’t be able to interpret those properties.

Converting a RSGF file to a SGF will put all those data into a SGF comments for each moves and variations, making it possible to review those data with a normal SGF editor. The resulting SGF file will be bigger.

Advanced uses

It is somehow possible to use GRP as a command line tool, to automatize the analysis. This is only possible when running directly from the Python sources.

Each bot comes with is own Python file:

Those python scripts can be used as follow:

Usage: python bot_analysis.py [--range=<range>] [--color=both] [--komi=<komi>] [--variation=<variation>] [--profil=<fast|slow>] [--no-gui] <sgf file1> <sgf file2> <sgf file3>

Here are a few examples:

#Analysis of mygame.sgf, first variation, using Leela (slow profile), all moves, both colors, komi as indicated by the SGF file:
python leela_analysis.py --profile=slow mygame.sgf

#Same as above, the three SGF files will be analysed, and using Leela fast profile:
python leela_analysis.py --profile=fast mygame1.sgf mygame2.sgf mygame3.sgf

#Same as above, but all the SGF files will be analysed, and the analysis will be performed by Ray:
python ray_analysis.py *.sgf

#Analysis of black moves only:
python ray_analysis.py --color=black mygame.sgf

#Analysis of moves 10 to 20, 44, 46 and 100 to 110 by AQ:
python aq_analysis.py --range="10-20, 44,46, 100-110" mygame.sgf

#Analysis of variation 3, with setting the komi at 7.5 (some SGF files do not include komi):
python aq_analysis.py --variation=3 --komi=7.5 mygame.sgf

If the –no-gui parameter is provided, GRP will be running only in the terminal, no graphical interface.

If no –profile parameter is provided, GRP will be using the slow profile.

The conversion from RSGF to SGF can also be done in command line:

python r2sgf.py game1.sgf
python r2sgf.py *.sgf

Those commands can be ran from the terminal, or from a bash script or batch script.