Searching for endgame positions in your games

using HIARCS Chess Explorer and pgn-extract

Introduction and welcome

That’s the first post in my new blog “64 Squares”: this time about chess. I am fascinated by chess and have ideas to share: investigations into my own games, books and chess resources reviews, little programs to answer questions that I have about chess, ideas for chess projects or start-ups and just various (hopefully) interesting observations. Last year I was often thinking: should I just start a chess blog? However, it always takes effort to start something: one thinks that the start needs to be glorious and just take the world by storm. So I was delaying it. But of course that’s not true, I can just start gently, write about interesting things and see what happens.

Let’s start with a little introduction: I am a programmer by trade and a chess fan and player. I’ve been playing chess on and off since I was a little kid. My current FIDE rating is 1677 (just got it this year) and my Chess.com rapid rating is 1800, so you can say that I am a relatively solid amateur. I am playing in OTB tournaments from time to time: in 2023 I played five rapid tournaments in London and one classical tournament in Basel, Switzerland. I am planning to play in Reykjavik Open in 2024! I am a big fan of chess books and accumulate and “sample” them like a maniac. “Sampling” - that’s how I call the process when I start a book, read a couple of chapters from here and there and then switch to another one. There are so many good books, that now I don’t even feel obliged to finish them (I still finish some though!).

My programming skills allow me to write programs and start little chess-related projects, do statistical analysis of games and answer questions that I have about chess. For example, I wrote a script that visualizes how I spend time during the game as opposed to my opponents, which was instructive to guide my time management.

That’s enough about me, let’s now get to the topic of this first post.

Searching for endgame types in your games

I like studying endgames, but I think I am still bad in them. I think that’s not just because I suck at endgames per se, but more because I usually have little time left to play them, since I tend to have less time than my opponents at the end of the game. And when you have under a minute left, those careful considerations and tricky “study-like” plans in the endgame just don’t have time to develop and materialize. Not really an excuse, of course, because if one knows the endgames really well, good moves become instinctive. But this takes effort and lots of practice.

So I decided to tackle rook endgames again, since they are quite common. But before starting I had a couple of questions:

• How common are rook endgames in my own games?

• How well am I doing there, do I win or lose more than I should?

Let’s limit our attention to a single rook endgames, i.e. one rook plus some pawns vs the same on the other side, and try to answer those questions.

Introducing HIARCS Chess Explorer

In order to answer those questions, I assume that one needs some kind of chess software like ChessBase or HIARCS Chess Explorer, because ChessCom and Lichess don’t do that analysis for you. Otherwise, you can just code it, if you are a programmer. Another option in-between is to use a specialised command-line tool for processing PGN files, like pgn-extract, which also has “material search” feature.

Famous ChessBase doesn’t work out of the box on macOS, so I went for a relatively cheaper alternative that works well on macOS: HIARCS Chess Explorer. It can quickly handle databases with 10+ million games, supports ChessBase format (so you can open popular ChessBase databases like MegaBase 2023), comes with its own database of games, and is quite stable and responsive. It has its own quirks, but overall looks like a good piece of software:

First thing I needed to do is to create a database with all the games I wanted to analyze. I downloaded all my games from FICS (I played there in 2009-2012), Lichess and Chess.com.

You can easily download your own games (split by color and time control if needed) from ChessCom and Lichess using openingtree.com and from FICS using ficsgames.org

After that I added my OTB games that I’d recorded and kept in Lichess studies.

Overall, I got 813 rapid and classical games (I didn’t include blitz in this analysis) in a chunky PGN file.

Material queries in HIARCS

HIARCS has a wonderful material quick search feature: you can press Cmd+F on the list of games and enter something like material:R*PvR*P which means find me the games with material “Rook + any number of pawns vs. Rook + any number of pawns”.

Then it presents you a list of games matching this material, and even positions at the first move after this material balance has been achieved, so it’s quite easy to have an overview of different rook endgames I played over the years. Here is a random sample:

I got 52 / 813 games in my filter. This means I’m getting rook endgames in 6.4% of my games and that’s the answer to my first question.

De la Villa claims in “100 Endgames You Must Know” that in total around 8% of games have this ending, so it looks like I face them slightly less often than an average player, but still quite often.

How often do I win rook endgames?

To answer that I needed a little preparatory work. First I had to canonicalize my usernames from different chess servers, so that I can quickly find games where I play White or Black using filters. To do that I just used find-and-replace in PGN file and replaced all my handles with simple “Ivan”.

Then I filtered down the Games List panel to games where I had White pieces using this quick search: ”Ivan-” and then another quick search to get down to required material ”AND material:R*PvR*P”, which gave me this filtered list:

I got 26 games with rook endgames as White, and when I clicked a little “i” button to show the filter information, it printed that my result is quite miserable +7=6-13 (i.e. 7 wins, 6 draws and 13 losses) and that my performance in those games is 103 rating points less than it should be!

For Black the result is similar: 26 games with rook endgames: +5=7-14, and again -100 performance, i.e. I am playing like I am 100 points weaker than my rating.

So, based on this somewhat limited dataset of 52 games we can say that I officially suck at rook endgames :)

Of course, this analysis is not all that “scientific”: it could be that I am entering rook endgames already being several pawns down or starting in otherwise worse positions. However, it’s not that likely, I think on average the positions will be more or less equal. But to account for that, we can run the engine at the beginning of several positions and check the evaluation.

Note that you can do other interesting searches with HIARCS: find positions where you had exactly one extra pawn in queen endgames, or opposite coloured bishops, etc. See the list of available queries here.

Bonus: Using pgn-extract, a free tool

Alternatively, if you don’t want to pay for ChessBase or HIARCS Chess Explorer for material searches, you can use the free pgn-extract tool, written by David Barnes from University of Kent, to do the same analysis. It takes some tinkering to get it running though.

Let’s say that you have your games in my.pgn file. Then, in order to see how many games achieved rook endgames you can use this command-line:

pgn-extract --materialy "3 b0n0q0r1p* b0n0q0r1p*" my.pgn

It tells the program that we are interested in 0 bishops, 0 knights, 0 queens, 1 rook and any number of pawns on both sides. If you are curious “3” at the beginning means “material stability”, i.e. this material balance should be present for at least 3 plies (half-moves). You can control this, but 3 matches the result that HIARCs is giving me, so I am using 3 here.

The tool will then will print all matching games in PGN format (so you can save them to another file and analyze separately) and the total at the end:

52 games matched out of 813.

If you want to get the results statistics, e.g. number of wins by Ivan, when he was playing, say, White, you can further filter down using “-T” option:

pgn-extract --materialy "3 b0n0q0r1p* b0n0q0r1p*" -TwIvan -Tr1-0 my.pgn

prints “7 games matched”, i.e. that I won 7 games when playing White in those rook engames, which again, matches what I had in HIARCS above. You can change “w” to “b” to get Black, and “-T1-0” to “-T0-1” or “-T1/2-1/2” to get losses and draws.

Help for pgn-extract, describing the options in more detail (the tool can do other exciting things!), can be found here.

Next Steps

Now after answering my questions and confirming that my rook endgames may indeed require some extra work, I’m planning to start looking at the actual positions from my games and trying to see what went wrong and why.

It is probably also worth doing an automatic blunder-check on those games to quickly find any obvious errors, so that I can discuss them with the coach or ponder on them myself.

Outro

I hope you’ve learnt something new from this post and now are eager to take a look at your own endgames. This is my first post here, but I plan to write more. So if you liked this, please subscribe to this Substack blog and next posts will be delivered to your mailbox, which is quite convenient!