Stockfish Progression

In the first quarter of 2013, Gary Linscott created the distributed testing framework known as "Fishtest". In conjunction with the use of GitHub, the development of Stockfish became a truly open and public project. With contributions from over 900 developers and testers, who have supplied ideas for over 15,000 tests and given over 600 years of CPU time, Fishtest has led to the dramatic increase in strength of Stockfish. In addition, the successful ideas that have come out of Fishtest have had a definite effect on development of other computer chess engines.

I have tried to measure the change in Elo for Stockfish, using Stockfish 2.3.1 (which I believe was the original "master" or reference version) as the reference. For my measurement, I have used the generic Linux 64bit compiles available at http://abrok.eu/stockfish/ . For each month, I downloaded the last development version available on the 22nd day (the chosen day due to SF 2.3.1 being dated 2012/09/22). Each version played each other 100 times (50 openings/reversed colors). Also included are the official releases.

CPUs: Xeon L5420
Time Control: 60" + 0.1"
Hash: 128 MB
Threads: 1
Openings: 50 positions 2 moves deep (a subset of the openings used in round 2 of TCEC)
Adjudication:  resign @ 450 cp for 4 moves; draw after 120 moves if score < 50 cp for 4 moves
UI: cutechess-cli
Total number of games: 159600 (games available upon request)

Note: There is a distortion in the Elo ratings for Stockfish 3 and Stockfish 4 due to 7 time forfeits (all by Stockfish 3 when playing Stockfish 4) that have not been replayed yet.