How to: quickly find regressions
You’ve forked or frozen a repo and haven’t kept up with recent dev, and now it’s time to get back merged with the head. You pull, a whole bunch of code comes down, and some random thing that worked before is now broken. Which of the 200 revisions you just pulled is responsible? Enter hg bisect. (git has a similar command).
‘hg bisect’ performs a binary search over your repo history to identify the exact place where a condition changed (where such a condition is usually “this feature works/doesn’t work”). You seed it with a known good and known bad revision, and it progressively bisects the history tree. bisect automatically updates to a new candidate revision, you test your condition, and then report whether the revision is good or bad. Repeat until bisect identifies the exact commit where your code broke. If your test can be automated or nearly so, the power of bisect becomes clear; you can identify regressions with extreme efficiency!
The situation that just happened to me: I pulled the latest javarosa for new touchforms dev, and repeat deletion had broken, where it worked in the fork touchforms was using.
I initially did ‘bisect’ between the new head and the old touchforms revision. However, as there was only one revision between them (the merge commit), it reported the merge as where things had gone wrong.. not very helpful. So I backtracked farther to where the touchforms fork and javarosa trunk had diverged. (You don’t want to go back farther than necessary, or else the project/build setup may have changed too much for you to easily automate your test).
Here’s how it went down:
$ hg update tip (known bad)28 files updated, 0 files merged, 0 files removed, 0 files unresolved$ hg bisect --reset$ hg bisect --bad$ hg update 3139 (suspected good)214 files updated, 0 files merged, 55 files removed, 0 files unresolved$ (perform test here -- rebuild javarosa, restart touchforms, perform actual test manually)$ hg bisect --goodTesting changeset 3226:889f0e2df338 (128 changesets remaining, ~7 tests)113 files updated, 0 files merged, 41 files removed, 0 files unresolved$ (perform test here)$ hg bisect --goodTesting changeset 3273:590723390250 (64 changesets remaining, ~6 tests)114 files updated, 0 files merged, 25 files removed, 0 files unresolved$ (perform test here)$ hg bisect --badTesting changeset 3254:2174bbdc1c38 (32 changesets remaining, ~5 tests)26 files updated, 0 files merged, 1 files removed, 0 files unresolved$ (perform test here)$ hg bisect --badTesting changeset 3237:30ab1f49dbd7 (16 changesets remaining, ~4 tests)55 files updated, 0 files merged, 7 files removed, 0 files unresolved$ (perform test here)$ hg bisect --goodTesting changeset 3241:6f4133beb2a1 (8 changesets remaining, ~3 tests)26 files updated, 0 files merged, 0 files removed, 0 files unresolved$ (perform test here)$ hg bisect --badTesting changeset 3239:efb8c4c9373a (4 changesets remaining, ~2 tests)6 files updated, 0 files merged, 0 files removed, 0 files unresolved$ (perform test here)$ hg bisect --badTesting changeset 3238:8b73830f8d63 (2 changesets remaining, ~1 tests)7 files updated, 0 files merged, 2 files removed, 0 files unresolved$ (perform test here) [build errors]$ hg bisect --skipDue to skipped revisions, the first bad revision could be any of:changeset: 3238:8b73830f8d63user: clayton sims date: Mon Dec 19 15:46:13 2011 -0500summary: Fixed some serious runtime speed issues and bugs. Enabled modifying storagechangeset: 3239:efb8c4c9373auser: clayton sims date: Mon Dec 19 20:45:27 2011 -0500summary: Got the lowest hanging fruit taken care of. Future speed improvements will
Share
Tags
Similar Articles
Another day, another Zero Day: What all Digital Development organizations should take away from recent IT security news
Even if you don’t work as a software developer, you probably heard about recent, high profile security issues that had IT Admins and developers frantically patching servers over the holidays and again more recently. Dimagi's CTO shares what these recent issues mean for Digital Development organizations.
Technology
January 28, 2022
Join the fight to support critical open source infrastructure
Open Source tools are a critical piece of global infrastructure, and need champions for long term investment
Technology
March 17, 2020
Two big lessons that Iowa and Geneva can teach us about technology in digital development
Last week brought two high profile technology failures into the global spotlight. Although these two mishaps may seem quite different at first glance, they both highlight challenges that are inherent in providing software in the public sector (regardless of locale) and illustrate cautionary lessons worth discussing for practitioners in Digital Development. The Iowa Caucus Debacle
Technology
February 7, 2020