Managing Vendor Branches
A common problem in software development is how to track components that are developed outside of your control. This is something that is normally considered an advanced topic, but we will show you how this can be handled in an elegant way using named branches in Mercurial.
Contents
Overview
Large software systems consist of many components and it is often the case that some of them are third-party modules. As such, the development of these modules outside of the our control. The vendor will regularly make new releases of his component, and for software components, each release will typically be accompanied with the source code so that downstream projects can make any necessary adjustments while integrating the component.
While it is great to get hold of the source code in order to have full control over the integration, this also poses a problem: how does one deal with the next release by the vendor? We cannot just overwrite the component with the new version since we have modified it to suit our purposes. What we want is to re-apply the modifications to the new version of the component, and this is something Mercurial is very good at helping with.
Let us begin by establishing the terminology. A vendor branch is a branch where you commit code drops from the upstream vendor. A code drop is a bunch of files as given to you by the vendor, be it via a zip-file, a tarball, or maybe as a checkout from a foreign version control system.
Hint
The last point is interesting in its own right: you can use the techniques described here to work with any version control system. So even if your colleagues are still stuck with CVS, you can still begin using Mercurial for your own development.
You can even send your changes back to the foreign version control system as a kind of vendor drop in the other direction. We will explain more about this below.
Creating a Vendor Branch
For the purposes of this guide, we will be working with a very small example application written in Java. The program will use an external library to compute Fibonacci numbers — we call this library libfib. We will see how we can import several versions of libfib while maintaining our own bugfixes on top of it. Mercurial will help us keeping track of what needs to be merged.
Alice will start by making a repository for her project and add a README file:
alice$ hg init fibapp alice$ cd fibapp alice$ echo 'Fun with Fibonacci numbers.' > README alice$ hg add README alice$ hg commit -m 'Start of project'
She will be using Java, so she also configures a .hgignore file right away:
alice$ hg status alice$ echo 'syntax: glob' > .hgignore alice$ echo '*.class' >> .hgignore alice$ hg add .hgignore alice$ hg commit -m 'Ignore Java class files'
She then gets to work… Alice is not the biggest math genius in the world, so she delegates the work of computing the Fibonacci numbers to a library, the — the libfib library mentioned above.
Alice unpacks the library and moves it into place:
alice$ cp -r $SRCDIR/libfib-1.0 libfib
The library files are of all untracked right now:
alice$ hg status ? libfib/Fibonacci.java ? libfib/README
She adds them all to Mercurial:
alice$ hg add adding libfib/Fibonacci.java adding libfib/README
She now needs to commit the library files. Instead of making the commit on the default brach, she creates a new named branch exclusively for this library. As we will see, this makes makes it easy to keep track of “her” code and “their” code in the future.
alice$ hg branch libfib marked working directory as branch libfib (branches are permanent and global, did you want a bookmark?) alice$ hg commit -m 'Added libfib version 1.0'
The libfib branch will be reserved exclusively for clean imports of the library. All normal development takes place on the default branch, so Alice updates back to this branch.
alice$ hg update default 0 files updated, 0 files merged, 2 files removed, 0 files unresolved
The library is no longer there, only the README file Alice added in the very beginning is present:
alice$ ls README
To bring the library into the default branch, Alice must merge the libfib branch into default:
alice$ hg merge libfib 2 files updated, 0 files merged, 0 files removed, 0 files unresolved (branch merge, don't forget to commit) alice$ hg commit -m 'Integrated libfib 1.0'
This has put the branch into place:
alice$ ls libfib README
The history graph now looks like this:
Next, she creates a small test program to test the library:
alice$ cp $SRCDIR/Test.java . alice$ javac Test.java alice$ java Test Fibonacci numbers: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...
She looks up the series on Wikipedia and is satisfied with the result. The test program goes into the repository:
alice$ hg add Test.java alice$ hg commit -m 'Created test program'
This is the final graph:
When looking at a graph like this, remember that while there is only one changeset on the libfib branch, the files that were added in revision 2 live on in revisions 3 and 4 because of the merge we did in revision 3.
Fixing Library Bugs
Alice does some further testing and notices that her test program enters a (seemingly) endless loop if she calls the Fibonacci function with a negative number. She would like to fix this so that the library function throws a IllegalArgumentException when it cannot compute the Fibonacci number.
The code for the library is this:
alice$ hg cat libfib/Fibonacci.java package libfib; public class Fibonacci { public static int fib(int n) { if (n == 0 || n == 1) return n; else return fib(n-1) + fib(n-2); } }
Alice wonders if it was really necessary to introduce a (buggy!) library for this, but decides to stick with it for now. She changes the Fibonacci.fib class method to throw the exception for negative input:
alice$ hg diff diff -r 0d2dcbc55e9f libfib/Fibonacci.java --- a/libfib/Fibonacci.java Tue Oct 05 00:00:00 2010 +0000 +++ b/libfib/Fibonacci.java Wed Oct 06 00:00:00 2010 +0000 @@ -2,6 +2,8 @@ public class Fibonacci { public static int fib(int n) { + if (n < 0) + throw IllegalArgumentException("negative input") if (n == 0 || n == 1) return n; else
She commits this change while still on the default branch and the resulting graph is:
It is very important that Alice makes the bugfix on the default branch and not on the libfib branch, since she needs to distinguish between changed made by her and changes made by the upstream project when importing new versions of the library.
Upgrading a Library
A little while later, the developers of libfib publishes a new version. Thought the new release only promisses better documentation, she decides to upgrade to it — she feels that it is important to upgrade in as small steps as possible.
The upgrade is done on the libfib branch. Alice starts by removing the old library completely and then moves the new library into place:
alice$ hg update libfib 1 files updated, 0 files merged, 1 files removed, 0 files unresolved alice$ rm -r libfib alice$ cp -r $SRCDIR/libfib-2.0 libfib
She can now inspect the differences directly with Mercuruial:
alice$ hg status M libfib/Fibonacci.java ! libfib/README ? libfib/README.txt
The upstream developers have apparently renamed the README file to README.txt. Alice used hg addremove to detect this automatically:
alice$ hg addremove --similarity 85 removing libfib/README adding libfib/README.txt recording removal of libfib/README as rename to libfib/README.txt (89% similar)
When using hg addremove yourself, you will have to experiment with the --similarity parameter to find a number that looks reasonable. The default for hg addremove is to search for renames with a similarity of 100 (meaning that the files must be identical) and by using a lower value, you can accomodate smaller or larger changes in the files. In the above case we see that the renamed README file was identified. You can see the difference like this:
alice$ hg diff --git libfib/README.txt diff --git a/libfib/README b/libfib/README.txt copy from libfib/README copy to libfib/README.txt --- a/libfib/README +++ b/libfib/README.txt @@ -1,7 +1,7 @@ libfib ====== -Version 1.0 +Version 2.0 What is it? -----------
The --git option is necessary to enable the extended patch format introduced by Git. The classic patch format cannot represent renamed files and instead shows the README.txt as completely new:
alice$ hg diff libfib/README.txt --nodates diff -r ffcf5bce1dad libfib/README.txt --- /dev/null +++ b/libfib/README.txt @@ -0,0 +1,9 @@ +libfib +====== + +Version 2.0 + +What is it? +----------- + +libfib is a (very) small Fibonacci number library for Java.
Regardless of the diff format used, you can convince yourself that Mercurial has recorded the rename correctly with the --copies switch to hg status:
alice$ hg status --copies M libfib/Fibonacci.java A libfib/README.txt libfib/README R libfib/README
Having convinced herself that the new files are correctly added to Mercurial, she commits the change and inspects the history graph:
alice$ hg commit -m 'Import of libfoo version 2.0'
The final step is to merge this new shiny version of libfoo with the default branch where she has made a small bugfix. The merge is done like any other merge: she first updates to the branch she wants to merge into and then calls hg merge:
alice$ hg update default 3 files updated, 0 files merged, 1 files removed, 0 files unresolved alice$ hg merge libfib merging libfib/Fibonacci.java 1 files updated, 1 files merged, 1 files removed, 0 files unresolved (branch merge, don't forget to commit)
The working copy now contains changes from both branches. Alice can inspect them with hg diff — first the changes made to libfib/Fibonacci.java that came in via the libfib branch:
alice$ hg diff libfib/Fibonacci.java diff -r 67becde235ca libfib/Fibonacci.java --- a/libfib/Fibonacci.java Sun Oct 03 00:00:00 2010 +0000 +++ b/libfib/Fibonacci.java Fri Oct 08 00:00:00 2010 +0000 @@ -1,6 +1,18 @@ package libfib; +/** + * Generate numbers from the Fibonacci sequence. + * + * The Fibonacci sequence start with 0, 1, 1, 2, 3, 5, 8, 13, ... + * + */ public class Fibonacci { + /** + * Generate a number in the Fibonacci sequence. + * + * @param n the zero-based index into the series. + * @return the nth Fibonacci number. + */ public static int fib(int n) { if (n < 0) throw IllegalArgumentException("negative input")
The diff shows the documentation changes the upstream developers promissed. She also checks the changes made to libfib/Fibonacci.java in the default branch to confirm that her bugfix is still present:
alice$ hg diff libfib/Fibonacci.java -r libfib diff -r 7bc0cbfe52ee libfib/Fibonacci.java --- a/libfib/Fibonacci.java Thu Oct 07 00:00:00 2010 +0000 +++ b/libfib/Fibonacci.java Fri Oct 08 00:00:00 2010 +0000 @@ -14,6 +14,8 @@ * @return the nth Fibonacci number. */ public static int fib(int n) { + if (n < 0) + throw IllegalArgumentException("negative input") if (n == 0 || n == 1) return n; else
The above diffs looks good, so Alice makes a commit:
The default branch now has the last version of libfoo with Alice’s bugfix applied.