aragost Trifork: Mercurial Kick Start Exercises


Task Based Development

In the previous section, we used bookmarks to track several concurrent lines of development (feature or topic branches). Here, we will explore a workflow based around using named branches instead.

Named branches differ from bookmarks in several respects:

Because of these differences, a project should document how named branches are used in the project. Whereas bookmarks can be created and deleted with no consequence, named branches persist and share a global name space. This means that the branches become an integral part of the overall workflow in the project and the team members need to agree on their use.

In the examples below, the team will use the company bug tracker as the source of long-term stable references. While this is a common setup, some projects only use named branches for tracking long-term release branches — they might use clones or bookmarks for tracking smaller bugfix and feature branches. That way they avoid the extra “bureaucrazy” associated with the more heavy-weight named branches.

Contents

Overview

Imagine a small team where a set of junior developers, Alice and Bob, will work on the tasks while a senior developer, Carla, will review and integrate the changes made by Alice and Bob.

The senior developer will be the only one who has write access to the main repository. She pulls changes from Alice and Bob and pushes then to the main repository after reviewing them. Alice and Bob pulls changes from the main repository:

flow.png

Using this flow, Carla can control what goes into the main repository, from which the final builds are made.

In a real setup, the repositories will most likely be located on different machines, but we will place all four repositories besides one another in our examples:

$ ls
alice
bob
carla
main

This is only to make the examples easier.

Working with a Branch

Alice begins her work by pulling in changes from the main repository:

alice$ hg pull ../main
pulling from ../main
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
(run 'hg update' to get a working copy)
alice$ hg update
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Alice now had an up-to-date clone of the main repository. The project consist of a single file, hello.txt:

alice$ cat hello.txt
Goodbye, World!

There is clearly a bug here, and Carla has put it in the company bug tracker as Issue 1. She tells Alice to fix it by changing the text to Hello, World!, which seems more appropriate.

Creating a Branch

Alice makes a new branch for her work:

alice$ hg branch issue-1
marked working directory as branch issue-1
(branches are permanent and global, did you want a bookmark?)
alice$ hg commit -m "Starting branch for Issue 1." 

The hg branch command does not change the history or working copy in itself, it only ensure that the next commit is “stamped” with issue-1.

The message issued is just a warning to remind you that you cannot delete the branch name later (though we will see that you can hide a branch if you don’t need it any longer). As described in the introduction, named branches are persistent, and it is therefore not recommended to use them unless you really want to permanently track where a changeset was made. In this project, the team uses the bug tracker as their reference and names the branches according to the issue they’re working on — that way it’s easy to find all commits related to a given bug later.

Normally, you cannot make an empty commit, but changing the branch name counts as a change, so Alice was able to make the commit. The repository has just two changesets:

alice-branched.png

Notice the branch names in the summary column. The names are only shown for the changesets that make up branch heads. The changeset on the default branch is drawn with a light blue cirle and the changeset on the issue-1 branch is drawn with a green circle.

She then begins hacking:

alice$ echo "Hey, World!" > hello.txt
alice$ hg diff
diff -r eb6db6528ec2 hello.txt
--- a/hello.txt Mon Mar 15 10:25:00 2010 +0000
+++ b/hello.txt Mon Mar 15 10:28:00 2010 +0000
@@ -1,1 +1,1 @@
-Goodbye, World!
+Hey, World!
alice$ hg commit -m "Fixed Issue 1." 

The world now looks like this:

alice$ hg glog
@  changeset:   2:2cdcbb0a5bfd
|  branch:      issue-1
|  tag:         tip
|  user:        Alice <alice@example.net>
|  date:        Mon Mar 15 10:30:00 2010 +0000
|  summary:     Fixed Issue 1.
|
o  changeset:   1:eb6db6528ec2
|  branch:      issue-1
|  user:        Alice <alice@example.net>
|  date:        Mon Mar 15 10:25:00 2010 +0000
|  summary:     Starting branch for Issue 1.
|
o  changeset:   0:e9eb044d45e0
   user:        Carla <carla@example.net>
   date:        Mon Mar 01 10:20:30 2010 +0000
   summary:     Initial import.

The two tip-most changesets are on the issue-1 branch, the root changeset is on a branch called default, despite no branch name being printed. Changesets are always on a branch in Mercurial, but we do not display it when they are on the default branch to reduce clutter. Note that changesets on different branches share the same changeset graph — the branch names simply give you an easy way to refer to the branch heads: instead of 2cdcbb0a5bfd you write issue-1 and instead of e9eb044d45e0 you write default. In this sense, branches work very much like “floating tags” that always point to the tip-most changeset on a particular branch. These tip-most changesets are exactly the ones highlighted with a light green circle in thg log:

alice-fixed-issue-1.png

Alice comments in the bug tracker that she has fixed the bug and that Carla should pull her changes.

Merging a Branch

Carla has been working too, while Alice fixed the bug, and has added a README file:

carla$ hg glog
@  changeset:   1:550bad1893cf
|  tag:         tip
|  user:        Carla <carla@example.net>
|  date:        Tue Mar 02 14:30:00 2010 +0000
|  summary:     Added README.
|
o  changeset:   0:e9eb044d45e0
   user:        Carla <carla@example.net>
   date:        Mon Mar 01 10:20:30 2010 +0000
   summary:     Initial import.

Carla pulls from Alice:

carla$ hg pull ../alice
pulling from ../alice
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads)

Mercurial tells her that she has a new head in the repository. This is visible with the hg heads command:

carla$ hg heads
changeset:   3:2cdcbb0a5bfd
branch:      issue-1
tag:         tip
user:        Alice <alice@example.net>
date:        Mon Mar 15 10:30:00 2010 +0000
summary:     Fixed Issue 1.

changeset:   1:550bad1893cf
user:        Carla <carla@example.net>
date:        Tue Mar 02 14:30:00 2010 +0000
summary:     Added README.

There are now also two branches in the repository:

carla$ hg branches
issue-1                        3:2cdcbb0a5bfd
default                        1:550bad1893cf

Using thg log, Carla can see how the new branch is attached to the default branch:

carla-pull-issue-1.png

The two heads are clearly visible and Carla updates to the issue-1 branch:

carla$ hg update issue-1
1 files updated, 0 files merged, 1 files removed, 0 files unresolved

Her working directory now looks like Alice’s working directory and Carla can examine the file. She is not entirely happy with the “modern” greeting chosen by Alice — Carla wants the good old classic “Hello” instead. They discuss this on the bug tracker and Alice agrees to make the change. Alice first checks that she is still on the issue-1 branch and she then fixes the bug:

alice$ hg branch
issue-1
alice$ echo "Hello, World!" > hello.txt
alice$ hg commit -m "Really fix Issue 1." 

Carla pulls the new change:

carla$ hg pull ../alice
pulling from ../alice
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
(run 'hg update' to get a working copy)

She can use hg update to go to the tip of her current branch (issue-1) or she can be explicit and say hg update issue-1 again:

carla$ hg update
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

She agrees that the bug is finally fixed.

Closing a Branch

The branch is no longer needed, so she marks it as closed:

carla$ hg commit --close-branch -m "Close branch, Issue 1 is fixed." 

This makes the issue-1 branch disappear from the hg branches list:

carla$ hg branches
default                        1:550bad1893cf

If she had not closed it, the hg branches list could easily become unwieldy. Next she updates to the default branch and merges with issue-1:

carla$ hg update default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
carla$ hg merge issue-1
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
carla$ hg commit -m "Merged fix for Issue 1." 

By updating to default first, she ensures that the merge changeset is created on default and not on issue-1, where it does not belong. Note how merging a branch is no different from merging any other two changesets — Mercurial has a very uniform history model.

Pushing a Branch

She will now push the whole thing to the main repository. She must use a --new-branch flag since she is pushing new named branches. This is to prevent accidentally pushing a named branch prematurely. Without the flag she gets a friendly reminder:

carla$ hg push ../main
pushing to ../main
searching for changes
abort: push creates new remote branches: issue-1!
(use 'hg push --new-branch' to create new remote branches)

Note

The --new-branch flag was introduced in Mercurial 1.6. Older versions of Mercurial required you to use --force instead. The danger of using --force is that it disables all checks, whereas --new-branch won’t let you do things like pushing multiple heads.

Sure enough, adding the flag makes the push go through:

carla$ hg push --new-branch ../main
pushing to ../main
searching for changes
adding changesets
adding manifests
adding file changes
added 6 changesets with 3 changes to 2 files

The main repository is now in this state:

carla-merge-issue-1.png

Longer Term Branches

While Alice was working on fixing Issue 1, Carla asked Bob to look into translating the hello.txt file into more languages. They agree to do Danish, German, and French and create Issue 2 for this. Like Alice, Bob will work on his own branch for this and he begins by pulling changes from the main repository:

bob$ hg pull ../main 
pulling from ../main
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
(run 'hg update' to get a working copy)
bob$ hg update
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
bob$ hg glog
@  changeset:   1:550bad1893cf
|  tag:         tip
|  user:        Carla <carla@example.net>
|  date:        Tue Mar 02 14:30:00 2010 +0000
|  summary:     Added README.
|
o  changeset:   0:e9eb044d45e0
   user:        Carla <carla@example.net>
   date:        Mon Mar 01 10:20:30 2010 +0000
   summary:     Initial import.

At this point, Alice’s branch had not yet been merged into the main repository, so Bob only sees the two first changesets made by Carla. He creates a branch and renames the hello.txt file to reflect its language:

bob$ hg branch issue-2
marked working directory as branch issue-2
(branches are permanent and global, did you want a bookmark?)
bob$ hg commit -m "Began Issue 2 branch." 
bob$ hg rename hello.txt hello.en.txt
bob$ hg commit -m "English translation." 

He quickly creates a Danish and a German version and commits those too:

bob$ echo "Hej, verden!" > hello.da.txt
bob$ echo "Hallo, Welt!" > hello.de.txt
bob$ hg add
adding hello.da.txt
adding hello.de.txt
bob$ hg commit -m "Danish and German translations." 

He needs some more time to research the French translation, so he postpones that change. His changeset graph is shown below and you’ll notice that his issue-2 branch has shown up in red:

bob-first-commits.png

Merging default into a Branch

Meanwhile, Carla has merged Alice’s bugfix into default:

carla-merge-issue-1.png

This bugfix is also appropriate for Bob’s branch, so he pulls from the main repository:

bob$ hg pull ../main
pulling from ../main
searching for changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 2 changes to 1 files (+1 heads)
(run 'hg heads' to see heads)
bob-pull-main.png

He now merges default into issue-2 in order to bring the bugfix to his branch:

bob$ hg merge default
merging hello.en.txt and hello.txt to hello.en.txt
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
bob$ hg commit -m "Merge in bugfix from default." 

Notice how smoothly Alice’s change to hello.txt was merged into the renamed hello.en.txt. Just like a good movie soundtrack, a good merge implementation is characterized by the fact that you do not notice it — it should just work and do the right thing. We will compare this to how Subversion handles this below. The resulting changeset graph looks like this:

bob-merge-bugfix.png

Using the colored branches, you can quickly see that the green issue-1 branch was merged into the blue default branch, and that the whole thing was merged into the pink issue-2 branch. The colors are huge help when you look at complex histories.

Bob has also finally figured out how to translate the file into French, so he makes a final commit before asking Carla to review the branch.

bob$ echo "Bonjour tout le monde !" > hello.fr.txt
bob$ hg add hello.fr.txt
bob$ hg commit -m "French translation." 

Happy with his work, he submits it to Carla, who pulls it into her repository:

carla$ hg pull ../bob
pulling from ../bob
searching for changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 5 changes to 4 files
(run 'hg update' to get a working copy)

Her repository now looks like this:

carla-pull-issue-2.png

Carla double-checks the files and decides to merge his work now. She updates to his branch to close it, and back to default to merge:

carla$ hg update issue-2
4 files updated, 0 files merged, 1 files removed, 0 files unresolved
carla$ hg commit --close-branch -m "Close branch, Issue 2 is fixed." 
carla$ hg update default
1 files updated, 0 files merged, 4 files removed, 0 files unresolved
carla$ hg merge issue-2
4 files updated, 0 files merged, 1 files removed, 0 files unresolved
(branch merge, don't forget to commit)
carla$ hg commit -m "Merged fix for Issue 2." 

The final changeset graph shows nicely how the default branch (in black) was merged into the issue-2 branch (in red), before issue-2 was merged into default again:

carla-merge-issue-2.png

This is a very typical pattern for longer lived branches: the default branch is periodically merged into the more experimental branch in order to propagate the latest bugfixes and changes. Keeping the branches in sync help make the merges easier. On each merge, Mercurial will find the nearest common ancestor and apply the changes since that point. With periodic merges, this ancestor point will not lie too far in the past, and the branches won’t have drifted too far away from each other.

Merging Renamed Files in Subversion

When merging two branches, it is crucial to be able to find the common ancestor between two branches. It is the changes made since this ancestor that needs to be merged into the target branch. Mercurial tracks this ancestor information automatically — it is an integral part of our history model and you have seen again and again in thg log.

The history in Subversion is not based on a graph like in Mercurial and for a very long time, Subversion had no support for tracking merges. This meant that you had to manually dig out the revision ranges you wanted to merge when invoking svn merge. Subversion 1.5 and later does track merges, but the support is incomplete. The implementation of merge tracking is described as extremely complex in Version Control with Subversion (the “SVN Book”). In particular, Subversion handles merges involving renamed files poorly.

Above, we merged the issue-2 branch into default. On default, the hello.txt file had been changed and on issue-2, the hello.txt file had been renamed to hello.en.txt. Mercurial did the right thing and applied the change to hello.en.txt while merging. Subversion, on the other hand, bails out with a message about a conflict:

$ svn merge --reintegrate file://$HOME/hello/branches/issue-2
--- Merging differences between repository URLs into '.':
A    hello.en.txt
   C hello.txt
Summary of conflicts:
  Tree conflicts: 1

Note also that one must use a special flag in Subversion when a branch is merged back into the trunk. No such flag is needed in Mercurial where there is nothing inherently special about the default branch (except that it is, well…, the default branch).

Grafting Changes

When named branches are in use for a longer time, it happens that changesets are needed on more than one branch. Above, Bob could simply pull all the changes from the main repository, but this is often not desirable because the change you want is mixed with other changes.

The typical scenario is a bugfix that is committed to default, when it should really have been committed to a branch for the last stable release. Consider a repository that has this graph:

carla-graft-before.png

Here two branches are used: default (in blue) for the ongoing development, and stable (in green) for bugfixes and releases. You can see how stable is merged into default after a bugfix in order to propagate bugfixes into the current development. When a release is made, the merge is made in the other direction — from default into stable — and a tag is added. The result is two parallel tracks of development, where the green stable branch always contain a subset of the changesets on the black default branch.

The final commit fixes an old bug and a little later, Carla realizes that this bug is also present in the 2.0 release they made earlier. The changeset should therefore really have been made on the stable branch. It is too late to simply merge default into stable, since that would also merge the experimental change.

The solution is to use hg graft to copy the changeset. While being on the stable branch, she does:

carla$ hg graft tip
grafting revision 12

Note

The graft command was introduced in Mercurial 2.0. In earlier versions you can use the transplant extension to copy changesets. It is a standard extension and you enable it by putting:

[extensions]
transplant =

in your configuration file. The transplan extension works by exporting the changeset as a patch and then applying this patch onto the target. If this fails, then you’ll have to resolve the conflict yourself based on a reject (.rej) file.

This copies the changeset to her branch:

carla-graft-after.png

Since the duplicate changeset has a different past than the original, it will get a new changeset hash. The copying is done by running three-way merge internally and so Carla will be able to resolve conflicts using her favorite three-way merge program. Most small bugfixes can be grafted cleanly, however. She can now merge the stable branch back into default:

carla$ hg update default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
carla$ hg merge stable
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
carla$ hg commit -m "Merge with stable." 

Note that changesets are not really moved with this technique. In a distributed environment, it is close to impossible to destroy history, so Mercurial will by default only let you append to your history. This means that fixing mistakes involve making a new changeset that undoes the mistake. In our case, we fixed the mistake by duplicating the changeset. When merging, these duplicates are automatically reconciled:

carla-graft-merged.png

Exercises

We will now split you into groups of five. Each group picks one who should play the role of Carla, i.e., who should pull from the others and decide what to merge. We will call this person C.

Now follow these steps:

  1. C starts by cloning an example repository from:

    https://bitbucket.org/aragost/cantons/
    

    When it has been cloned, C starts the built-in webserver in Mercurial with hg serve.

  2. The other group members make a clone from C. You’ll need to exchange IP addresses at this point. You can also enable the zeroconf extension, wait a second, and then do hg paths. Hopefully you will be able to see each other’s repositories in the list.

    On Windows, you should beware that since hg serve makes the repository available on port 8000 by default, Mercurial will try to name the clone something like 172.16.0.1:8000, which is an illegal filename on Windows. So you should specify a second argument to hg clone in order to give the clone a suitable name.

  3. In the clone, you will find a file named cantons.txt. It has a list of Swiss canton abbreviations. The abbreviations need to be expanded, e.g., the line saying:

    * FR
    

    should be changed to say:

    * FR: Fribourg
    

    C will appoint a range of letters to each of the other group members.

  4. Make a branch for your range, e.g., with hg branch range-a-b. Then expand each abbreviation. Make a commit after expanding the abbreviations for a single letter.

  5. While people are busy expanding abbreviations, C will add another section to the file. It should look like this:

    By Language
    ===========
    
    The official languages of Switzerland are French, German, Italian, and
    Romansh:
    
    French
    ------
    
    French is spoken in: VD, ...
    
    
    German
    ------
    
    German is spoken in: FR, ...
    
    
    Italian
    -------
    
    Italian is spoken in: GR, ...
    
    
    Romansh
    -------
    
    Romansh is spoken in: GR.
    

    Fill in the missing cantons for French, German, and Italian and commit the change. Wait for the others to finish expanding their abbreviations.

  6. When someone is finished expanding his abbreviations, he should use hg serve and let C know where to find his work, just like we did it above.

  7. C will now pull from other group members and merge their changes into default. If everybody has paid attention and edited their section only, then there should be no conflicts.

  8. Finally, note that Wikipedia is there to help you if you get stuck :-)

Next we will experiment with resolving conflicts:

  1. Have two group members edit the same line — one can add a line with the French canton name, the other with the Italian name.

  2. The conflict will be discovered when C pulls the changes and runs hg merge. Depending on the exact configuration of Mercurial, a merge tool will be run. Under Windows and under Linux, KDiff3 is the default tool. It looks like this when started:

    cantons-merge-before.png

    The bottom pane holds the result of the merge, and there is currently three merge conflicts. You can right-click on a <Merge Conflict> line and choose to resolve it by including lines from file B (your local version) or C (the other version). In our case we want to include both changes:

    cantons-merge-after.png

    Save the output and exit KDiff3. Mercurial will notice that the merge tool exited successfully and mark the files as resolved. Check with hg resolve --list.

    If Mercurial is not configured to start a merge tool, then you will have to edit the file yourself to remove the conflict markers inserted by Mercurial. Use hg resolve --mark cantons.txt when done to mark the file as resolved.

  3. Commit the result of the merge as normal.

Summary

We have shown you how named branches can be used to add structure to a Mercurial repository. They make it easy to work with several distinct branches of development at the same time, without mixing them up.