I met a bunch of smart people at the Montreal Ubuntu 9.04 release party, and I talked to one person a lot about the distributed SCM Git. I had heard of distributed SCM’s but had never had any reason to try anything other than SVN. Still, I was intrigued, and so decided to look into it. Here are some useful links:
For an overview of what Git is, see Linus Torvald’s tech talk at Google. For an overview of how to use Git, see Randall Schwartz’s talk at Google. For an overview of how I have so far successfully used it in my GSoC project, read on.
My goal was to use Git to solve a particular problem, which was as follows:
There is an SVN repository A, and a CVS repository B. The code in B was probably forked at some point from the code in A. B was iterated on in a private sandbox before being put into CVS, so B has no history, just HEAD. A has also changed since the time it was forked. I need to take the code in B and compare it to each of the major release tags in A, so as to determine which tag is the most similar to B, and therefore which tag B was most likely to have been forked from. This basically means diffing two different filesystem trees. Because Git is supposed to be very good at merging different branches based on content, I assumed it would be able to give me some very intelligent diff output.
In this contex, A is the GWT SVN repo, and B is a project in Eclipse’s CVS repo. The Eclipse project org.eclipse.swt.e4.jcl appears to be forked from GWT, so I needed to determine when it was forked, what changes were made after, and (hopefully) why.
The person I met at the Ubuntu party, Derek, wrote me very excellent instructions on how to do this, which I have gratuitously copied and pasted here.
Here are the general steps: - Use git-svn-clone to clone the Subversion repository A. - Checkout the CVS HEAD of repository B into a separate workspace. - Create a new branch in the git-svn repository. You can branch starting from any tag or commit, but the closer that the branch origin is to the original fork tag, the fewer differences you will likely encounter. So, try to correctly guess the fork tag and branch from that guess. - Overwrite the workspace associated with the new branch with all the files in the CVS HEAD workspace. Stage and then commit the changes into the Git repository. - For every Subversion "tag" branch in Git (Git stores every Subversion tag as a Git branch, not as a Git tag), display the differences between the branch that contains CVS HEAD and the Git Subversion "tag" branch. Note the one that gives you the fewest differences. Specific commands (untested): # Clone the Subversion repository A. Option "--stdlayout" tells Git that the repository has the standard branches, tags, trunk layout git svn clone --stdlayout http://domain.com/repositoryA # Create a branch from a tag git branch subversiontag cvsbranch # Switch to branch "cvsbranch" git checkout cvsbranch # Overwrite files in Git "cvsbranch" workspace with those from CVS HEAD workspace. Make sure that you don't copy the CVS metadeta directory in each workspace directory. Consider using rsync which can ignore CVS and Subversion metadata directories. # Stage all CVS changes git add --all # Commit CVS changes git commit --message "Added changes from CVS repository B" # List all Git Subversion tag branches git branch | grep svn # Compare each Git Subversion tag branch with tag "cvsbranch". Option "--name-only" displays only the names of the files that changed: for tag in `git branch -r | grep tags`; do git diff --name-only $tag cvsbranch; done # Note the tag that produced the fewest differing files.
While this mostly worked great, I ran into one gotch using Git from the Ubuntu 8.04 repo (git version 1.5.4.3), which was that git-svn did not capture all of the tags from svn. When I tried it with git from the 8.10 repo (git version 1.5.6.something), it worked fine. Really, I should be git-cloning the latest git, but I haven’t quite figured out how to do that successfully. Nevertheless, beware.
I also tried using svn2git, which is supposed to basically be svn-git with some manipulations to make the imported svn repo more git-like. I spent an hour trying to get this to work, and met with marginal success. It turned out not to be neccessary.
In the end, I ended up using the following code to guage similarity.
The first one just counts the number of files that changed between each tag and the cvs branch.
olpc@OLPC:~/workspace-gsoc/svn/user/super/com/google/gwt/emul/java$ for tag in `git-branch -a | grep tag`; do echo $tag; git-diff --relative --name-only $tag cvsbranch | wc; done;
tags/1.3.1
151 151 3767
tags/1.3.3
151 151 3767
tags/1.3.3@288
151 151 3767
tags/1.4.10
150 150 3746
tags/1.4.59
150 150 3746
tags/1.4.60
150 150 3746
tags/1.4.60@1399
150 150 3746
tags/1.4.61
150 150 3746
tags/1.4.61@1504
150 150 3746
tags/1.4.62
150 150 3746
tags/1.4.62@2104
150 150 3746
tags/1.5.0
78 78 1874
tags/1.5.0@2941
78 78 1874
tags/1.5.1
70 70 1709
tags/1.5.1@3391
70 70 1709
tags/1.5.2
70 70 1709
tags/1.5.2@3587
70 70 1709
tags/1.5.3
73 73 1756
tags/1.5.3@3776
73 73 1756
tags/1.6.0
78 78 1863
tags/1.6.0@4621
78 78 1863
tags/1.6.1
78 78 1863
tags/1.6.1@4846
78 78 1863
tags/1.6.2
78 78 1863
tags/1.6.2@5035
78 78 1863
tags/1.6.3
78 78 1863
tags/1.6.3@5110
78 78 1863
tags/1.6.4
78 78 1863
tags/1.6.4@5112
78 78 1863
tags/1.6.4@5189
78 78 1863
This second analysis actually counts the number of lines in each diff
between each tag and the cvs branch.
olpc@OLPC:~/workspace-gsoc/svn/user/super/com/google/gwt/emul/java$ for tag in `git-branch -a | grep tag`; do echo $tag; git-diff --relative $tag cvsbranch | wc; done; tags/1.3.1 18257 77427 515421 tags/1.3.3 18257 77427 515421 tags/1.3.3@288 18257 77427 515421 tags/1.4.10 18261 78017 524249 tags/1.4.59 18158 77486 520272 tags/1.4.60 18158 77486 520272 tags/1.4.60@1399 18158 77486 520272 tags/1.4.61 18158 77486 520272 tags/1.4.61@1504 18158 77486 520272 tags/1.4.62 18158 77486 520272 tags/1.4.62@2104 18158 77486 520272 tags/1.5.0 5667 22620 165036 tags/1.5.0@2941 5667 22620 165036 tags/1.5.1 4357 17759 127102 tags/1.5.1@3391 4357 17759 127102 tags/1.5.2 3920 15766 113373 tags/1.5.2@3587 3920 15766 113373 tags/1.5.3 4045 16322 117623 tags/1.5.3@3776 4045 16322 117623 tags/1.6.0 4974 19823 144457 tags/1.6.0@4621 4974 19823 144457 tags/1.6.1 4986 19881 144836 tags/1.6.1@4846 4986 19881 144836 tags/1.6.2 4986 19881 144836 tags/1.6.2@5035 4986 19881 144836 tags/1.6.3 4986 19881 144836 tags/1.6.3@5110 4986 19881 144836 tags/1.6.4 4986 19881 144836 tags/1.6.4@5112 4986 19881 144836 tags/1.6.4@5189 4974 19823 144457
So, it looks like 1.5.2 has significantly fewer changes than any other tag. The diff to 1.5.2 is attached to this email. It basically looked like 48 new files were added, 0 were removed and many were modified.
Once I had established which GWT SVN tag was most similar, it was trivial to diff the branches and get meaningful output:
olpc@OLPC:~/workspace-gsoc/svn/user/super/com/google/gwt/emul/java$ git-diff --relative cvsbranch tags/1.5.2 > 1-5-2.diff
I won’t post the resulting diff here, but suffice it to say that it was exactly what I was looking for, and I am now moving onto the task of analyzing the diff to figure out exactly what was changed, and why.
My final impressions of git: I’m very happy with it. The ability to import an entire SVN repo so that it can be used offline; the ease with which one may branch and merge; and the ability to push changes back up into the svn repo, are all things that will be very useful to me. I think I’m actually going to get to use Git a lot on my GSoC project.