Distributed model for bundles (was: [TxMt] Bundle and language help)

[Allan Odgaard]

On 2. Aug 2007, at 17:41, Timothy Bates wrote:

> What is the procedure for making a new (small) new language bundle
> available?
>
>  Add to the SVN? (need privileges, of course)

The procedure to get it into the official repository is to make the  
bundle available then I can add it under Review generally with a  
bunch of review comments and when that is addressed it will be moved  
to non-review.

> Host it oneself?

I’ve wanted to write up something about this for some time now, but  
basically I want to get away from the centralized repository we  
currently have -- it puts all the administrative work on me  
(including judging what should be public etc.), and makes it too easy  
for people with too broad commit access to mess with bundles w/o  
first discussing the changes. At the same time it makes it difficult  
for people w/o commit access to contribute (and/or results in non- 
committers sending me a “replacement bundle” for something hosted,  
and leaves it up to me to track what was actually changed, and to  
check if the “replacement” was based on the latest bundle we had in  
the repository).

The solution I am pondering is switching to a distributed version  
control system (like git or mercurial). This has a few advantages  
such as allowing people w/o commit access to still do regular commits  
to the bundle they work on (applied to their local clone of the bundle).

Later they can either send the commits to the bundle maintainer or  
decide to simply make their bundle public. Because of how these  
version control systems are designed, it is no problem to initially  
clone e.g. the Blogging bundle from Brad Choate (which he would have  
made public), and then later switch to fetching updates from Brett  
Terpstra. Or maybe I would clone Brad’s version, make that public  
under the bundles.macromates.com domain, and when Brett makes his  
continued work public, I can cherry-pick some of his improvements for  
the bundle I am making public -- the user of course could decide to  
clone my bundle, and track my updates, but also himself cherry-pick  
some of Brett’s improvements that I didn’t pick.

IOW these (distributed) systems are designed to *not* have a single  
authority (which is ideal for bundles since many of the bundles have  
seen a change in primary maintainer) and they are designed to keep  
*all* the history (which is good because our bundles generally have  
many contributors, and sometimes one contribution (i.e. one patch I  
receive) is actually many changes, for which it would be nice to have  
the full history).

Also, rather than sending a patch to the maintainer, ideally the  
contributor makes his bundle public, and tells the maintainer to pull  
from that. In practice the maintainer of a bundle will have a public  
version and a local version, those are mostly in sync, and he pushes  
from his local clone to the public, and when a contributor tells him  
that the contributor has a change, the maintainer pulls from the  
contributors public bundle -- the maintainer can test that, and say  
there are some problems, he can fix those, push it as a branch to his  
public bundle, and ask the contributor to update to that, and go  
through his changes, and when the contributor okays the fixes, the  
maintainer merges the branch into master (here the contributor could  
again commit to the work-in-progress, lather, rinse, repeat).

The advantage here is a) it is simpler to exchange these “work in  
progress” patches, and b) all history is preserved. Also, while a  
feature is being developed, users can select to follow those changes,  
even though they do not happen in the “official” bundle (a bit like  
what we have recently started to do by using svn branches, just not  
sucking as much wrt branch handling ;) ).

The reason I am still hesitant to switch though is:

  1. If we make it completely distributed, we still need a central  
list of bundles and an easy way to grab them (so need to work out a  
system for that).

  2. While there are free DSCM hosting solutions, it would probably  
be good if I created one, as I think it would be ideal to have just  
one or two people able to commit to a bundle (and I’d rather not  
encourage cluttering the existing hosting services with TM bundles).  
So e.g. for the LaTeX bundle, both Brad, Haris, and Robin would each  
have their own bundle, and probably I would link to Haris’ as the  
official one, which Brad and Robin will track, and Haris will  
integrate their improvements as he sees fit.

  3. Learning curve; already some users are not too happy about the  
current reliance on svn, so I expect some resistance if switching to  
something even more alien. Also, the idea is that it will make it  
easier to contribute, which includes doing local commits, but often  
people do not realize that they want to contribute before after they  
have done many changes -- so I would like this to somehow be  
integrated, i.e. all bundles included with TM are clones and have  
full repository info, and each change you make in the bundle editor  
asks you for a commit message, and does a commit behind-the-scenes,  
which would also mean ditching my tmDelta system and simply rely on  
the version control system to figure all this out -- but all this is  
easier said than done, and I am not sure it is worth bothering all  
users with this.

  4. Sometimes we make batch changes to all bundles, e.g. when me  
moved the ruby plist bundle to osx/plist -- if we distribute the  
system, such batch changes are not really possible. Some goes for  
Michael’s work standardizing scope names. He would have to clone all  
bundles, work on these, and then ask all the maintainers to pull his  
changes (or give him push/commit access).

Comments about this subject encourages! :)