Hi all!
[For the impatient reader: Just some food for thought, no solution given].
The discussion about remote projects made me think that ideally Virtual File Systems (VFS) really do not belong into the editor, rather they should be part of the OS. An example is the OS X webdavfs [1], which allows to mount webdav servers with the Finder, or the (poorly working and read only) ftp file system [2]. Something like that for SFTP would be the ultimate solution.
Under linux there is an interesting project called Fuse [3], which basically hooks a generic driver into the kernels VFS and communicates with user-space file system 'modules' (the OS X webdavfs works somewhat similar). Someone else recently wrote a driver for FreeBSD [4] that allows to use the Fuse file system modules.
The problem is that the OS X VFS implementation seems to be rather under-documented and auite the moving target [5]. It changed a lot for Tiger. It has developed independent from FreeBSD, so porting the FreeBSD Fuse driver is not at all trivial (lack of documentation again not helping).
I briefly entertained a bridge that would make a remote sftp server appear like a webdav server, allowing it to be mounted locally using the webdavfs and without extra software on the server. While this is relatively easy to do (at least compared to the alternatives), it has one major back-draw: DAV [6] (complex [dare I say convoluted] as it seems) does not support the notion of file permissions. And the thought of making all files pushed to the server executable (or not executable) is not very appealing.
The ftpfs [2] on OS X uses yet another approach: It presents itself to the OS as an NFS server, and translates NFS to FTP. This may seem like a convoluted approach (and it is), but has the advantage that it can be completely implemented in user space, and the NFS protocol encapsulates it from changes in the kernels VFS implementation. So one could write a bridge/gateway program that serves NFS and translates it to SFTP. Or (though I have not had a look at feasibility) it might even be possible to make this Fuse compatible, giving easy access to many of the file system modules available for Fuse ([7], or makings of a RailsFS [8]). Sometimes I wish I wouldn't have to work for food...
Gerd
[1] man mount_webdav [2] man mount_ftp [3] http://fuse.sourceforge.net/ [4] http://wikitest.freebsd.org/moin.cgi/FuseFilesystem [5] http://developer.apple.com/qa/qa2001/qa1242.html [6] http://www.webdav.org/ [7] http://fuse.sourceforge.net/filesystems.html [8] http://redhanded.hobix.com/inspect/ railsfsAfterACoupleMinutesOfToolingWithFuseWhoa.html
On Sep 23, 2005, at 4:11 PM, Gerd Knops wrote:
The problem is that the OS X VFS implementation seems to be rather under-documented and auite the moving target [5]. It changed a lot for Tiger. It has developed independent from FreeBSD, so porting the FreeBSD Fuse driver is not at all trivial (lack of documentation again not helping).
... The ftpfs [2] on OS X uses yet another approach: It presents itself to the OS as an NFS server, and translates NFS to FTP. This may seem like a convoluted approach (and it is), but has the advantage that it can be completely implemented in user space, and the NFS protocol encapsulates it from changes in the kernels VFS implementation.
...
Erm, I'd like to point out that, as [5] mentions, in Tiger VFS's are now fairly well insulated from Kernel changes, and if I remember WWDC correctly, the Kernel changes in 10.4 should make things like VFS's a lot more forward-compatible. So it really isn't so much the moving target anymore.
On Fri, Sep 23, at 3:28 PM, Kevin Ballard wrote:
On Sep 23, 2005, at 4:11 PM, Gerd Knops wrote:
Erm, I'd like to point out that, as [5] mentions, in Tiger VFS's are now fairly well insulated from Kernel changes, and if I remember WWDC correctly, the Kernel changes in 10.4 should make things like VFS's a lot more forward-compatible. So it really isn't so much the moving target anymore.
True, and a kernel-level interface to something like Fuse would be the more efficient and elegant solution. Though considering their comments about stacking VFS plug-ins in that same document shows that they do tend to change their minds. Also binary compatibility for kexts between OS releases is very hard to maintain, and the fact remains that documentation is poor and kernel-level development is quite the black magic. NFS on the other hand is pretty well understood and they will think twice about dropping NFS support. But all this is philosophical until someone actually sits down and does something about this...
Gerd
On Sep 23, 2005, at 5:19 PM, Gerd Knops wrote:
On Fri, Sep 23, at 3:28 PM, Kevin Ballard wrote:
On Sep 23, 2005, at 4:11 PM, Gerd Knops wrote:
Erm, I'd like to point out that, as [5] mentions, in Tiger VFS's are now fairly well insulated from Kernel changes, and if I remember WWDC correctly, the Kernel changes in 10.4 should make things like VFS's a lot more forward-compatible. So it really isn't so much the moving target anymore.
True, and a kernel-level interface to something like Fuse would be the more efficient and elegant solution. Though considering their comments about stacking VFS plug-ins in that same document shows that they do tend to change their minds.
They've been pretty consistent on this, actually. Remember, that was over four years ago. Ever since, they've been warning developers that the VFS interface would change in the future. They inherited the VFS architecture from BSD, remember, they didn't create it. The stacking stuff came "for free". Tiger is the first time they've made a commitment to VFS binary compatibility.
Also binary compatibility for kexts between OS releases is very hard to maintain, and the fact remains that documentation is poor and kernel-level development is quite the black magic. NFS on the other hand is pretty well understood and they will think twice about dropping NFS support. But all this is philosophical until someone actually sits down and does something about this...
Probably someone with either corporate support or a ton of free time will have to sit down and dive through the WebDAV FS code to figure out how to build a FuseFS equivalent.
Chris
-
Chris Thomas -
Gerd Knops -
Gerd Knops -
Kevin Ballard