From oleg@pobox.com Fri Aug 21 12:59:32 1998
Message-Id: <6rkjin$90g$1@nnrp1.dejanews.com>
From: oleg@pobox.com
To: oleg@pobox.com
Subject: Bridging HTTP FS and a Network File System MCFS
Date: Fri, 21 Aug 1998 19:59:18 GMT
Keywords: HTTP, MC, VFS, CGI, NFS, virtual file system, WinNT, IIS
Newsgroups: comp.os.linux.development.system,comp.infosystems.www.misc,comp.os.linux.misc,comp.client-server,comp.os.misc
CC: mc-devel@nuclecu.unam.mx
Organization: Deja News - The Leader in Internet Discussion
Summary: Announcing adapter between an HTTP Virtual File System and Midnight Commander's MCFS
X-Article-Creation-Date: Fri Aug 21 19:59:18 1998 GMT
X-Http-User-Agent: Mozilla/4.06 (Macintosh; I; PPC, Nav)
Status: RO

This is to announce an adapter between an HTTP virtual file system and
Midnight Commander's native MCFS. The latter is a NFS-like file system
over a TCP-based "RPC" protocol. The adapter makes it possible for an
_unmodified_ Midnight Commander (MC) to view, write, copy etc. files
on a remote computer, which can be _any_ UNIX or Win95/WinNT box
capable of running a Perl CGI script. The adapter features an
intelligent caching of remote resources -- files and directories --
maintained via a list of VNodes. It fully supports reading as well
_writing_/truncating/creating of files on a remote HTTP host. As MCFS
implements a regular filesystem API -- open(), opendir(), write(),
read(), stat() "calls" -- the adapter may serve as an example of a
network file system based on HTTP/CGI protocols.

The adapter, mcserv, is a server for MCFS and a client for HTTP
VFS. It translates file/directory access orders from MC into HTTP FS
requests, and maps HTTP VFS server's responses back to the MCFS
client. The adapter maintains a local cache of fetched files and
directories: VNodes. If a request comes for a cached resource, the
adapter politely submits an 'If-modified-since:' HTTP header to the
server. The adapter currently implements the following MCFS commands:
        open(), read(), write(), lseek(), close(), stat(), lstat(), fstat(),
        opendir(), readdir(), closedir(), readlink().
The adapter fully supports all standard open() modes: O_RDONLY as well
as O_RDWR, O_WRONLY, O_CREAT, O_EXCL, O_TRUNC and O_APPEND. 180+ lines
of the title comments in mchttp-client.cc explain in detail how the
above commands are mapped into HTTP requests, and how VFS server's
responses are translated back.

A MCHFS-server.pl -- a CGI Perl script -- is a HTTP VFS server that
takes requests from the adapter above and does something
intelligent. The title comments to the script (215 lines) talk in more
detail what exactly that means. I have run the server on HP-UX boxes
under Netscape and Apache HTTP servers, and on a WinNT box running
IIS. This made file systems of these computers available to a MC on a
Linux box -- as if they were Midnight Commander's local
filesystems. The adapter was run either on the Linux box itself, or
some third (HP-PA) computer. As was mentioned above, the Linux box
runs a "virgin" 4.1 version of MC (in particular, the one that is
included in a S.u.S.E. Linux distribution).

A MC-HTTP-FS archive contains the complete and self-contained source
code for the server and the adapter, and an INSTALL document. A
MC-HTTP-FS.dr file in the archive tells what all the other files are
for.

I'm yet to implement unlink(), rename(), mkdir(), and chmod() VFS
calls. I should also look into persistent HTTP connections that HTTP
1.1 allows and encourages. An HTTP option of transmitting of only
selected pieces of a requested file (a byte range) also looks
appealing.


References

The MCFS-HTTPFS adapter distribution,
        [Obsolete URL to http-mcfs.tar.gz] [102,407 bytes]

The original announcement of the HTTP VFS
        http://pobox.com/~oleg/ftp/HTTP-VFS.html


Please mail comments, questions, trouble reports to me.

        Thank you,
                Oleg