RFS : The Replicated File System

This application is my first attempt to play with so called P2P & GRID environments. It is not a finite product, and does not suffer any comparison with 'real-life' P2P software (like eMule or BitTorrent for example). Actually it does not deserve the same goal, even if a real working stuff was not the idea behind this small project... but let me try to play the game anyway :-P

Functional Description

The basic principle of RFS is, as the name says, to replicate a FileSystem (actually the contents of a folder on a computer's drive) on other hosts, all being inter-connected thtough a P2P-like network. Each RFS node can export a local folder to its 'peers', and receives events from them as well.

Basically, each RFS node has local threads that scans the exported folder, and detect changes in it. When started (the UI allows to start/stop local scanning), they will look for :

• new files
• updated files
• deleted files

When a change is detected in the export folder, the node broadcasts the appropriate event to its peers. Then, the event is handled by the peers and the change on the remote folder is reflected locally (via download - new and updated files - or local deletion).

What can I do with this ?

I quickly thought about the following stuff :

• sharing files with friends : it's a pretty cool way to share your documents, MP3s, photos etc easily ! Just drop your stuff in the export folder and your mates will get it directly ;
• replicated back-ups : RFS can be installed on a computer to back-up some data from there on several other hosts in a seamless way. Moreover, it can be manipulated remotely ;
• collaborative work : replication allows to keep documents automatically synchronized, and everybody's got them locally, so you don't need to worry about getting everything before you go off-line ;
• ...

Missing features

The following features are currently missing in RFS.

• deep sync : allow to select a node and sync with it, with the possibility to select files individually) ;
• disconnected mode support (keep unsent events and re-send when peer available : at the moment you miss everything that occurs while you're not connected) ;
• multiple export folder support (at the moment you can only export 1 single folder) ;
• port range definition (ports are more or less obtained randomly, so firwalling is difficult)
• better implementation of file monitoring (the current one is heavy and unefficient, e.g. locked files in the export dir !) ;
• ...

NOTE

There is absolutely no plan that they will be implemented some day !

Installation & Usage

RFS is based on the Java platform (SE), and relies on the great ProActive GRID & P2P library from INRIA. RFS includes the required libraries from ProActive, but you've got to have a JDK (>= 1.4) installed and configured.

Installation

Installing RFS is pretty easy, simply unpack the distribution archive where you want.

Configuration

Have a look at /etc/rfs.properties and set the values in there. You also have to edit /bin/init.sh and set the install root dir as the RFS_HOME environment variable.

TODO

Explain LAN & Internet deploys !

Usage

To use RFS, first you have to start an RFS node. You have two options here : either starting alone, or joining an existing network, and this is done via command-line arguments passed to the startup script (at the moment you have to open a command prompt to start everything).

It goes a little somethin' like this :

%> cd <RFS_INSTALL_DIR>/bin
%> rfs <node_name> <join_through_name>@<join_through_host>

With :

• node_name : the name of the node (like a pseudo)
• join_through_name : the name of the node you want to join through
• join_through_host : the host where the node you want to join through is running

This should bring up the RFS console, a basic UI that allows you to start/stop the file scanning thread, and see events and downloads.

You can control (start or stop) the file scanning thread to avoid overhead on your host if you don't plan to export but just receive : this thread is used only for export matters, you will still be synchronized with other's exports even if you stop local scanning. Also, changes to your export folder should be detected by your node when the thread starts again next time, so don't hesitate to turn it off sometimes, specially if there's no activity in your local export dir.

Don't hesitate to email me (remi@rvkb.com) if you have any question about RFS.

Have fun,

Remi - remi@rvkb.com