Oxfs: A Fast SFTP File System

YOU NEED OXFS

Oxfs is a user-space network file system similar to SSHFS, and the underlying data transfer is based on the SFTP protocol. Oxfs introduces an asynchronous refresh policy to solve the jamming problem caused by the mismatch between network speed and user operation file speed. When Oxfs writes a file, it first writes to the local cache file and submits an asynchronous update task to update the content to the remote host. Similarly, when reading a file, it is preferred to read from a local cache file. Oxfs's data cache eventually falls to disk, and even if it is remounted, the history cache can still be used.

It can be said that Oxfs is a better alternative to SSHFS :)

GET STARTED

Linux

Setup

$ sudo apt-get install python3.7
$ python3.7 -m pip install oxfs --user

Run Oxfs

$ mkdir remote
$ sudo oxfs -s user@xxx.xxx.xxx.xxx -m remote -r /home/oxfs -p /tmp/oxfs

Darwin

Setup

$ brew install python3
$ mkdir ~/.venv
$ python3 -m venv ~/.venv/oxfs
$ source ~/.venv/oxfs/bin/activate
$ pip install oxfs

Run Oxfs

$ mkdir remote
$ sudo oxfs -s user@xxx.xxx.xxx.xxx -m remote -r /home/oxfs -p /tmp/oxfs

Get Help

$ oxfs --help
usage: oxfs [-h] [-s HOST] [-m MOUNT_POINT] [-r REMOTE_PATH] [-p CACHE_PATH]
            [-l LOGGING] [-d] [-v]

optional arguments:
  -h, --help            show this help message and exit
  -s HOST, --host HOST  ssh host (for example: root@127.0.0.1)
  -m MOUNT_POINT, --mount_point MOUNT_POINT
                        mount point
  -r REMOTE_PATH, --remote_path REMOTE_PATH
                        remote path, default: /
  -p CACHE_PATH, --cache_path CACHE_PATH
                        oxfs files cache path
  -l LOGGING, --logging LOGGING
                        set log file, default: /tmp/oxfs.log
  -d, --daemon          run in background
  -v, --verbose         print verbose info

PERFORMANCE

Environment

VPS: BandwagonHost (SPECIAL 10G KVM PROMO V3 - LOS ANGELES - CN2)
VPS Operating System: Centos 7 x86_64 bbr
Host: Intel® Core™ i5-4210U CPU @ 1.70GHz × 4 , SSD 125.5 GB
Host Operating System: Ubuntu 18.04.2 LTS x86_64 reno
Ping Latency: 190 ms

Method

$ iozone -R -b output.xls -p -c -r 1k -s 20m -l 1 -u 1

OXFS vs SSHFS

SSHFS performance (default parameters).

SSHFS enable the kernel cache by default, that’s why re-read is deadly fast.

Oxfs performance (default parameters).

Oxfs just enable the fuse auto_cache flag.

A dead simple, fast SFTP file system