Skip to content

A high-performance, drop-in replacement for Node.js fs module, powered by Rust.

License

Notifications You must be signed in to change notification settings

CoderSerio/rush-fs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

70 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rush-FS

English | 中文

Written in Rust NPM Version License Beta Contributors

API-aligned with Node.js fs for painless drop-in replacement in existing projects; get multi-fold performance in heavy file operations, powered by Rust.

Installation

npm install @rush-fs/core
# or
pnpm add @rush-fs/core

When you install @rush-fs/core, the package manager should automatically install the platform-specific native binding for your OS/arch via optionalDependencies (e.g. @rush-fs/rush-fs-darwin-arm64 on macOS ARM). If the native binding is missing and you see "Cannot find native binding", try:

  1. Remove node_modules and the lockfile (package-lock.json or pnpm-lock.yaml), then run pnpm install (or npm i) again.
  2. Or install the platform package explicitly:
    macOS ARM: pnpm add @rush-fs/rush-fs-darwin-arm64
    macOS x64: pnpm add @rush-fs/rush-fs-darwin-x64
    Windows x64: pnpm add @rush-fs/rush-fs-win32-x64-msvc
    Linux x64 (glibc): pnpm add @rush-fs/rush-fs-linux-x64-gnu

Migration from rush-fs: The package was renamed to @rush-fs/core. See CHANGELOG.md for details.

Usage

import { readdir, stat, readFile, writeFile, mkdir, rm } from '@rush-fs/core'

// Read directory
const files = await readdir('./src')

// Recursive with file types
const entries = await readdir('./src', {
  recursive: true,
  withFileTypes: true,
})

// Read / write files
const content = await readFile('./package.json', { encoding: 'utf8' })
await writeFile('./output.txt', 'hello world')

// File stats
const s = await stat('./package.json')
console.log(s.size, s.isFile())

// Create directory
await mkdir('./new-dir', { recursive: true })

// Remove
await rm('./temp', { recursive: true, force: true })

Benchmarks

Tested on Apple Silicon (arm64), Node.js 24.0.2, release build with LTO. Run pnpm build && pnpm bench to reproduce.

Where Rush-FS Shines

These are the scenarios where Rust's parallelism and zero-copy I/O make a real difference:

Scenario Node.js Rush-FS Speedup
readdir recursive (node_modules, ~30k entries) 281 ms 23 ms 12x
copyFile 4 MB 4.67 ms 0.09 ms 50x
readFile 4 MB utf8 1.86 ms 0.92 ms 2x
readFile 64 KB utf8 42 µs 18 µs 2.4x
rm 2000 files (4 threads) 92 ms 53 ms 1.75x
access R_OK (directory) 4.18 µs 1.55 µs 2.7x
cp 500-file flat dir (4 threads) 86.45 ms 32.88 ms 2.6x
cp tree dir ~363 nodes (4 threads) 108.73 ms 46.88 ms 2.3x
glob large tree (node_modules/**/*.json, ~30k entries) vs fast-glob 303 ms 30 ms ~10x

On Par with Node.js

Single-file operations have a ~0.3 µs napi bridge overhead. Recursive glob on a small tree is on par with node-glob; on large trees (e.g. node_modules) Rush-FS wins (see table above).

Scenario Node.js Rush-FS Ratio
stat (single file) 1.45 µs 1.77 µs 1.2x
readFile small (Buffer) 8.86 µs 9.46 µs 1.1x
writeFile small (string) 74 µs 66 µs 0.9x
writeFile small (Buffer) 115 µs 103 µs 0.9x
appendFile 30 µs 27 µs 0.9x
glob recursive (**/*.rs, small tree) vs node-glob 22 ms 40 ms 1.8x (node-glob faster at this scale)

Where Node.js Wins

Lightweight built-in calls where napi overhead is proportionally large:

Scenario Node.js Rush-FS Note
existsSync (existing file) 444 ns 1.34 µs Node.js internal fast path
accessSync F_OK 456 ns 1.46 µs Same — napi overhead dominates
writeFile 4 MB string 2.93 ms 5.69 ms Large string crossing napi bridge

Parallelism

Rush-FS uses multi-threaded parallelism for operations that traverse the filesystem:

API Library concurrency option Default
readdir (recursive) jwalk auto
glob ignore 4
rm (recursive) rayon 1
cp (recursive) rayon 1

Single-file operations (stat, readFile, writeFile, chmod, etc.) are atomic syscalls — parallelism does not apply.

Key Takeaway

Rush-FS excels at recursive / batch filesystem operations (readdir, glob, rm, cp) where Rust's parallel walkers deliver significant speedups (e.g. 12x readdir, 50x copyFile). For single-file operations it performs on par with Node.js. The napi bridge adds a fixed ~0.3 µs overhead per call, which only matters for sub-microsecond operations like existsSync.

cp benchmark detail (Apple Silicon, release build):

Scenario Node.js Rush-FS 1T Rush-FS 4T Rush-FS 8T
Flat dir (500 files) 86.45 ms 61.56 ms 32.88 ms 36.67 ms
Tree dir (breadth=4, depth=3, ~84 nodes) 23.80 ms 16.94 ms 10.62 ms 9.76 ms
Tree dir (breadth=3, depth=5, ~363 nodes) 108.73 ms 75.39 ms 46.88 ms 46.18 ms

Optimal concurrency for cp is 4 threads on Apple Silicon — beyond that, I/O bandwidth becomes the bottleneck and diminishing returns set in.

How it works

For the original Node.js, for example on readdir, directory reads run serially in the native layer, and each entry is turned into a JS string on the V8 main thread, which adds GC pressure:

graph TD
    A["JS: readdir"] -->|Call| B("Node.js C++ Binding")
    B -->|Submit Task| C{"Libuv Thread Pool"}

    subgraph "Native Layer (Serial)"
    C -->|"Syscall: getdents"| D[OS Kernel]
    D -->|"Return File List"| C
    C -->|"Process Paths"| C
    end

    C -->|"Results Ready"| E("V8 Main Thread")

    subgraph "V8 Interaction (Heavy)"
    E -->|"Create JS String 1"| F[V8 Heap]
    E -->|"String 2"| F
    E -->|"String N..."| F
    F -->|"GC Pressure Rising"| F
    end

    E -->|"Return Array"| G["JS Callback/Promise"]
Loading

With Rush-FS, for example on readdir, the hot path stays in Rust: build a Vec<String> (or use Rayon for recursive readdir), then hand one array to JS. No per-entry V8 allocation during the walk:

graph TD
    A["JS: readdir"] -->|"N-API Call"| B("Rust Wrapper")
    B -->|"Spawn Task"| C{"Rust (or Rayon pool if recursive)"}

    subgraph "Rust 'Black Box'"
    C -->|"Syscall: getdents"| D[OS Kernel]
    D -->|"Return file list"| C
    C -->|"Store as Rust Vec<String>"| H[Rust Heap]
    H -->|"No V8 yet"| H
    end

    C -->|"All Done"| I("Convert to JS")

    subgraph "N-API Bridge"
    I -->|"Batch create JS array"| J[V8 Heap]
    end

    J -->|Return| K["JS Result"]
Loading

Other sources of speed in Rush-FS: recursive readdir uses jwalk with a Rayon thread pool for parallel directory traversal; cp and rm (recursive) can use Rayon for parallel tree walk and I/O; glob runs with a configurable number of threads. Across APIs, keeping the hot path in Rust and handing a single result (or batched data) to JS avoids repeated V8/GC overhead compared to Node’s C++ binding.

Status & Roadmap

We are rewriting fs APIs one by one.

Legend

  • ✅: Fully Supported
  • 🚧: Partially Supported / WIP
  • ✨: New feature from @rush-fs/core
  • ❌: Not Supported Yet

readdir

  • Node.js Arguments:
    path: string; // ✅
    options?: {
      encoding?: string; // 🚧 ('utf8' default; 'buffer' not supported)
      withFileTypes?: boolean; // ✅
      recursive?: boolean; // ✅
      concurrency?: number; // ✨
    };
  • Return Type:
      string[]
      | {
        name: string, // ✅
        parentPath: string, // ✅
        isDir: boolean // ✅
      }[]

readFile

  • Node.js Arguments:
    path: string; // ✅
    options?: {
      encoding?: string; // ✅ (utf8, ascii, latin1, base64, base64url, hex)
      flag?: string; // ✅ (r, r+, w+, a+, etc.)
    };
  • Return Type: string | Buffer

writeFile

  • Node.js Arguments:
    path: string; // ✅
    data: string | Buffer; // ✅
    options?: {
      encoding?: string; // ✅ (utf8, ascii, latin1, base64, base64url, hex)
      mode?: number; // ✅
      flag?: string; // ✅ (w, wx, a, ax)
    };

appendFile

  • Node.js Arguments:
    path: string; // ✅
    data: string | Buffer; // ✅
    options?: {
      encoding?: string; // ✅ (utf8, ascii, latin1, base64, base64url, hex)
      mode?: number; // ✅
      flag?: string; // ✅
    };

copyFile

  • Node.js Arguments:
    src: string; // ✅
    dest: string; // ✅
    mode?: number; // ✅ (COPYFILE_EXCL)

cp

  • Node.js Arguments (Node 16.7+):
    src: string; // ✅
    dest: string; // ✅
    options?: {
      recursive?: boolean; // ✅
      force?: boolean; // ✅ (default: true)
      errorOnExist?: boolean; // ✅
      preserveTimestamps?: boolean; // ✅
      dereference?: boolean; // ✅
      verbatimSymlinks?: boolean; // ✅
      concurrency?: number; // ✨
    };

mkdir

  • Node.js Arguments:
    path: string; // ✅
    options?: {
      recursive?: boolean; // ✅
      mode?: number; // ✅
    };
  • Return Type: string | undefined (first created path when recursive)

rm

  • Node.js Arguments:
    path: string; // ✅
    options?: {
      force?: boolean; // ✅
      maxRetries?: number; // ✅
      recursive?: boolean; // ✅
      retryDelay?: number; // ✅ (default: 100ms)
      concurrency?: number; // ✨
    };

rmdir

  • Node.js Arguments:
    path: string // ✅

stat

  • Node.js Arguments:
    path: string // ✅
  • Return Type: Stats
    • Numeric fields: dev, mode, nlink, uid, gid, rdev, blksize, ino, size, blocks, atimeMs, mtimeMs, ctimeMs, birthtimeMs
    • Date fields: atime, mtime, ctime, birthtimeDate objects ✅
    • Methods: isFile(), isDirectory(), isSymbolicLink(), ...
  • Error distinction: ENOENT vs EACCES

lstat

  • Node.js Arguments:
    path: string // ✅
  • Return Type: Stats

fstat

  • Status: ❌

access

  • Node.js Arguments:
    path: string; // ✅
    mode?: number; // ✅ (F_OK, R_OK, W_OK, X_OK)

exists

  • Node.js Arguments:
    path: string // ✅
  • Return Type: boolean

open

  • Status: ❌

opendir

  • Status: ❌

close

  • Status: ❌

unlink

  • Node.js Arguments:
    path: string // ✅

rename

  • Node.js Arguments:
    oldPath: string // ✅
    newPath: string // ✅

readlink

  • Node.js Arguments:
    path: string // ✅
  • Return Type: string

realpath

  • Node.js Arguments:
    path: string // ✅
  • Return Type: string

chmod

  • Node.js Arguments:
    path: string // ✅
    mode: number // ✅

chown

  • Node.js Arguments:
    path: string // ✅
    uid: number // ✅
    gid: number // ✅

utimes

  • Node.js Arguments:
    path: string // ✅
    atime: number // ✅
    mtime: number // ✅

truncate

  • Node.js Arguments:
    path: string; // ✅
    len?: number; // ✅

glob

  • Node.js Arguments:
    pattern: string; // ✅
    options?: {
      cwd?: string; // ✅
      withFileTypes?: boolean; // ✅
      exclude?: string[]; // ✅
      concurrency?: number; // ✨
      gitIgnore?: boolean; // ✨ default false (align with Node.js fs.globSync)
    };

symlink

  • Node.js Arguments:
    target: string // ✅
    path: string // ✅
    type?: 'file' | 'dir' | 'junction' // ✅ (Windows only, ignored on Unix)

link

  • Node.js Arguments:
    existingPath: string // ✅
    newPath: string // ✅

mkdtemp

  • Node.js Arguments:
    prefix: string // ✅
  • Return Type: string
  • Uses OS-level random source (/dev/urandom on Unix, BCryptGenRandom on Windows) with up to 10 retries ✅

watch

  • Status: ❌

Changelog

See CHANGELOG.md for a summary of changes in each version. Release tags are listed in GitHub Releases.

Contributing

See CONTRIBUTING.md for the full development guide: environment setup, Node.js reference, Rust implementation, testing, and benchmarking.

Publishing (Maintainers Only)

Releases are handled by the Release workflow: it builds native binaries for macOS (x64/arm64), Windows, and Linux, then publishes the platform packages and the main package to npm.

  1. Secrets: In the repo Settings → Secrets and variables → Actions, add NPM_TOKEN (npm Classic or Automation token with Publish permission).
  2. Release: Either run Actions → Release → Run workflow (uses the current package.json version on main), or bump version in package.json and Cargo.toml, push to main, then create and push a tag: git tag v<version> && git push origin v<version>.
  3. Changelog: Update CHANGELOG.md before or right after the release (move entries from [Unreleased] to a new version heading and add the compare link).

The workflow injects optionalDependencies and publishes all packages; no need to edit package.json manually for release.

License

MIT

About

A high-performance, drop-in replacement for Node.js fs module, powered by Rust.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages