• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

OpenZFS NAS (BSD, Illumos, Linux, OSX, Solaris, Windows + Storage Spaces) with napp-it web-gui

As an additional note:

The new rc10 with the integrated Perl web server and client-server encryption is copied to /opt (no longer /xampp, as XAMPP for Windows is no longer required). This also resolves the critical Apache < v2.4.64 security issue in XAMPP.

If you want to update an older napp-it cs in /xampp/csweb-gui and retain settings like jobs or groups, copy the config folder _cfg (and _log, if present) to /opt/csweb.

Under Windows, Windows Defender may flag nc64.exe as a hacker tool. This is Netcat for Windows, which Napp-it uses for ZFS replication (any OS to any OS) because it is already included by default in Linux or Unix. Without nc64, LAN replication to or from Windows is not possible. As an alternative, there is S3 backup/restore, which can be used to transfer any folders or file systems, including ACL save/restore.
 
OpenZFS 2.4.1 on Windows

https://github.com/openzfsonwindows/openzfs/releases
https://github.com/openzfsonwindows/openzfs/issues



** rc12

a lot of big changes, test calmly

  • fix MountManager confusion from snapshot mounts
  • fix zfs_vss_pool_remove not removing snapshots on pool export
  • zfs_vnops_windows_lib: also filter snapshot DCBs from BusRelations
  • zfs_vnops_windows_lib: exclude snapshot mounts from BusRelations reply
  • zfs_ctldir: return mounted snapshot root vnode from zfsctl_root_lookup
  • icp: fix AVX2-VAES GCM path for both kernel and userland on Windows
  • icp/simd: add VAES and VPCLMULQDQ runtime detection, enable AVX2-VAES GCM path
  • icp: restore ASMABI on aesni_gcm_encrypt/decrypt declarations
  • zfs_windows_unmount: rmdir reparse-point directory after deletion
  • zfs_windows_unmount: clear DCB mounted_on before DeleteReparsePoint
  • zfs_vnops_windows_lib: fix FileStandardInformation size for ADS
  • wosix_open: strip trailing backslash from paths
  • posix/zutil: fix zpool import -d . from UNC CWD
  • zfs_vnops_windows: fix FileStreamInformation for directories
  • zfs_vnops_windows: fix fsutil fsinfo volumeinfo storage IOCTL failures
  • zfs_vnops_windows: advertise FILE_SUPPORTS_BLOCK_REFCOUNTING when feature is enabled
  • zfs_vnops_windows: implement FileLinkInformationEx support
  • zfs_vnops_windows: fix git clone (pack file corruption + unlink failure)
  • zfs_vnops_windows: implement FSCTL_OFFLOAD_READ and FSCTL_OFFLOAD_WRITE
  • zfs_vss/vfsops/vnops: fix three BSODs in VSS snapshot path

** rc11

  • Show selected letter in driveletter property
  • Delete required admin privileges, fixed
  • Unmounted BSOD fix
  • Fix mount timeout
  • Volume Shadow Copy provider for Previous Versions
 
Napp-it cs rc10.1
- improved non blocking behaviour of webserver.pl
- backup job for system settings
- System > Backup/Restore, manual backup, restore of all or selected system settings.
- Disks > Enclosure Maps (visualisation of disk locations in a backplane, set + printout)

eg (per member several maps, unterschiedliche Brands, horizontal, vertical 2,4,6,8,16,24,60,90 disks,
no special hardware. create Maps/ different brands, assign disks, printout

1782374768979.png
 
#############

## napp-it cs rc 10.2_dev


- encrypted zstream instead ssh, netcat or mbuffer for remote replication
Release v1.1.0 · guenther-alka/zstream

Go application, transfer is encrypted, has buffering like mbuffer,
can limit transfer like pv, can show progress


- Antivir like Avast may block zstream.
This is a common false positiv problem with Go apps, allow zstream


# Windows dev release ps or cmd shell (copy paste to an admin terminal)
cmd /c "curl.exe -ksLo %TEMP%\cs_setup_dev.bat https://www.napp-it.org/cs_setup_dev.bat && %TEMP%\cs_setup_dev.bat


# non Windows dev release
curl -ksL https://www.napp-it.org/cs_setup_dev.sh | sh
 
AI for development and new features

The new Claude 5 models (Sonnet 5, Fable 5) represent a true epochal leap in the quality and efficiency of AI-assisted software development. While they still make mistakes and require diligent verification of their output, this is precisely where agentic models come into play. Claude is not only capable of directly versioning and modifying code, but can also verify it independently—for instance, by accessing logs or dynamically generating improved logging on the fly. In a multi-OS cluster solution like napp-it cs, tools like pscp or plink allow this loop to extend seamlessly to remote FreeBSD, Illumos, Linux, macOS, Solaris, or Windows member nodes.

I am currently using Windows for development via Claude Desktop and MCP, with other operating systems serving as the member servers.

howto


Code: 
================================================================================
  napp-it cs -- Claude Desktop Setup (English)
  AI-assisted development with direct filesystem access,
  SSH tools and ZFS snapshot backup
  File: csweb-gui/_my/howto.ai/setup_claude_en.info
================================================================================

================================================================================
  Preword
================================================================================
napp-it is a copy and run user extendable web management suite to manage single
servers or whole servergroups. It consists of the frontend webserver (pure Perl)
and backend services for remote control via encrypted socket connections. There
are no requirements/ modifications outside the napp-it cs folder /opt/csweb-gui

Frontend and backend run on Free-BSD, Illumos, Linux, OSX, Solaris and Windows.


================================================================================
  OVERVIEW
================================================================================

This setup gives Claude full access to the napp-it cs project (C:\opt\csweb-gui)
without any ZIP upload/download cycle. Claude reads and edits files directly on
disk. Version control is handled via robocopy mirroring to ZFS snapshots on an
OmniOS/Illumos NAS.

Components:
  1. Claude Desktop for Windows       -- AI client with MCP extensions
  2. MCP Filesystem Extension         -- direct read/write access to C:\opt
  3. MCP Desktop Commander            -- Windows terminal, shell, processes
  4. SSH tools (PuTTY suite)          -- remote access to member servers
  5. Robocopy + ZFS snapshots         -- backup and versioning


================================================================================
  1. PREREQUISITES
================================================================================

  - Windows 10/11 (64-bit)
  - napp-it cs installed at C:\opt\csweb-gui
  - Strawberry Perl at C:\opt\perl (included in the napp-it cs package)
  - NAS / OmniOS host with ZFS reachable on LAN (here: 192.168.2.203)
  - SMB share "opt" on the NAS (\\192.168.2.203\opt) as sync target
  - Internet access for Claude Desktop download


================================================================================
  2. INSTALL CLAUDE DESKTOP
================================================================================

  1. Download: https://claude.ai/download  -> "Download for Windows"
  2. Run the installer (claude-setup.exe) -- no admin rights required
  3. Start Claude Desktop, sign in with your claude.ai account
     (Recommended: Claude Pro, approx. $17/month -- larger context window,
      no throttling during intensive development sessions)
  4. On first start the chat opens directly -- no further setup needed


================================================================================
  3. SET UP MCP FILESYSTEM EXTENSION
================================================================================

  The Filesystem extension gives Claude direct read/write access to selected
  directories -- no file upload needed.

  3.1  Install the extension

    Claude Desktop -> Menu (☰ top left) -> Settings -> Extensions
    -> Search "Filesystem" -> Install / Add

    Alternatively: Claude Desktop -> Settings -> Developer -> Edit Config
    Open: %APPDATA%\Claude\claude_desktop_config.json

  3.2  Configure allowed directories

    Edit the filesystem entry in claude_desktop_config.json:

    {
      "mcpServers": {
        "filesystem": {
          "command": "npx",
          "args": [
            "-y",
            "@modelcontextprotocol/server-filesystem",
            "C:\\opt"
          ]
        }
      }
    }

    This grants Claude access to C:\opt\ and all subdirectories.
    NOTE: C:\opt\ssh\ contains the private SSH key -- Claude only reads it
    for command construction and never writes to it.

  3.3  Restart Claude Desktop

    After changing claude_desktop_config.json: fully quit Claude Desktop
    (tray icon -> Quit) and restart.

  3.4  Test access

    In chat: "List the files in C:\opt" -- Claude should list the folders
    without any file having been uploaded.


================================================================================
  4. SET UP MCP DESKTOP COMMANDER (Windows Terminal / Shell)
================================================================================

  Desktop Commander gives Claude access to a persistent Windows shell
  (PowerShell/cmd), can start processes, search files, and more.

  4.1  Installation

    Claude Desktop -> Settings -> Extensions -> Search "Desktop Commander"
    -> Install

    Or manually in claude_desktop_config.json:

    {
      "mcpServers": {
        "desktop-commander": {
          "command": "npx",
          "args": ["-y", "@wonderwhy-er/desktop-commander"]
        }
      }
    }

  4.2  Restrict allowed directories (optional)

    Desktop Commander can execute commands system-wide. For tighter control:
    set allowedDirectories to ["C:\\opt"] in Desktop Commander settings.

  4.3  Typical use

    Claude can directly:
      - Run perl -c syntax checks
      - Start sync_version.bat
      - Call plink.exe / pscp.exe
      - Start/stop processes (server.pl, webserver.pl)
      - Tail logs (Get-Content ... -Wait)


================================================================================
  5. SET UP SSH TOOLS (PuTTY suite)
================================================================================

  SSH tools give Claude remote access to member servers and the NAS host
  for snapshot management and log analysis.

  5.1  Install PuTTY tools

    Place the following files under C:\opt\ssh\:
      plink.exe      SSH command-line client   -- from putty.org
      pscp.exe       SCP/SFTP file transfer    -- from putty.org
      puttygen.exe   Key conversion utility    -- from putty.org
      id_rsa.ppk     Private key (PuTTY format)

    Download: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
    -> download plink.exe, pscp.exe, puttygen.exe directly

  5.2  Create or convert an SSH key

    Option A -- Create a new key:
      Run puttygen.exe -> Generate -> move mouse -> Save private key
      -> save as C:\opt\ssh\id_rsa.ppk
      Copy the public key text from the text field

    Option B -- Convert an existing OpenSSH key:
      puttygen.exe -> Conversions -> Import Key -> id_rsa (OpenSSH format)
      -> Save private key -> C:\opt\ssh\id_rsa.ppk

  5.3  Deploy the public key to target hosts

    On each member server (root@HOST):
      mkdir -p ~/.ssh
      echo "ssh-rsa AAAA... your-key" >> ~/.ssh/authorized_keys
      chmod 600 ~/.ssh/authorized_keys

    Same on the NAS host (192.168.2.203).

  5.4  Test the connection

    In PowerShell / cmd:
      C:\opt\ssh\plink.exe -ssh -batch -i C:\opt\ssh\id_rsa.ppk root@192.168.2.203 hostname

    Expected: NAS hostname without any password prompt.
    On the very first connection: confirm the host key fingerprint once
    (stored in registry, -batch works afterwards without any prompts).

  5.5  Pre-accept host keys (required for -batch mode)

    Connect once WITHOUT -batch to store the host key:
      C:\opt\ssh\plink.exe -ssh -i C:\opt\ssh\id_rsa.ppk root@192.168.2.203 hostname
    -> Answer "Store key in cache? (y/n)" with y.
    All subsequent -batch calls work silently after that.


================================================================================
  6. ROBOCOPY SYNC + ZFS SNAPSHOT BACKUP
================================================================================

  6.1  Set up the SMB share on the NAS

    On the OmniOS/Illumos NAS:
      zfs create tank/daten1/opt           (or your pool name)
      zfs set sharesmb=name=opt tank/daten1/opt
      svcadm restart smb/server

    From Windows: \\192.168.2.203\opt should be accessible.
    Optionally map a drive letter:
      net use Z: \\192.168.2.203\opt /persistent:yes

  6.2  sync_version.bat

    File: C:\opt\sync_version.bat
    See the existing script (already present at C:\opt).

    The script:
      1. Creates ZFS snapshot daten1/opt@TIMESTAMP via plink.exe
      2. Runs robocopy C:\opt -> \\192.168.2.203\opt /MIR
      3. Excludes tmp\ and ssh\ from the sync
      4. Writes a log to C:\opt\sync_version.log

  6.3  Set up autosnap on the NAS (safety net)

    On the NAS -- ZFS autosnap every 15 minutes via cron:
      crontab -e
      */15 * * * * zfs snapshot daten1/opt@auto-$(date +\%Y\%m\%d_\%H\%M)

    Or configure via napp-it cs Jobs > Snap.
    Recommended retention: 8 days (approx. 768 snapshots).

  6.4  Snapshot recovery

    List available snapshots:
      C:\opt\ssh\plink.exe -ssh -batch -i C:\opt\ssh\id_rsa.ppk root@192.168.2.203 ^
        "ls /daten1/opt/.zfs/snapshot/ | tail -20"

    Restore a single file:
      C:\opt\ssh\pscp.exe -i C:\opt\ssh\id_rsa.ppk ^
        "root@192.168.2.203:/daten1/opt/.zfs/snapshot/TIMESTAMP/csweb-gui/PATH/TO/FILE" ^
        C:\opt\csweb-gui\PATH\TO\FILE

    After recovery: run perl -c on the restored file.


================================================================================
  7. CLAUDE DESKTOP -- GENERAL INSTRUCTIONS (System Prompt)
================================================================================

  Claude Desktop -> Menu -> Settings -> General Instructions for Claude:

  Recommended text:
  -----------------------------------------------------------------------
  napp-it cs project is at C:\opt\csweb-gui.
  At session start read the guideline: csweb-gui/data/howto.ai/guideline.info
  and all *.info files in that folder and in _my/howto.ai/.
  Before larger changes: run sync_version.bat and wait for success.
  For recovery: ZFS snapshots accessible via pscp.exe
  (see _my/howto.ai/extra.info).
  -----------------------------------------------------------------------

  These instructions apply automatically to every new chat.


================================================================================
  8. TYPICAL WORKFLOW (daily use)
================================================================================

  1. Start Claude Desktop
  2. Open a new chat
  3. Claude confirms automatically:
       - Project C:\opt\csweb-gui read
       - Guideline and all *.info files read
       - sync_version.log checked (backup current?)
       - admin.pl / server.pl version strings shown
  4. Describe the task (paste a screenshot if helpful)
  5. For larger changes: Claude runs sync_version.bat first
  6. Claude edits files directly (no ZIP upload needed)
  7. Reload the browser page -> changes visible immediately
  8. For server.pl changes: restart the backend via napp-it cs


================================================================================
  9. REMOTE ACCESS TO MEMBER SERVERS (logs, diagnostics)
================================================================================

  Read logs:
    plink.exe -ssh -batch -i C:\opt\ssh\id_rsa.ppk root@192.168.2.185 ^
      "tail -50 /opt/csweb-gui/tmp/error.log"

  ZFS status:
    plink.exe -ssh -batch -i C:\opt\ssh\id_rsa.ppk root@192.168.2.185 ^
      "zpool status"

  Fetch file from member:
    pscp.exe -i C:\opt\ssh\id_rsa.ppk ^
      root@192.168.2.185:/opt/csweb-gui/tmp/error.log ^
      C:\tmp\error_185.log

  Push file to member:
    pscp.exe -i C:\opt\ssh\id_rsa.ppk ^
      C:\opt\csweb-gui\data\cs_server\server.pl ^
      root@192.168.2.185:/opt/csweb-gui/data/cs_server/server.pl


================================================================================
  10. TROUBLESHOOTING
================================================================================

  Claude cannot see files in C:\opt
    -> Filesystem extension not installed or wrong path in config
    -> Check claude_desktop_config.json, restart Claude Desktop

  plink.exe asks for password
    -> Host key not yet accepted: connect once without -batch
    -> Wrong PPK key: puttygen.exe -> Load -> id_rsa.ppk -> Verify

  robocopy fails (access denied)
    -> Check SMB share: net use \\192.168.2.203\opt
    -> Store Windows credentials for the NAS:
       Control Panel -> Credential Manager -> Windows Credentials -> Add

  Claude edits the wrong files
    -> Always check: Claude reads _my/howto.ai/ last (overrides standard)
    -> Ask in chat: "Which file did you just edit?"

  modifier.js only 13 bytes ("# placeholder")
    -> Known issue: restore from last snapshot
    -> Use the pscp.exe command from extra.info

================================================================================
 
New in napp-it cs Jul.09: Job type: Filesync with rclone
https://www.napp-it.org/index_en.html

Sync: "Server 1 ZFS Filesystem 1" -> "Server 2 Filesystem 2"
Before everyone says this has existed for ages with rsync, robocopy, and zfs send — here's the feature list:
  • Direct sync any-to-any, including e.g. Windows <-> FreeBSD <-> Solaris (daisy-chain capable)
  • Encrypted sync via z-stream tunnel with one-time key (https://github.com/guenther-alka/z-stream/releases)
  • Restartable (file-based), unlike incremental replication
  • Creates ZFS snapshots (source and destination)
  • ACLs are preserved (within a single OS)
  • Practical up to about 1TB — beyond that, zfs send is unbeatably faster

Since the last version
  • Single folder (/opt/csweb-gui), copy and run, no dependencies (https://www.napp-it.org/downloads_en.html)
  • Frontend web GUI for managing the local server (free) or a group via an encrypted backend connection
  • Frontend and backend on any OS (FreeBSD, Illumos, Linux, OSX, Solaris, and Windows)
  • 2FA with mobile authenticator
  • S3, Storage Spaces, and ZFS with ACL, user, and share management
  • Encrypted replication of unencrypted filesystems via z-stream
  • Easily extensible with custom menus (optimized for Claude AI-assisted work, https://www.napp-it.org/pdf/csweb-gui_use_ai.pdf)
 
napp-it cs and AI-assisted work, a few tips
The latest Anthropic 5 AI models with local agents are a genuine generational leap in terms of code quality and result effectiveness. napp-it is an ideal candidate for anyone who wants to (or needs to) try this out — best done on a Windows frontend with the Claude Desktop app + Claude Pro + MCP Filesystem and Desktop Commander + plink and pscp — because:

Claude can directly edit menus and scripts and read/optimize logs, using tools like plink and pscp on remote hosts too
Custom napp-it cs menu extensions can be used for all kinds of multiuder/multihost things, not just storage
Changes take effect immediately after a page reload (no extra compile or deployment delays)

This makes it possible, when facing a problem, to say: "Create a menu that...", "optimize and test the script until it works...", "improve the logging for troubleshooting..." — then go get a coffee. Claude will keep working on it until it runs and document everything cleanly. You can then test the final functionality instead of the 50 intermediate steps. Claude Pro costs about 17 USD/month. Depending on the model and working style, that can get you quite far — or not: https://the-decoder.de/claude-fable...-strategien-um-die-kosten-deutlich-zu-senken/ (use chrome to translate on the fly)

Anyone developing with AI for the first time has to change their sense of role. Until now, a software developer was both "architect" and "craftsman." As the craftsman, they always knew the complete project — its general goals, requirements, and interdependencies. With AI, I picture the process like this: I'm the architect, creating and planning the building concept, commissioning construction phases and supervising them. The AI is something like a "craftsman subcontractor" that I can assume works extremely precisely and competently. Unfortunately, a different company shows up for every construction phase, sometimes from abroad, so that communication beyond the single, exact work order is difficult. On top of that, they're all craftsmen with narrow, singular talents — perfect bricklayers and painters, but without an eye for the big picture, and extremely eager to keep working. A "make a hole in the wall" quickly turns into "I repainted the wall while I was at it since the scaffolding was already up."
A particular problem is chat sessions with limited knowledge transfer to a new session. If Claude has direct access to the local disk, the project data is at least always current. However, knowledge of the overall concept and project planning, as well as the most recent work steps or individual pieces of functionality, gets lost. If you don't want to re-explain all the principles and functionality, the type of client-server communication, and a thousand other details in every single chat session, it's essential to create a guideline, document general behaviours, and create info files on important individual pieces of functionality. On top of that, the most recently started and completed work steps need to be documented, along with a to-do list of open items.
All of that is already set up. A "read guideline" at the start of a Claude chat session takes care of all of this automatically, because all of these points are stored in /howto.ai/guideline.info.
 
What's going on?

napp-it cs consists of two Frontend services

  • webserver.pl (for common and private menus)
  • auto.pl (job management)

and two Backend Services

  • server.pl (encryped remote control)
  • monitor.pl (cache management, update/restart handling)
On Windows you can start them minimized as an option.
You can then check actions and data flow between frontend and backend

1783780493597.png
 
AI performance depends on the efficiency of the AI (Anthropic Claude is currently leading) and on how well it is trained for a given environment (my napp-it cs comes with extensive training data).

Helpdesk process (1h)

  • unclear error message
  • initial analysis by Claude AI, with follow-up question for logs
  • code analysis
  • code improvement

That was for an error message reported in a forum. With a local Claude installation including local and remote access (file and shell to cluster member servers), analysis + fix + test would have taken 5 minutes.

(use Chrome to translate on the fly)
https://www.hardwareluxx.de/community/threads/napp-it-cs-web-gui-für-fast-jeden-zfs-server-oder-servergruppen.1349039/page-11#post-31228989
 
Not everyone has a cluster with a dozen servers. Regardless of the OS, whether it’s a 1 GB mini IoT device or a petabyte system, whether using Storage Spaces, S3 object storage, or ZFS—not everyone wants to manage everything centrally with simple, fast, encrypted file sync or any-to-any ZFS replication (e.g., via https://github.com/guenther-alka/cs-stream).

However, that is precisely our target audience with napp-it cs. S3 integration, in particular, is a hot topic right now. It allows you to publish local files to the internet easily and securely (e.g., setting up a private cloud with RustFS). RustFS can also run independently in a "set-and-forget" mode to keep two servers or buckets in bidirectional sync. High availability and real-time backups over the internet or LAN, plain and simple.

What’s currently missing is the bridge between "shared multi-user SMB with ACLs" and object storage, which inherently lacks multi-user locking with file ACLs. This is where I'm stepping in with the cs-sync module.

With cs-sync, my goal is to keep two folders—such as an SMB share on a NAS and an S3 share—in sync in real time using event triggers (reacting instantly on change, without needing full file comparisons), including SMB ACLs. Regardless of the OS, this is now possible thanks to Go, with the option for real-time automated backups of changes over the network to a backup system (see https://github.com/guenther-alka/cs-sync).

I would love to get your feedback or suggestions on this concept! A first preview featuring napp-it cs and real-time sync is coming soon. Anyone interested in serious testing should consider getting Claude Pro along with the Filesystem and Desktop Commander extensions/agents (on Windows or other OSs via Plink/SCP SSH access for Claude to inspect scripts, logs, and members) for analysis, stress testing, audits, etc.
 
napp-it cs 26.06 07.29 rc
Update: Menu About > Frontend Update

-missing private or osx menus
-osx setup: https://www.napp-it.org/pdf/apple_osx_en.pdf

Go based multi os cs-stream job
https://github.com/guenther-alka/cs-stream/releases-ZFS replication or filebased sync (via rclone)
-encrypted, buffered or io limited tunnel any to any

Go based multi os cs-sync background service
https://github.com/guenther-alka/cs-sync/releases-event driven local uni/bi service with folder ACL support for realtime sync (eg SMB and S3 shares)
-event driven remote uni sync service with folder ACL support for realtime backup (encrypted)

Status ample (current and last 10m/h/day/week)
Pool, Cap, Disk, IO, RAM, Jobs, Sync, Last

Main menu "Servergroup" shows running sync services on members
 
The new event driven (on file modifications) realtime cs-sync in napp-it cs is not only a perfect method to keep local S3 object storage and local SMB shares bidirectionally in sync but a new approach for realtime sync between data and backup, best paired with ZFS snaps on destination to have backup versions eg snaps hourly-keep 24, daily-keep 24, ..

remains an important aspect:

Behavior on slow or unstable connections

Handling disconnects and slow links is a core design goal, not an afterthought -- local-only 1.x never had to deal with this at all. See cs-sync-2.0-design.info section 3 for the full rationale.


  • Disconnects (flaky WLAN, link flaps) are normal operation, not errors. No timeout, no alarm, just a log line. Every change that can't reach the remote right now goes into a persisted pending queue that survives both process restart and host reboot.
  • Coalesced per path. A file that changes 100 times while offline is transferred once, latest state only -- the queue stores "this path needs sync," not an event log.
  • Reconnect uses exponential backoff (1s -> 2s -> ... capped at 5min) before a retry counts against a file's failure count, so a flapping link can't burn through the retry budget in seconds.
  • Backpressure, not blocking. The filesystem watcher never blocks on the network -- it can produce changes faster than a slow link can carry them indefinitely; the sender just drains the queue at whatever speed the link allows.
  • Queue depth is unbounded by design. A deep queue just means "slow link + lots of data," which has to keep working. The retry limit (10 attempts, see backoff above) applies only to individual files that keep failing for their own reasons (permission errors, unreadable source, etc.) -- such a file is quarantined and logged so it stops blocking the rest of the queue, everything else keeps syncing.
  • Atomic writes + end-to-end hash. Every transfer goes to a temp file, is hash-verified, then renamed into place -- a drop mid-transfer never leaves a half-written file that size/mtime could later mistake for current.
  • Torn-copy detection. If the source changes while a slow transfer is still reading it, a before/after size+mtime mismatch discards the copy and re-queues it -- no locking needed.
  • --bwlimit throttles a remote leg so an initial full sync over a narrow/shared link doesn't saturate it for everything else using the same connection.

Not yet implemented: delta transfer for large partially-changed files (a changed file is always sent in full -- see Known gaps above) and a queue disk-space warning threshold for very long outages. (an option for a next release for very large files)
 

S3 object storage (eg RustFS) + ZFS​


RustFS/S3 object storage and ZFS have different use cases
If combined eg with a bidir sync between smb and S3 shares with SMB folder ACL inheritance respected,
the result is much more than a simple addition of features

RustFS modes

Single Node, single disk
Single Node, multiple disk (RustFS software raid)
Multi node (HA Cluster), each with single or multiple disks

RustFS on ZFS — my summary

RustFS single disk mode on top of a ZFS pool is superior to RustFS software raid/-native multi-disk erasure coding in every dimension — checksums (block-level + self-heal vs. object-level only), snapshots (COW, near-free vs. per-object versioning at real storage cost), caching (ARC/L2ARC/SLOG hybrid tiering vs. plain page cache), and no redundant Reed-Solomon CPU overhead for redundancy ZFS already provides. "Single disk" here means a single RustFS target — the ZFS pool underneath can itself be a mirror/RAIDZ/Draid and Anyraid in near futurw across multiple physical disks, so no physical redundancy is actually given up.


This holds even for large multi-node deployments. Distributed erasure coding across nodes is still the right tool for whole-node-failure tolerance without replication lag, single-namespace capacity beyond one node, and aggregate multi-node throughput — but each node's own local storage should still be a ZFS-redundant pool underneath, not RustFS-managed raw disks. Local disk faults get absorbed by ZFS before they ever reach the distributed EC layer at all, with no downside.

Ideal: RustFS single node, single target-on-ZFS, plus RustFS bucket/site replication for redundancy across nodes/locations — for small/medium setups this is the whole answer; for large setups it's the same pattern per node, with distributed EC layered on top only when node-count/throughput/namespace needs actually call for it.

Open wish, not yet real: exposing ZFS snapshots of RustFS's data directory as read-only S3 object versions natively in RustFS. Not implemented today — would need RustFS to actively integrate with .zfs/snapshot/ and reconcile that against its own object metadata DB, a real feature request to the project, not a config option. Until then, the two mechanisms stay complementary but separate: zfs snapshot for whole-store point-in-time recovery, RustFS's own object versioning for granular per-object history.

The one remaining exception: hosts without ZFS. If OpenZFS on Windows/macOS support matures to production-ready, that exception may shrink or disappear — worth confirming current status before relying on it for winpool or macOS hosts specifically. OpenZFS is released on OSX now with a nearly ready rc on Windows.

This is how i integrate it in my napp-it cs single server or Cluster management tool for S3, Storage Spaces and ZFS

Gea
 
OpenZFS on Windows 2.4.1rc14 (and 2.4.1 released on OSX)

OpenZFS on Windows 2.4.1rc14
https://openzfsonwindows.org/

** rc14
  • Pageout write stalls in rangelock
  • panic exceptions were masked
  • Unmount fixes, cause finally known of poor unmount
  • Change condvar to be posix compliant in mutex_exit -> cv_broadcast cases
  • Setting mountpoint would compute wrong path Issues: https://github.com/openzfsonwindows/openzfs/issues/619
btw
OpenZFS on OSX 2.4.1 has released state, not a release candidate
https://openzfsonosx.org/
 
Hi _Gea
Thanks for all your tireless work for napp-it and the community. I use your product since many years, and I am still thrilled by how much easier it makes my life, given that I am in no way experienced in the Linux world, and really just enjoy tinkering with a home server ;-)
With VMware still not allowing vTPM for clients if you do not have a vcenter in use (thus paying a really significant amount of money for the license), I had enough and I am now planning to reinstall my entire server with Proxmox. I intend to keep the "All-in-One" approach I very much appreciated in the last few years, as I have only one server.
I am planning the setup with Gemini, and it basically tells me that I can keep the same way of doing like under ESXi, more or less. Of course, when importing the NAS-VM (OmniOS running napp-it), I have to directly import the disk to Proxmox. Luckily enough, I had it in its own vmdk. I will then delete the old pools and create them from scratch. AI tells me to have one big pool built of 6 vdevs (mirror) (I keep 2 hot spares) under Proxmox, so OmniOS does not need to care about handling disk/data security and I can create single disks there. Is this assessment correct? It would need quite a lot less RAM, if I am not mistaken. I got my hands on 14 x 3.4TB Netapp SSD's which I hope will perform nicely and provide enough space, still. Any hint or cautionary note from your side for this setup would be much appreciated ;-)
Thanks,
Cap
 
It fully depends whether you need/want the OmniOS kernelbased SMB server as it fully supports ntfs alike ACL with SMB groups in groups and Windows SID as extended ZFS attribute (ACL reference). If you need ACL, OmniOS is so much easier than SAMBA/Linux. In such a case AiO on Proxmox with an OmniOS VM is ok.

If some basic Posix ACL folder permissions are good enough, you should use Proxmox not only as Hypervisor but as barebone NAS too,
just install SAMBA or the faster ksmbd and acl. Then use the new copy and run napp-it cs (no dependencies, no modifications on Proxmox),
https://www.napp-it.org/downloads.html

setup at console:
Code: 
curl -ksL https://www.napp-it.org/cs_setup.sh | sh
 
Coming from ESXi, there are two things I'm used to / that matter to me:

  1. When I take a ZFS snap of a VM storage, it shouldn't just contain the virtual disks but also the VM configuration, so I can restore a VM from a ZFS snap (creating a snap, unlike PBS, is instant with nothing to copy, and even zfs send over LAN is much faster). On ESXi that was a .vmx file in the VM folder. On Proxmox, VM configurations live in /etc/pve.

Solution: I've added a snap option "include regular folder" to the snap job. This lets me capture the state of /etc/pve at snap time along with the snap.

Another use case would be Windows, to have c:/Users/"me" with personal documents on NTFS included in the ZFS snap.

  1. Consistent VM snaps
    A ZFS snap is always consistent for the host managing the ZFS pool, thanks to copy-on-write and ZFS's internal snapshot handling. It looks completely different for VMs and their guest filesystems on ZFS. A ZFS snapshot on the host doesn't care whether their filesystems stay consistent. They end up in the snap in the same state as after an unexpected power outage. The risk of corrupted data can be reduced by making sure no write operations happen during the snap (VM freeze). Proxmox has various methods for that. I've now integrated zfs snapshot into a Go-based tool that automatically provides various freeze options to reduce the likelihood of a corrupted VM in the snap. The only way to avoid it entirely would be to snap while offline, or to also save RAM state — but with current RAM sizes that's very slow.

Go tool (included in current napp-it CS in autosnap and replication jobs)
https://github.com/guenther-alka/cs-freeze4snap
 
You must log in or register to reply here.
Back
Top