--- title: Large File Migration with NFS and fpsync description: "Concrete template for migrating large file volumes to a STACKIT File Storage share using a migration VM with dual NFS mounts and fpsync-based delta syncs." sidebar: badge: text: "STACKIT" variant: success scfAsset: managed: false category: "runbook" external: false tags: ["design-and-mobilize", "use-cases", "replatform", "file-data", "file-service", "fpsync", "nfs"] maintainers: - user: "lukas.weberruss" role: true website: true source_url: "https://framework.stackit.cloud/migration/assetcontainer/stackit/runbook-file-migration-nfs-fpsync/" source_file: "docs/migration/assetcontainer/stackit/runbook-file-migration-nfs-fpsync.mdx" --- ## Use case - **Category**: Large file data migration - **Target**: STACKIT File Service - **Method**: Migration VM with source and target NFS mounts, synchronized with fpsync ## Network and architecture constraints (validated) - **SFS is SNA-attached**: STACKIT File Storage is connected to a STACKIT Network Area (SNA), uses private interconnection subnets, and requires SNA routing tables. See File Storage concepts and SNA routing tables. - **Mount access is network-restricted**: NFS mounts are controlled by Resource Pool IP ACLs and Share Export Policies, so the migration host IP must be explicitly allowed. See Mounting resource pools and shares. - **SNA is regional**: Private SNA traffic is regional; cross-region traffic is possible through public internet. See Network Area concepts. - **Provider comparison is consistent**: Comparable NFS services such as AWS EFS mount targets are also private by design (no public IP on mount targets), so a private path is required there as well. See AWS EFS network access. > From the STACKIT docs: [Concepts and terminology › Interconnection](https://docs.stackit.cloud/products/storage/file-storage/basics/concepts/#interconnection) (Source updated 22.04.2026, copied 05.10.2026) A STACKIT File Storage is always connected to an STACKIT Network Area (SNA). Therefore, a dedicated subnet of the SNA-Network is reserved and used to interconnect the whole SNA with the STACKIT File Storage (routed per default). It can be restricted by policies, if required. STACKIT File Storage requires routing tables to be enabled in the STACKIT Network Area. More information on how to do so can be found in the [routing tables docs](https://docs.stackit.cloud/products/network/core-networking/network-area/basics/routing-tables/). When configuring the SNA, please ensure the following requirements are met: - There are sufficient IP networks available within the SNA. - The minimum size for any new subnet must be set to at least /29. - For the SFS interconnection, at least one subnet with /28 and four subnets with /29 are required. ## Mandatory prerequisites - **Migration host placement**: The migration VM runs in an SNA project and region that can mount the target SFS share. - **Dual mount feasibility**: Source and target exports can both be mounted on one migration VM. - **Private L3 path to source NFS**: The migration VM can reach the source NFS endpoint over private routing (same private domain, peering, or site-to-site VPN). - **NFS policy alignment**: ACL/security policy allows required client IP ranges and NFS traffic in both directions. - **Network feasibility**: Latency and throughput are sufficient for parallel sync. - **Permission model aligned**: UID/GID and ACL translation rules are defined. - **Consistency model defined**: Initial sync, delta sync windows, and final freeze are agreed. Check the effective share permissions in addition to private reachability. The migration host needs read-write access to the target, and ownership mapping must be validated before the first fpsync run. > From the STACKIT docs: [Mounting Resource Pools and Shares › Mounting a Share](https://docs.stackit.cloud/products/storage/file-storage/basics/mounting-resource-pools-and-shares/#mounting-a-share) (Source updated 04.02.2026, copied 06.10.2026) When a Share is created, you can optionally pass it a Share Export Policy, to control which IPs can mount the Share, and with which permissions. **If you don’t attach any Share Export Policy to the Share, mounting the Share inherits the rules of the Resource Pool**. In other words, the IP ACL of the Resource Pool is applied and the client can only mount the Share in read-only mode. A Share has a field called Mount Path, that looks like this: `10.2.1.1:/rp\_VKL20Ub/my-share`. It is mountable the same way as the Resource Pool. Keep in mind that: - In order to have read-write access on a Share, you **need to create a Share Export Policy** beforehand and attach it to the Share. - A Share does not have a fixed size. By default, every Share in a Resource Pool have access to all the space of the Resource Pool. You can limit the space a Share consumes. - If you apply a Share Export Policy to the Share, you can define a subset of the network in the Resource Pool IP ACL. If you define a network that is bigger than the Resource Pool IP ACL, then the Resource Pool IP ACL will take precedence. - To ensure proper ownership, you have to adjust the NFSv4 ID domain to `stackit.cloud` beforehand. This can be done in `/etc/idmapd.conf` followed by the bash command `nfsidmap -c`. ## Not suitable when - **No private path exists**: Source NFS is not reachable from the SNA-connected migration host through controlled private connectivity. - **Dual NFS mount is blocked**: Policy or network constraints prevent simultaneous mounts. - **Protocol mismatch exists**: Source endpoint does not provide compatible NFS access. ## VPN feasibility - **VPN is a valid option**: This runbook works with VPN when the VPN connects the source network to the target-side SNA and routes are advertised correctly. - **VPN scope is site-to-site**: STACKIT VPN is designed for site-to-site connections and SNA-based projects. See STACKIT VPN product overview. - **Operational readiness required**: Validate tunnel stability, MTU behavior, and sustained throughput before cutover. ## When to choose this variant - **Choose NFS+fpsync when**: One migration host can mount both source NFS and target SFS directly and you need fast iterative delta cycles. - **Do not choose NFS+fpsync when**: Source access is not NFS-compatible or no controlled private connectivity to source can be established. - **Alternative**: Use the [rclone bridge runbook](/migration/assetcontainer/stackit/runbook-file-migration-rclone-bridge/) when dual NFS mounting is not feasible. ## Migration VM placement and trade-offs - **Preferred placement (STACKIT side)**: Run the migration VM in STACKIT, attached to the SNA. This keeps the SFS write path private and makes target-side routing and ACL control more direct. - **Alternative placement (source side)**: Run a transfer host near the source only when required by source constraints. This is usually harder for dual-mount operation to SFS and often shifts complexity to relay patterns. - **VPN implication**: For source reachability, site-to-site VPN is the preferred helper. In this topology, source traffic to the migration VM runs through the VPN tunnel. ## Recommended topology: STACKIT migration VM with site-to-site VPN to source This topology is the primary pattern for VPN-supported dual mounts in this runbook. ```d2 style.font-size: 22 direction: right Source: "Source environment" { SourceNFS: "Source NFS export" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/networking/network.svg } } VPN: "Site-to-site VPN tunnel" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/networking/vpn.svg link: https://docs.stackit.cloud/products/network/connectivity-hybrid-multi-cloud/vpn/ } STACKIT: "STACKIT" { grid-columns: 2 MigVM: "Migration VM (fpsync)" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/computing/virtual-machine.svg link: https://docs.stackit.cloud/products/compute-engine/server/ } SNA: "SNA" { grid-columns: 2 RT: "Routing tables" SFS: "STACKIT File Storage share" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/computing/file-storage.svg link: https://docs.stackit.cloud/products/storage/file-storage/ } } } Source.SourceNFS -> VPN: "NFS" VPN -> STACKIT.MigVM: "NFS in tunnel" STACKIT.MigVM -> STACKIT.SNA.RT: "NFS over SNA" STACKIT.SNA.RT -> STACKIT.SNA.SFS: "routed path" # Invisible edges to stabilize center alignment of outer blocks. Source -> VPN: { style.opacity: 0 } VPN -> STACKIT: { style.opacity: 0 } ``` ## Alternative topology with source-side fpsync client and HAProxy TCP proxy Use this variant when you want to keep the `fpsync` client on the source side and avoid mounting NFS on the STACKIT jump host itself. - **Pattern**: HAProxy on the jump host works as a TCP pass-through endpoint for NFS traffic (`tcp/2049`) towards SFS. - **Ingress path**: The source-side client reaches HAProxy through the **public IP of the jump host**. - **Security controls on jump host**: Security group / ACL on the jump host allows inbound `tcp/2049` only from approved source client public IP ranges. - **No target mount on jump host**: The jump host forwards NFS sessions and does not need to mount the SFS share locally. - **Shared client model**: The source-side migration client can mount source NFS directly and target NFS through the HAProxy endpoint, then run `fpsync` between both mount points. - **Policy prerequisite**: SFS ACL/export policy must allow the effective client path and source IP model of this setup. ```d2 style.font-size: 22 direction: right Source: "Source environment" { grid-columns: 2 Client: "Source migration client (fpsync)" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/computing/virtual-machine.svg } SourceNFS: "Source NFS export" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/networking/network.svg } } Internet: "Public internet path" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/networking/ip.svg } STACKIT: "STACKIT" { grid-columns: 2 Jump: "Jump host with HAProxy (TCP, Public IP)" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/computing/virtual-machine.svg link: https://docs.stackit.cloud/products/compute-engine/server/ } SG: "Security group / ACL\nAllow inbound tcp/2049\nfrom approved source public IPs" SNA: "SNA" { grid-columns: 2 RT: "Routing tables" SFS: "STACKIT File Storage share" { icon: ../../../../../../../../libs/ui/figma-assets/architecture-symbols/src/lib/assets/computing/file-storage.svg link: https://docs.stackit.cloud/products/storage/file-storage/ } } } Source.Client -> Source.SourceNFS: "NFS mount" Source.Client -> Internet: "NFS to jump host public IP" Internet -> STACKIT.SG: "TCP 2049" STACKIT.SG -> STACKIT.Jump: "allowed ingress" STACKIT.Jump -> STACKIT.SNA.RT: "forwarded NFS" STACKIT.SNA.RT -> STACKIT.SNA.SFS: "routed path" # Invisible edges to stabilize center alignment of outer blocks. Source -> Internet: { style.opacity: 0 } Internet -> STACKIT: { style.opacity: 0 } ``` ### Why this can be useful - **Logging**: HAProxy TCP logs provide one central trace point for NFS session attempts, connection errors, and backend availability. - **Timeout control**: HAProxy timeout settings (`timeout connect`, `timeout client`, `timeout server`) give explicit control over stuck or long-running connections. - **Operational guardrails**: You can apply controlled connection handling and clear failure behavior at a single ingress point. ### Trade-offs and risks - **Extra hop**: Adds one network hop and one additional component in the data path. - **Proxy bottleneck risk**: Jump host sizing and HAProxy tuning become throughput-critical. - **Service semantics validation**: NFS over TCP proxying must be validated end-to-end in your target policy and support model. - **High availability needed**: Without HA design, the proxy host can become a single point of failure. ## Variant comparison: VPN dual-mount VM vs HAProxy TCP proxy | Criterion | Variant A: STACKIT migration VM with dual mounts (VPN to source) | Variant B: Source client + HAProxy TCP proxy | | -------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------- | | **Security surface** | Smaller runtime chain; fewer middle components in data path. | Extra proxy layer to harden and operate; centralized ingress control possible. | | **Performance** | Usually higher throughput potential (direct dual mount, fewer hops). | Additional hop and proxy processing reduce peak throughput in most environments. | | **Stability** | Fewer moving parts; depends on VPN and migration VM stability. | Additional failure domain (HAProxy host/service); needs HA design for robustness. | | **Monitoring and logging** | Relies mainly on host, NFS client, and network metrics. | Strong centralized TCP visibility and timeout observability at proxy layer. | | **Timeout handling** | Mostly OS/NFS client behavior on migration VM. | Explicit timeout control in HAProxy plus client-side timeout behavior. | | **Operational complexity** | Simpler baseline architecture. | More components to configure, tune, and troubleshoot. | ## Shared end-state for both variants For both architectures, the effective migration flow can end at the same `fpsync` model: - Source-side or STACKIT-side client has two NFS mount points (source and target). - `fpsync` runs iterative sync cycles between these mount points. - Final freeze window and final pass remain identical in principle. ## Operational flow on the migration VM - **Step 1 (source mount)**: The migration VM mounts the source NFS export through the private path of the selected topology (for this runbook: site-to-site VPN tunnel to source). - **Step 2 (target mount)**: The same migration VM mounts the SFS target path by NFSv4.1 (TCP 2049) through SNA routing tables. - **Step 3 (sync cycles)**: `fpsync` runs iterative delta cycles between both mounted paths until cutover. - **Step 4 (final pass)**: After source freeze, run the final `fpsync` pass and complete integrity checks. ## Throughput and concurrency tuning - **fpsync worker parallelism**: Increase parallel workers with `fpsync -n ` and tune in controlled increments. - **Starting point and scaling**: Start with moderate parallelism, observe throughput and error rate, then increase until gains flatten or retries rise. - **NFS client tuning**: Validate mount options such as `rsize`, `wsize`, and `nconnect` (where supported) for both source and target mounts. - **Network path quality**: Keep MTU, packet loss, and latency stable across the source path (including VPN) and the SNA target path. - **VM sizing**: Ensure migration VM CPU, memory, and NIC bandwidth are sufficient for concurrent file traversal and transfer. - **Storage-side limits**: Check source export limits and SFS-side throughput behavior so worker scaling does not exceed service-side bottlenecks. - **Workload profile split**: Test large-file and small-file datasets in dedicated test sets; small files often require higher metadata parallelism, not only bandwidth. - **Measurement discipline**: Track effective MB/s, files/s, retransmits, retries, and server load per tuning step. ## Implementation template ### Phase 1: Prepare migration VM 1. Harden migration VM and configure logging. 2. Mount source and target NFS paths with verified options. 3. Run baseline read/write probes and record throughput. ### Phase 2: Initial and delta sync 1. Run initial fpsync pass. 2. Capture transfer statistics and error files. 3. Schedule delta sync cycles until cutover window. ### Phase 3: Final sync and handover 1. Activate final write freeze on source window. 2. Run final fpsync pass and verify integrity sample. 3. Hand over mounted target path to consuming workload. ## Validation checklist - **File integrity**: Sample checksum and count verification completed. - **Permission integrity**: ACL and ownership spot checks completed. - **Performance evidence**: Effective throughput and total duration documented. - **Operational handover**: Monitoring and ownership confirmed.