Large File Migration with rclone Bridge

Use case

• Category: Large file data migration
• Target: STACKIT File Service
• Method: rclone transfer path when direct dual NFS mount is unavailable

Network and architecture constraints (validated)

• rclone does not remove SFS private access rules: Even with rclone, the target SFS share remains NFS-based and network-restricted by SNA routing and ACL/export policies. See File Storage concepts and Mounting resource pools and shares .
• Bridge host must be dual-reachable: The transfer host must reach source and target under the actual network model, not only theoretically.
• Provider reality is similar elsewhere: Private file-share services from other providers (for example AWS EFS) are also not internet-exposed by default, so private connectivity is required for migration paths. See AWS EFS network access .

Mandatory prerequisites

Validate the bridge host’s effective mount permissions before starting a copy. An allowed network path alone does not establish read-write access to the target share.

From the STACKIT docsMounting Resource Pools and Shares › Mounting a ShareSource updated 04.02.2026 · copied 06.10.2026

When a Share is created, you can optionally pass it a Share Export Policy, to control which IPs can mount the Share, and with which permissions. If you don’t attach any Share Export Policy to the Share, mounting the Share inherits the rules of the Resource Pool. In other words, the IP ACL of the Resource Pool is applied and the client can only mount the Share in read-only mode.

A Share has a field called Mount Path, that looks like this: 10.2.1.1:/rp\_VKL20Ub/my-share. It is mountable the same way as the Resource Pool.

Keep in mind that:

• In order to have read-write access on a Share, you need to create a Share Export Policy beforehand and attach it to the Share.
• A Share does not have a fixed size. By default, every Share in a Resource Pool have access to all the space of the Resource Pool. You can limit the space a Share consumes.
• If you apply a Share Export Policy to the Share, you can define a subset of the network in the Resource Pool IP ACL. If you define a network that is bigger than the Resource Pool IP ACL, then the Resource Pool IP ACL will take precedence.
• To ensure proper ownership, you have to adjust the NFSv4 ID domain to stackit.cloud beforehand. This can be done in /etc/idmapd.conf followed by the bash command nfsidmap -c.

What is this?

This section is copied from the STACKIT docs automatically, several times a day. It cannot be changed here. Changes belong in the STACKIT docs.

• Target reachability from bridge host: The bridge host can mount the target SFS share over private connectivity in the corresponding SNA scope.
• Source reachability from bridge host: The bridge host can reach the source endpoint (internet, private network, or VPN-attached).
• Endpoint access validated: Migration run environment can access both source and target endpoints.
• Protocol mapping validated: rclone backend configuration is available for each side.
• Credential governance defined: Secret storage and rotation responsibilities are assigned.
• Consistency windows agreed: Initial copy, incremental sync, and final freeze windows are approved.

Not suitable when

• No secure bridge exists: A compliant run environment cannot reach both endpoints.
• Target SFS cannot be reached privately: No route from bridge host to SNA-attached SFS is available.
• Uncontrolled source mutations: Write-heavy source cannot provide a final consistency window.

VPN feasibility

• VPN is a valid bridge enabler: VPN can provide the missing private path between source-side networks and SNA resources.
• VPN scope is site-to-site: STACKIT VPN is site-to-site and SNA-based, so the bridge design must use network-to-network connectivity, not point-to-site assumptions. See STACKIT VPN product overview .
• Routing and policy checks are mandatory: Confirm route exchange and security rules for data-plane traffic before production sync.

When to choose this variant

• Choose rclone bridge when: Dual NFS mounting on one host is not feasible, but one controlled bridge host can reach both source and SFS target.
• Do not choose rclone bridge when: No single secure run environment can reach both endpoints with required bandwidth and stability.
• Alternative: Use the NFS and fpsync runbook when direct dual NFS mounting is possible.

Exception variant: Source-side transfer host with internet SFTP relay

Use this variant only when dual NFS mounting on one migration VM is not feasible.

If you can place one migration VM in STACKIT and mount both source and target by private connectivity (including site-to-site VPN to source), prefer the NFS and fpsync runbook.

• Flow: AWS EFS -> AWS EC2 (NFS mount) -> Internet (SSH/SFTP) -> STACKIT jump host -> SFS (NFS mount).
• Tooling: On AWS EC2, use rclone with an SFTP remote targeting the STACKIT jump host, or alternatively rsync/scp over SSH.
• Validation conditions: Jump host is SFS-reachable, SFTP ingress is controlled, throughput is realistic, integrity checks are deterministic, and SFTP target path maps unambiguously to mounted SFS path.

Throughput and concurrency tuning

• Transfer parallelism: Increase concurrent file transfers with rclone --transfers <parallelism>.
• Metadata/check parallelism: Scale listing and comparison work with --checkers <parallelism> to avoid metadata bottlenecks.
• SFTP backend tuning: For SFTP-heavy paths, evaluate backend-specific options such as SFTP concurrency and chunk size based on server capability.
• Network path quality: Stabilize latency, packet loss, and MTU between source transfer host and STACKIT jump host.
• Encryption overhead: SSH/SFTP encryption can become CPU-bound on either side; validate CPU headroom on EC2 and jump host while scaling transfers.
• NFS target path tuning: Tune NFS mount behavior on the jump host (rsize, wsize, and nconnect where supported) because final writes go to SFS over NFS.
• File-size profile awareness: Large files usually benefit from transfer parallelism; high counts of small files often require stronger checker/listing tuning.
• Measurement discipline: Track effective MB/s, files/s, retries, transfer failures, and CPU load after each tuning change.

Implementation template

Phase 1: Build transfer bridge

1. Prepare migration host and secure credential injection.
2. Configure rclone remotes for source and target.
3. Run dry-run and baseline performance sampling.

Phase 2: Controlled copy and sync

1. Run initial copy with controlled parallelism.
2. Run incremental sync cycles and review error logs.
3. Track retries and classify unresolved transfer errors.

Phase 3: Finalization

1. Trigger final source freeze window.
2. Run final sync and integrity checks.
3. Approve handover to consuming workload and archive logs.

Validation checklist

• Transfer completeness: File count and size parity validated.
• Integrity sample: Hash comparison sample completed.
• Error handling evidence: Failed transfers reviewed and resolved or approved.
• Security evidence: Credential and access logs documented.