How to Speed Up BPI-R4 Development with NFS Root and Cross-Chrooting

How to Speed Up BPI-R4 Development with NFS Root and Cross-Chrooting

By: Director - AKADATA LIMITED

This article is very much a work in progress and will continue evolving over the coming days as we explore and document the BPI-R4 — with the ultimate goal of unleashing Wi-Fi 7 capabilities and achieving up to 34Gbps bandwidth.

If you’re curious about owning one of these boards — and possibly getting it in your hands within 48 hours (sometimes even the next day depending on when you order) — scroll to the end of this article for details.

Numbers are limited. While no further stock is currently guaranteed, we may bring in additional boards if there’s enough interest.

This guide shows how to dramatically reduce development time on the Banana Pi BPI-R4 (or similar ARM64 SBCs) by leveraging:

✅ An x86_64 build server (e.g. a workstation or server with plenty of RAM and fast NVMe)

✅ binfmt_misc and QEMU user emulation for transparent chroot into ARM64

✅ NFS-mounted root filesystems for fast read/write

✅ chroot environment usable from both the SBC and the x86_64 host

Here’s how to set it up step by step.


1. Prepare Your Build Server

Install QEMU user-static and binfmt_misc

On Arch Linux:

pacman -S qemu-user-static-binfmt qemu-system-aarch64

On Debian/Ubuntu:

apt install qemu-user-static binfmt-support

This ensures your host can transparently run ARM64 binaries via qemu-aarch64-static.

Verify registration:

ls /proc/sys/fs/binfmt_misc

Look for qemu-aarch64.

If not registered automatically:

echo ':qemu-aarch64:M::\x7fELF\x02\x01\x01:\xff\xff\xff\xff\xff\xff\xff\xff:/usr/bin/qemu-aarch64-static:CF' > /proc/sys/fs/binfmt_misc/register

2. Create ZFS Datasets or NFS Shares

On the build server (example with ZFS):

zfs create nvmepool/bpi/rootfs

For multiple distros:

for i in debian-bookworm debian-bullseye alpine archlinuxarm openwrt gentoo; do \
  zfs create nvmepool/bpi/rootfs/$i; \
done

Export over NFS:

Edit /etc/exports:

/mnt/nvmepool/bpi/rootfs 172.16.0.0/24(rw,no_root_squash,no_subtree_check,async)

Apply:

exportfs -arv

3. Bootstrap Debian/Ubuntu Rootfs

Example for Debian Bookworm:

debootstrap --arch=arm64 bookworm /mnt/nvmepool/bpi/rootfs/debian-bookworm \
  http://deb.debian.org/debian

4. Mount NFS Root on BPI-R4

On the BPI-R4:

mkdir -p /mnt/build/rootfs/debian-bookworm
mount -t nfs -o vers=3 172.16.0.1:/mnt/nvmepool/bpi/rootfs/debian-bookworm /mnt/build/rootfs/debian-bookworm

Check access:

ls /mnt/build/rootfs/debian-bookworm

5. Fix Locales Inside Chroot

On either host or BPI-R4:

chroot /mnt/build/rootfs/debian-bookworm /bin/bash

Then:

apt update
apt install locales -y
sed -i 's/# en_GB.UTF-8 UTF-8/en_GB.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
update-locale LANG=en_GB.UTF-8
export LANG=en_GB.UTF-8
export LC_ALL=en_GB.UTF-8

Check:

locale

6. Use the Chroot on Both Sides

  • On the worker (x86_64):
arch-chroot /mnt/nvmepool/bpi/rootfs/debian-bookworm /bin/bash
  • On the BPI-R4:
chroot /mnt/build/rootfs/debian-bookworm /bin/bash

Same environment, two platforms!

7. Use this Chroot helper on your worker host

This script can be used to build out your shiny new sdcard, it will save time adding compiler etc before first mount, also you can add users and change passwords once inside the chroot

#!/usr/bin/env bash

# ------------------------------------------------------------
# bpichroot - BPI-R4 chroot helper script
#
# Author: Director, AKADATA LIMITED
#
# This script helps mount either:
#   - the SD card root filesystem (physical card plugged in)
#   - OR an NFS-mounted root filesystem from the network
#
# Why?
#   - To prepare chroot environments for building Banana Pi R4 images
#   - To enable distcc distributed compilation using NFS root
#
# Usage:
#   sudo bpichroot mount sd      # mount SD card root and chroot
#   sudo bpichroot mount nfs     # mount NFS root and chroot
#   sudo bpichroot unmount       # cleanly unmount all mounts
#
# ------------------------------------------------------------
#!/usr/bin/env bash
set -e

# Paths
BPI_SD_MOUNT="/mnt/bpi"
BPI_NFS_MOUNT="/mnt/bpi-4-nfsroot"

# Shared mount logic
do_mounts() {
    local CHROOT="$1"

    echo "Mounting essential filesystems into $CHROOT ..."

    # Ensure all required mount points exist and mount them safely
    for dir in proc sys sys/kernel/security dev dev/pts dev/shm; do
        full_path="$CHROOT/$dir"
        if [ ! -d "$full_path" ]; then
            echo "Creating missing directory: $full_path"
            mkdir -p "$full_path"
        fi
        # Only attempt to mount if not already mounted
        if ! mountpoint -q "$full_path"; then
            case "$dir" in
                proc)
                    mount -t proc proc "$full_path"
                    ;;
                sys)
                    mount -t sysfs sys "$full_path"
                    ;;
                sys/kernel/security)
                    mount -t securityfs securityfs "$full_path" || true
                    ;;
                dev)
                    mount -t tmpfs tmpfs "$full_path"
                    ;;
                dev/pts)
                    mount -t devpts devpts "$full_path"
                    ;;
                dev/shm)
                    mount -t tmpfs tmpfs "$full_path"
                    ;;
            esac
        else
            echo "$full_path already mounted."
        fi
    done
}

do_unmounts() {
    local CHROOT="$1"
    echo "Unmounting essential filesystems from $CHROOT ..."
    for dir in dev/shm dev/pts dev sys/kernel/security sys proc; do
        full_path="$CHROOT/$dir"
        if mountpoint -q "$full_path"; then
            umount -l "$full_path" || true
        fi
    done
}

mount_bpi_sd() {
    echo "Searching for BPI-ROOT and BPI-BOOT..."
    BPIROOT=$(blkid -L BPI-ROOT || true)
    BPIBOOT=$(blkid -L BPI-BOOT || true)
    if [ -z "$BPIROOT" ]; then
        echo "Error: BPI-ROOT not found!"
        exit 1
    fi
    if [ -z "$BPIBOOT" ]; then
        echo "Error: BPI-BOOT not found!"
        exit 1
    fi
    echo "Found BPI-ROOT: $BPIROOT"
    echo "Found BPI-BOOT: $BPIBOOT"
    mkdir -p "$BPI_SD_MOUNT"
    mount "$BPIROOT" "$BPI_SD_MOUNT"
    mount "$BPIBOOT" "$BPI_SD_MOUNT/boot"
    if [ ! -f "$BPI_SD_MOUNT/usr/bin/qemu-aarch64-static" ]; then
        cp /usr/bin/qemu-aarch64-static "$BPI_SD_MOUNT/usr/bin/"
    fi
    do_mounts "$BPI_SD_MOUNT"
    echo "Chrooting into SD BPI root..."
    chroot "$BPI_SD_MOUNT" /bin/bash -c "source /etc/profile; exec bash"
}

mount_bpi_nfs() {
    echo "Preparing NFS root..."
    if [ ! -d "$BPI_NFS_MOUNT" ]; then
        echo "Error: NFS root not mounted at $BPI_NFS_MOUNT."
        exit 1
    fi
    if [ ! -f "$BPI_NFS_MOUNT/usr/bin/qemu-aarch64-static" ]; then
        cp /usr/bin/qemu-aarch64-static "$BPI_NFS_MOUNT/usr/bin/"
    fi
    do_mounts "$BPI_NFS_MOUNT"
    echo "Chrooting into NFS BPI root..."
    chroot "$BPI_NFS_MOUNT" /bin/bash -c "source /etc/profile; exec bash"
}

unmount_bpi() {
    echo "Unmounting any BPI root..."
    do_unmounts "$BPI_SD_MOUNT"
    do_unmounts "$BPI_NFS_MOUNT"
    umount -l "$BPI_SD_MOUNT/boot" || true
    umount -l "$BPI_SD_MOUNT" || true
    echo "All unmounted."
}

case "$1" in
    mount)
        if [ "$2" == "sd" ]; then
            mount_bpi_sd
        elif [ "$2" == "nfs" ]; then
            mount_bpi_nfs
        else
            echo "Specify root type to mount: sd or nfs"
            exit 1
        fi
        ;;
    unmount)
        unmount_bpi
        ;;
    *)
        echo "Usage: $0 {mount sd|mount nfs|unmount}"
        exit 1
        ;;
esac

Benefits

✅ Avoid SD card wear

✅ Enjoy NVMe speeds from your worker

✅ Build huge packages on your x86_64 CPU and test seamlessly on ARM64

✅ Snapshot rootfs states with ZFS

✅ Swap between distros easily

This approach cuts development time massively—especially when building large kernels or tools like Frank’s BPI router images.


root@BPI-R4:/# lscpu
Architecture:            aarch64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                  4
On-line CPU(s) list:   0-3
Vendor ID:               ARM
Model name:            Cortex-A73
Model:               0
Thread(s) per core:  1
Core(s) per cluster: 4
Socket(s):           -
Cluster(s):          1
Stepping:            r1p0
CPU(s) scaling MHz:  83%
CPU max MHz:         1800.0000
CPU min MHz:         800.0000
BogoMIPS:            26.00
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
Vulnerabilities:
Gather data sampling:  Not affected
Itlb multihit:         Not affected
L1tf:                  Not affected
Mds:                   Not affected
Meltdown:              Not affected
Mmio stale data:       Not affected
Retbleed:              Not affected
Spec rstack overflow:  Not affected
Spec store bypass:     Not affected
Spectre v1:            Mitigation; __user pointer sanitization
Spectre v2:            Mitigation; CSV2, BHB
Srbds:                 Not affected
Tsx async abort:       Not affected
root@BPI-R4:/#
root@worker:/# lscpu
Architecture:                aarch64
CPU op-mode(s):            32-bit, 64-bit
Address sizes:             46 bits physical, 48 bits virtual
Byte Order:                Little Endian
CPU(s):                      32
On-line CPU(s) list:       0-31
Vendor ID:                   GenuineIntel
BIOS Vendor ID:            Intel
Model name:                Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
BIOS Model name:         Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz  CPU @ 2.1GHz
BIOS CPU family:         179
CPU family:              6
Model:                   79
Thread(s) per core:      2
Core(s) per cluster:     16
Socket(s):               1
Cluster(s):              1
Stepping:                1
CPU(s) scaling MHz:      68%
CPU max MHz:             3000.0000
CPU min MHz:             1200.0000
BogoMIPS:                4192.23
Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology
nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dn
owprefetch cpuid_fault epb cat_l3 cdp_l3 intel_ppin ssbd ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_p
t xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi md_clear flush_l1d
Virtualization features:
Virtualization:            VT-x
Caches (sum of all):
L1d:                       512 KiB (16 instances)
L1i:                       512 KiB (16 instances)
L2:                        4 MiB (16 instances)
L3:                        40 MiB (2 instances)
NUMA:
NUMA node(s):              2
NUMA node0 CPU(s):         0-7,16-23
NUMA node1 CPU(s):         8-15,24-31
Vulnerabilities:
Gather data sampling:      Not affected
Indirect target selection: Not affected
Itlb multihit:             KVM: Vulnerable
L1tf:                      Mitigation; PTE Inversion; VMX vulnerable
Mds:                       Vulnerable; SMT vulnerable
Meltdown:                  Vulnerable
Mmio stale data:           Vulnerable
Reg file data sampling:    Not affected
Retbleed:                  Not affected
Spec rstack overflow:      Not affected
Spec store bypass:         Vulnerable
Spectre v1:                Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Spectre v2:                Vulnerable; IBPB: disabled; STIBP: disabled; PBRSB-eIBRS: Not affected; BHI: Not affected
Srbds:                     Not affected
Tsx async abort:           Vulnerable
root@worker:/#

Now we are a little stuck without a compiler, lets install this on either side

apt update
apt upgrade -y

# Essential build environment
apt install -y build-essential

# Useful development tools
apt install -y \
    git \
    curl \
    wget \
    vim \
    sudo \
    gnupg \
    ca-certificates \
    pkg-config \
    libncurses5-dev \
    libncursesw5-dev \
    libssl-dev \
    flex \
    bison \
    bc \
    fakeroot \
    cpio \
    libelf-dev \
    libudev-dev \
    libpci-dev \
    libiberty-dev \
    autoconf \
    automake \
    libtool \
    liblz4-tool \
    python3 \
    python3-pip \
    rsync \
    lsb-release

# Optional, but helpful:
apt install -y \
    htop \
    ncdu \
    tmux \
    tree \
    silversearcher-ag

Now we have a compiler on the NFSROOT and from the BPI

(bpi-bookworm)root@BPI-R4:/# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/12/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 12.2.0-14+deb12u1' --with-bugurl=file:///usr/share/doc/gcc-12/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-12 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (Debian 12.2.0-14+deb12u1)
(bpi-bookworm)root@BPI-R4:/#

And from workers side, within the binfmt_misc chroot

root@worker:/# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/12/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 12.2.0-14+deb12u1' --with-bugurl=file:///usr/share/doc/gcc-12/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-12 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (Debian 12.2.0-14+deb12u1)
root@worker:/#

And a final validation

(bpi-bookworm)root@BPI-R4:/# gcc --version
ld --version
make --version
gcc (Debian 12.2.0-14+deb12u1) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

GNU ld (GNU Binutils for Debian) 2.40
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
GNU Make 4.3
Built for aarch64-unknown-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
(bpi-bookworm)root@BPI-R4:/#
root@worker:/# gcc --version
ld --version
make --version
gcc (Debian 12.2.0-14+deb12u1) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

GNU ld (GNU Binutils for Debian) 2.40
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
GNU Make 4.3
Built for aarch64-unknown-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
root@worker:/#

Happy hacking!

How to Buy a Pre-Built BPI-R4

If you’re in the UK and eager to dive in quickly, AKADATA LIMITED has a couple of spare BPI-R4 boards ready to go — fully assembled and soak-tested, with U-Boot upgraded to unlock all 8GB of RAM. Each board comes fitted with the Wi-Fi radio module, case, antennas, pigtails, SD Card, and NVMe.

We’re also including a reliable UART setup — using ESP-01 modules and an ESP-Prog — so you’ll have a solid serial console connection for development or recovery.

Important Note: The Linux kernel for the BPI-R4 is still under active development, as documented on Frank’s BPI-R4 project page. Some features (including Wi-Fi) remain a work in progress. These boards are sold strictly for development and experimentation, and lack of certain functions like Wi-Fi is not a reason for return under warranty.

The full package is available for around £475, including next-day postage and VAT at 20%. A one-year warranty applies (excluding damage from user-modified firmware), and we’re happy to provide remote support to help unbrick boards if needed.

If you choose to return a board for reasons other than fault, a 10% restocking fee applies, and the buyer covers return postage in cases of change of mind.

If you’re interested and want one in your hands within 48 hours, you’ll find our phone number on the Privacy page of our website.