Jetson Nano Kubernetes Kernel Guide

Build a Custom Kernel for Modern K8s Support

🚀 Unlock Full Kubernetes Support on Jetson Nano

The NVIDIA Jetson Nano is a powerful and compact platform for AI and edge computing, but its default kernel is missing critical features required by modern Kubernetes workloads. If you’ve run into persistent kube-proxy sync errors, broken container networking, or limited storage support, you’re not alone — the out-of-the-box experience just isn’t Kubernetes-friendly.

This guide provides a step-by-step walkthrough for building and deploying a fully customized, production-grade Linux kernel for Jetson Nano that enables enterprise-grade Kubernetes features. By enabling modules like netfilter, eBPF, iSCSI, NFS, and CNI bridge networking, you’ll be able to run real-world Kubernetes applications with confidence — directly on your Jetson Nano.

Whether you’re managing IoT edge workloads, experimenting with K3s, or turning your Nano into a full-blown node in a lightweight Kubernetes cluster, this guide will help you get there with automation, safety, and performance in mind.

Whether you’re running edge workloads, deploying a small K8s cluster, or just experimenting, this custom kernel will unlock the full potential of your Jetson Nano.

---

---

🎯 Project Overview

Aspect	Details
Target Platform	Jetson Nano (all variants)
Source Kernel	Linux 4.9.337-tegra (L4T 32.7.4)
Custom Kernel	Linux 4.9.337-fresh
Goal	Modern Kubernetes networking + storage
Time to Build	60–90 minutes
Skill Level	Intermediate to Advanced

---

Automated Scripts

To get started quickly, you can download all three automated scripts (jetson-backup.sh, jetson-build-kernel.sh, and jetson-deploy-and-verify.sh) directly from this Gist:

curl -O https://gist.githubusercontent.com/acerbetti/b5c14d3ecb7e3c31169fac334cb1e95f/raw/jetson-backup.sh
curl -O https://gist.githubusercontent.com/acerbetti/b5c14d3ecb7e3c31169fac334cb1e95f/raw/jetson-build-kernel.sh
curl -O https://gist.githubusercontent.com/acerbetti/b5c14d3ecb7e3c31169fac334cb1e95f/raw/jetson-deploy-and-verify.sh
chmod +x jetson-*.sh

📁 These scripts encapsulate the full backup, build, and deployment workflow and are safe to run as documented below.

Backup Script: jetson-backup.sh

• Creates complete system backup
• Validates SSH connectivity
• Archives Image, DTB, modules, and configs
• Generates recovery instructions

Build Script: jetson-build-kernel.sh

• Validates Ubuntu 18.04 environment
• Downloads Linaro GCC 7.3.1-2018.05 toolchain
• Applies Kubernetes-critical kernel config patches
• Builds kernel, DTBs, and modules
• Outputs a deployment-ready .tar.gz package

Deploy Script: jetson-deploy-and-verify.sh

• Verifies package integrity
• Creates rollback backups
• Installs Image, DTB, and modules
• Tests critical kernel features post-boot

---

⚙️ Detailed Kernel Configuration

Critical Kubernetes Fixes

• CONFIG_NETFILTER_XT_MATCH_NFACCT=m — fixes kube-proxy sync errors
• CONFIG_BRIDGE_NETFILTER=m — enables advanced CNI plugins

iSCSI + NFS Support

• CONFIG_ISCSI_TCP=m and CONFIG_SCSI_ISCSI_ATTRS=m
• CONFIG_NFS_V4_1=y, CONFIG_NFS_V4_2=y, CONFIG_NFSD=m

Networking Drivers

• CONFIG_VXLAN=m, CONFIG_IPVLAN=m, CONFIG_MACVLAN=m, CONFIG_VETH=m

Container Runtime Support

• Cgroup configs: CONFIG_CGROUP_DEVICE, CONFIG_CGROUP_PIDS, etc.
• Namespaces: CONFIG_NET_NS, CONFIG_USER_NS, etc.

Security and Observability

• CONFIG_BPF_SYSCALL=y, CONFIG_SECURITY_APPARMOR=y, etc.
• CONFIG_PERF_EVENTS=y, CONFIG_FTRACE=y

✅ All configurations validated by the build script

---

⚡ Quick Start

# 1. Backup (any SSH-enabled machine)
./jetson-backup.sh

# 2. Build (on Ubuntu 18.04 build machine)
./jetson-build-kernel.sh

# 3. Deploy (copy package, then run on Jetson)
scp jetson-nano-kernel-fresh-*.tar.gz jetson@<JETSON-IP>:~/
ssh jetson@<JETSON-IP>
./jetson-deploy-and-verify.sh

⏳ Total time: ~60–90 minutes (45 min build + 15 min deploy/verify)

---

🔍 Manual Process

If you’re curious or want to customize deeply:

1. Setup Ubuntu 18.04 build machine
2. Download Linaro toolchain + L4T kernel source
3. Extract current Jetson config
4. Apply configuration patch
5. Build kernel, DTB, and modules
6. Package and transfer to Jetson
7. Deploy + verify kernel manually

The automated scripts encapsulate all these steps.

---

✅ Deployment and Validation

After rebooting into your new kernel:

uname -r  # Expect: 4.9.337-fresh
lsmod | grep -E 'nfacct|iscsi|nfs|vxlan'  # All should be present

Verify:

• kube-proxy has no nfacct errors
• NFS/iSCSI mounts work
• CNI plugins (e.g., Calico, Flannel) are functional
• Container runtime (Docker/containerd) works fine
• Modules load cleanly

---

🧯 Troubleshooting and Recovery

Boot Failure?

• Use serial console or SD card reader
• Restore Image, DTBs, and modules from backup

Module Errors?

• Ensure version match: uname -r == modules directory
• Run depmod -a and modprobe

Network Broken?

• Check ip link, brctl, iptables
• Reload bridge and netfilter modules

Container Runtime Not Starting?

• Restart Docker/containerd
• Check cgroup mounts

Rollback steps are documented in the backup directory.

---

📊 Complete Implementation Summary

Feature	Before (Stock)	After (Custom)
kube-proxy Errors	❌ Frequent	✅ Resolved
iSCSI Support	❌ Missing	✅ Full Support
NFS v4.1 Support	⚠️ Limited	✅ Complete
VXLAN/Overlay Network	⚠️ Not Working	✅ Enabled
BPF for Cilium	⚠️ Unusable	✅ Fully Supported

Deployment Package:

jetson-nano-kernel-fresh-YYYYMMDD-HHMMSS.tar.gz
├── Image-fresh
├── dtb/
├── modules/
├── BUILD_INFO.txt
└── INSTALLATION_GUIDE.txt

Performance Impact:

Metric	Change
Boot Time	+2s
Memory Usage	+20MB baseline
Storage	+500MB
Container Perf	Improved

---

🧠 Final Thoughts & Lessons Learned

This project wasn’t just about building a kernel — it was about learning how deep and fragile the base assumptions in edge Linux systems can be when placed under Kubernetes.

Here’s what I learned:

• Toolchain matters: The exact version of Linaro GCC (7.3.1-2018.05) is essential. Using anything newer breaks the L4T kernel source.
• Backup is not optional: Kernel missteps can leave you bootlooped or blind. Automating rollback is a life-saver.
• Modules must match: Kernel version and module directories must match exactly or modprobe will silently fail.
• Container support is delicate: You must explicitly enable cgroup and namespace features — they’re not defaults.
• Serial console access is gold: When the GUI fails, UART saves.

With the right configs and patience, Jetson Nano goes from a hobbyist board to a production-grade, cloud-native edge node. The transformation is real — and now fully reproducible.