2026
The Linux memory-management mechanism underneath the incident: pages, pageblocks, CMA, FOLL_PIN, FOLL_LONGTERM, migration, and the second pin that returned ENOMEM. Part 3 of Memory Registration, All the Way Down.
A production debugging story across NCCL, ConnectX, nvidia-peermem, DMA-BUF, retsnoop, and an inherited six-gigabyte CMA reserve. Part 2 of Memory Registration, All the Way Down.
A ground-up walk through DMA, RDMA, PCIe BARs, memory registration, MKeys, nvidia-peermem, and DMA-BUF. Part 1 of Memory Registration, All the Way Down.
An NCCL 2.17.1 deep dive: an abort path waiting for a peer-visible close, a file descriptor duplicated by fork(), and the shutdown() that 2.18.1 added.
An RDMA deep dive: a GPUDirect visibility fence registered four bytes of host memory, and forked children started segfaulting on pointers the parent could read fine. The page-granular fork rule hiding under ibv_reg_mr().
A recurring production SEV, an RDMA device that was present and not found, and the decades-old linker rule that split one library's state into two. An InfiniBand deep dive from the linker's side. Part 2 of Linking & Loading.
Trace one command from keystroke to main(), through the kernel, loader, and PLT/GOT, until two classic deploy errors stop being mysterious.