WRITTEN IN PLAIN AMERICAN ENGLISH.
About
CLAY TRIBUNE.
Advertisement

Developer Builds Tool to Find Out Why Bun’s Zig Build Ran So Slow

A developer built a tracer to measure Bun's build time, finding a linker step consuming most of the duration.

By mitch·5 min read
A timeline chart showing a linker step towering above many smaller build tasks.

A developer created a tool named buildprof with one specific aim: to determine why Bun’s Zig build took far longer than its Rust build. The reason proved to be a linker operation that ran by itself at the conclusion, lasting more than sixteen minutes.

A single timeline displays every process a build command launches, along with all of its subprocesses. Time runs left to right, with bar width showing how long each process took. Child processes sit beneath the ones that started them.

Profiling a Build in Real Time

A clean build of ripgrep was captured on video, with the developer showing the entire process from start to finish. The tool buildprof is used by placing it before any build command, allowing the recording to track exactly how the build proceeds.

Advertisement

The buildprof command runs five distinct build commands. Each one is a separate line of execution. The first runs make -j16. The second invokes cargo build. The third executes ninja -C out/target. The fourth simply calls just build. The fifth runs ./dev/custom-build-script.sh from its location.

Jarred Sumner, the chief architect of the Bun JavaScript runtime, made a claim about speed that prompted someone to want to settle an argument. His tweet said Bun’s new Rust build ran more than five times faster on Linux than its old Zig build. The distinction that counted was how each build was compiled: the Zig build used Full LTO, while the Rust build used ThinLTO.

Repeating the Numbers

A developer attempted to recreate the results by checking out Bun 1.3.14 and Bun 1.4.0. He then composed scripts to simulate their Linux x64 CI builds within a 6-core, 12-thread Linux virtual machine, keeping track of the build sequence and its dependencies all along. The elapsed times lined up with Jarred’s.

Then he pointed buildprof at the actual builds.

The Process Tree Approach

Build systems do not consist of a solitary program. Instead, when you enter cargo build or zig build, the tool determines what needs to be reconstructed, arranges the ordering among parts, and decides what can proceed side by side. It sets off compilers, code generators, archivers, linkers, and any number of scripts. Each of those scripts then starts further programs, which in turn start still more.

Cargo describes its work in terms of crates. Ninja describes its work in terms of build edges. CMake describes its work by generating instructions for another build system to use. But from the operating system’s point of view, most of these systems mostly look like processes launching other processes.

A Rust build might contain a chain like this:

cargo
└── rustc
└── cc
└── collect2
└── ld.lld

A record of when each subprocess began and concluded was set down, arranged along a line of time, and through it several pleasing qualities emerged.

  • It is build-system agnostic: Cargo, Ninja, Zig, Make, and most other build systems spawn processes, so no special integration is needed.
  • It naturally includes custom scripts: repository setup, dependency fetching, code generators, asset processors.
  • It follows files between build steps: recording which files each process reads and writes lets you see which steps produce the inputs for others, even across build systems.

Finding the Bottleneck

Using buildprof, the developer captured the CI build during the Zig era. Upon inspection, he found a major issue: the ld.lld linker command consumed an enormous amount of time by itself. It executed at the end of the build, taking well over sixteen minutes — roughly two-thirds of the total build duration.

The linker’s command line appeared after a click, and buildprof had recorded it automatically. The presence of Full LTO matched what Jarred had claimed. With the link taking so long, it had become the primary suspect.

The process tree gave no sign of who was truly behind those sixteen minutes. Thankfully, LLD keeps its own record of internal timing events, and buildprof can bring them in using the --compiler-traces flag.

This time, the developer turned on the traces before recording the final link.

What the Traces Showed

A single step took up two-thirds of the Zig build’s time, and that step was the bottleneck.

An online video documents the entire ripgrep build process. The buildprof project calls GitHub home. The takeaway remains straightforward: should a build prove sluggish, the profiler points toward the source of the delay.

Tool Function
buildprof Records process trees and lays them on a timeline
LLD Linker that records internal timing events
Full LTO Brings all compilation units into one large optimization job
ThinLTO Preserves separation so work can run in parallel

A developer describes how a tool pointed toward an answer through a chain of actions. The essential moves were laid out in sequence.

  1. Reproducing the numbers by replaying Bun’s CI builds on a 6-core, 12-thread Linux VM.
  2. Recording the process tree with buildprof, which showed the linker dominating the build.
  3. Enabling LLD’s internal timing events with the --compiler-traces flag.
  4. Confirming Full LTO was the setting used in the slow build.

The point here is that build profilers can reveal problems that aren’t clear from looking at a build system. The developer created a tool to find an answer, and that answer wasn’t inside the compiler or the build system. It turned out to be in the linker instead.

See the video the story is built around at Lalitm.

Advertisement

Leave a Reply

Your email address will not be published. Required fields are marked *