Testing

🧭 Where we left off Welcome back to our technical journey on measuring binary test coverage! In part 4 we introduced funkoverage, a native, high-performance eBPF-based coverage tracer that leverages uprobe_multi to capture function entry events in GNU/Linux with less than 2% overhead. It parses static library dependencies using ldd during installation and maps uprobes to all discovered functions. But there was a big fat elephant in the room: dlopen(). Some of the most complex, modular software in the world — such as web servers with dynamic modules, plugin-based enterprise applications, and multi-protocol databases — load their dependencies at runtime on-the-fly. Because these libraries are not declared in the ELF binary’s DT_NEEDED header, they are invisible to ldd during installation. Today, we are going to chase the dynamic loading ghost, explore how we solved this with an elegant, highly scalable, event-driven eBPF JIT instrumentation strategy, and battle-test it live on standard, unmodified production binaries like Nginx and OpenSSL! ...

🤔 What’s a syscall, anyway? Image credits: Mike Norris via Pexels Every time your program reads a file, writes to a socket, or allocates memory, at some point it has to ask the kernel to actually do it. That ask is a syscall, the narrow, well defined boundary where userspace code crosses into the kernel. write(), read(), open(), mmap(), these aren’t just library functions, they’re the entire vocabulary your program has for talking to the outside world. Everything else, fwrite, Python’s file.write(), std::fs::File, is just a wrapper around this same handful of kernel entry points. ...

🧭 Where we left off Welcome back to our ongoing series on measuring test coverage for binary programs! In part 1 we used Go’s built-in -cover flag — clean and accurate, but only works if you own the source and can recompile. In part 2 we used valgrind and gdb to trace gzip without touching its source. In part 3 we explored Intel PIN, a proper dynamic binary instrumentation framework — powerful, but it came with a ~100MB proprietary C++ SDK and was limited to x86_64. ...

🎉 Happy New Year! As anyone working with OpenQA knows, it is a powerful tool for automated testing. But sometimes, the workflow around re-triggering those tests for investigation purposes can feel a bit… Manual. Recently, I found myself in a repetitive cycle while debugging complex test scenarios. My workflow looked something like this: Take a known “good” job URL. Open a scratchpad editor. Craft a long openqa-clone-job command with tons of specific override parameters (BUILD=0, custom git branches, skipping specific modules, etc.). Run the command in the terminal, wait for the output. Painfully visual scan of the terminal output, to identify the newly created job URLs, and select them with my mouse, to copy them. Paste them into openqa-mon or a text file to track their progress. Till now I used a Bash script that kind of helped, but modifying giant arrays of arguments for every different debugging scenario was annoying and error-prone. ...