SystemTap

In computing, SystemTap () is a scripting language and tool for dynamically instrumenting running production Linux-based operating systems. System administrators can use SystemTap to extract, filter and summarize data in order to enable diagnosis of complex performance or functional problems. SystemTap consists of free and open-source software and includes contributions from Red Hat, IBM, Intel, Hitachi, Oracle, the University of Wisconsin-Madison and other community members. SystemTap debuted in 2005 in Red Hat Enterprise Linux 4 Update 2 as a technology preview. After four years in development, SystemTap 1.0 was released in 2009. SystemTap runs fully supported in all Linux distributions including RHEL / CentOS 5 since update 2, SLES 10, Fedora, Debian and Ubuntu. Tracepoints in the CPython VM and JVM were added in SystemTap 1.2 in 2009. In November 2019, SystemTap 4.2 included prometheus exporter. SystemTap files are written in the SystemTap language (saved as .stp files) and run with the stap command-line. The system carries out a number of analysis passes on the script before allowing it to run. Scripts may be executed with one of three backends selected by the --runtime= option. The default is a loadable kernel module, which has the fullest capability to inspect and manipulate any part of the system, and therefore requires most privilege. Another backend is based on the dynamic program analysis library DynInst to instrument the user's own user-space programs only, and requires least privilege. The newest backend is based on eBPF byte-code, is limited to the Linux kernel interpreter's capabilities, and requires an intermediate level of privilege. In each case, the module is unloaded when the script has finished running. Scripts generally focus on events (such as starting or finishing a script), compiled-in probe points such as Linux "tracepoints", or the execution of functions or statements in the kernel or user-space. Some "guru mode" scripts may also have embedded C, which may run with the -g command-line option.