Skip to main content
emisar

Linux debugging toolkit

v0.3.0

General-purpose Linux diagnostics + low-level remediation: process and memory tops, vmstat/iostat snapshots, socket inventories, per-PID inspection, kernel-state checks, network reachability, plus fix-it actions (drop_caches, kill_pid by signal, sysctl_set). Use as the first-touch pack when something is wrong.

31 allowed by default 5 need approval by default
Pack ID
debugging
Vendor
emisar
OS
linux
Actions
36
Required binaries. Install these on the host before relying on the pack — an action that calls a missing one fails at run time.
iostat ping ps sar ss sysctl vmstat

Install

emisar pack install fetches this pack, re-validates it, and verifies its content hash against the --hash below — the exact bytes this page was rendered against, so a tampered copy is rejected — before copying it into the runner's packs dir. The command reloads a running daemon itself; no manual restart.

on the runner host 
sudo emisar pack install debugging --hash sha256:ce1c2556b932d436756a5fbc92a661fa9c121ce1ebcc72d58005cf0c6db67ded

Setup

Operates on the local runner host — no credentials needed.

Host access

Run these commands yourself on the runner host. Emisar shows and copies setup recipes; it never runs them.

Inspect protected process state owned by other users.

debugging.netstat_listendebugging.lsof_portdebugging.top_open_filesdebugging.pid_contextdebugging.pid_env_keysdebugging.pid_connectionsdebugging.pid_argvdebugging.pid_environdebugging.pid_cwddebugging.pid_fdsdebugging.pid_fd_summarydebugging.pid_statusdebugging.pid_threadsdebugging.pid_limitsdebugging.pid_iodebugging.tcp_retrans_top

Grant process-inspection capabilities to the Emisar service

Grant access 
sudo install -d -m 0755 /etc/systemd/system/emisar.service.d
sudo rm -f /etc/systemd/system/emisar.service.d/10-debugging-host-access.conf
printf '%s\n' '[Service]' 'AmbientCapabilities=CAP_SYS_PTRACE CAP_DAC_READ_SEARCH' | sudo tee /etc/systemd/system/emisar.service.d/20-debugging-process-inspection.conf >/dev/null
sudo systemctl daemon-reload
sudo systemctl restart emisar
Verify access 
systemctl show emisar --property=AmbientCapabilities --value | grep -Fwi cap_sys_ptrace
systemctl show emisar --property=AmbientCapabilities --value | grep -Fwi cap_dac_read_search

Impact: Every Emisar action on this runner can inspect, attach to, and modify processes outside the runner user, including their arguments, environments, file descriptors, memory-related metadata, and secrets, and bypass host file read and directory search permissions.

Read protected kernel slab metadata from /proc/slabinfo.

debugging.slabtop

Grant protected-file read access to the Emisar service

Grant access 
sudo install -d -m 0755 /etc/systemd/system/emisar.service.d
sudo rm -f /etc/systemd/system/emisar.service.d/10-debugging-host-access.conf
printf '%s\n' '[Service]' 'AmbientCapabilities=CAP_DAC_READ_SEARCH' | sudo tee /etc/systemd/system/emisar.service.d/21-debugging-kernel-files.conf >/dev/null
sudo systemctl daemon-reload
sudo systemctl restart emisar
Verify access 
systemctl show emisar --property=AmbientCapabilities --value | grep -Fwi cap_dac_read_search

Impact: Every Emisar action on this runner can bypass file read and directory search permissions across the host, including files unrelated to slab diagnostics.

Read the protected kernel message ring buffer.

debugging.dmesg_taildebugging.dmesg_oom

Grant kernel-log access to the Emisar service

Grant access 
sudo install -d -m 0755 /etc/systemd/system/emisar.service.d
sudo rm -f /etc/systemd/system/emisar.service.d/10-debugging-host-access.conf
printf '%s\n' '[Service]' 'AmbientCapabilities=CAP_SYSLOG' | sudo tee /etc/systemd/system/emisar.service.d/22-debugging-kernel-log.conf >/dev/null
sudo systemctl daemon-reload
sudo systemctl restart emisar
Verify access 
systemctl show emisar --property=AmbientCapabilities --value | grep -Fwi cap_syslog

Impact: Every Emisar action on this runner can read or clear the host kernel message ring buffer, change console logging, and view restricted kernel addresses. Kernel logs may contain device, process, address, and failure details.

Signal processes owned by other users.

debugging.kill_pid

Grant process-signalling access to the Emisar service

Grant access 
sudo install -d -m 0755 /etc/systemd/system/emisar.service.d
sudo rm -f /etc/systemd/system/emisar.service.d/10-debugging-host-access.conf
printf '%s\n' '[Service]' 'AmbientCapabilities=CAP_KILL' | sudo tee /etc/systemd/system/emisar.service.d/23-debugging-process-signal.conf >/dev/null
sudo systemctl daemon-reload
sudo systemctl restart emisar
Verify access 
systemctl show emisar --property=AmbientCapabilities --value | grep -Fwi cap_kill

Impact: Every Emisar action on this runner can signal processes owned by other users, including terminating or force-killing them.

Open ICMP sockets where the host has a closed ping group range.

debugging.ping_host

Grant raw-socket access to the Emisar service

Grant access 
sudo install -d -m 0755 /etc/systemd/system/emisar.service.d
sudo rm -f /etc/systemd/system/emisar.service.d/10-debugging-host-access.conf
printf '%s\n' '[Service]' 'AmbientCapabilities=CAP_NET_RAW' | sudo tee /etc/systemd/system/emisar.service.d/24-debugging-network.conf >/dev/null
sudo systemctl daemon-reload
sudo systemctl restart emisar
Verify access 
systemctl show emisar --property=AmbientCapabilities --value | grep -Fwi cap_net_raw

Impact: Every Emisar action on this runner can open raw sockets, including sending crafted packets and reading traffic on this host's interfaces.

Read privileged kernel stacks or change protected kernel state.

debugging.pid_stackdebugging.drop_cachesdebugging.sysctl_set

Run the Emisar service as root

Grant access 
sudo install -d -m 0755 /etc/systemd/system/emisar.service.d
sudo rm -f /etc/systemd/system/emisar.service.d/10-debugging-host-access.conf
printf '%s\n' '[Service]' 'User=root' 'Group=root' | sudo tee /etc/systemd/system/emisar.service.d/30-debugging-root.conf >/dev/null
sudo systemctl daemon-reload
sudo systemctl restart emisar
Verify access 
test "$(systemctl show emisar --property=User --value)" = root

Impact: Every Emisar action on this runner executes as root. Debug actions can read other processes' arguments and environments, kill processes, and change kernel settings.

Notes

  • Basic diagnostics such as vmstat, loadavg, and host-permitted process reads work unprivileged. Protected process, network, and kernel state is mapped below.
  • Apply every host-access recipe that covers an action you enable. The recipes are additive and remove the retired blanket-root drop-in when first applied.

Verify it works

Runs debugging.loadavg, a low-risk read that confirms the pack can reach its target. Run it on the host once the pack is installed; pack install runs it for you.

on the runner host 
sudo emisar pack verify debugging

Install and configure a pack walks through the whole sequence on a host.

Actions 36 total

View on GitHub
  • debugging.disk_free exec low Low — read-only or trivially reversible

    df + mounts

    Return `df -hT` output. Filesystem type, size, used, avail, and mountpoint for every mounted filesystem. Use as the first check when a write fails with ENOSPC or when /var/log has gone dark.

    View source on GitHub
  • debugging.dmesg_oom script low Low — read-only or trivially reversible

    OOM-kill events from dmesg

    Filter dmesg for OOM-killer events. Returns the kernel log lines showing process id, name, RSS, and OOM score for every killed process. The "why did mysqld vanish?" answer. Falls back to `journalctl -k` when dmesg is not permitted (needs CAP_SYSLOG / root, or journal read access via systemd-journal / adm). Read-only.

    View source on GitHub
  • debugging.dmesg_tail script medium Medium — changes state, easily reversible

    Recent kernel messages

    Return the last N kernel log lines. Surfaces OOM kills, hardware errors, network link flaps, and dropped packets. Reads the kernel ring buffer via dmesg; when that is not permitted (modern kernels gate it behind CAP_SYSLOG) it falls back to `journalctl -k`, which works when the runner can read the journal (root, or a member of systemd-journal / adm). Read-only.

    View source on GitHub
  • debugging.drop_caches exec high High — service-affecting

    echo <n> > /proc/sys/vm/drop_caches

    Force the kernel to drop pagecache / dentries / inodes. Use only when a benchmark or memory-fragmentation test requires a cold cache — never on prod for "I want more free RAM". Production RSS appears to drop briefly, then the cache repopulates and the next workload is slower until it warms back up.

    View source on GitHub
  • debugging.iostat exec low Low — read-only or trivially reversible

    iostat per-device sample

    Run `iostat -xz 1 N` to capture N one-second samples of extended per-device statistics. Surfaces await, %util, queue depth. Use to prove or disprove "the disk is slow" before chasing application code.

    View source on GitHub
  • debugging.kernel_taint exec low Low — read-only or trivially reversible

    Kernel taint state

    Read /proc/sys/kernel/tainted. A non-zero value means a binary module, a proprietary driver, or a kernel crash has compromised the integrity of the running kernel. The number is a bitmask; this action returns both the raw value and the decoded flags.

    View source on GitHub
  • debugging.kill_pid exec high High — service-affecting

    kill -<signal> <pid>

    Send a signal to one process by PID. SIGTERM is the polite default — the process gets a chance to flush state. SIGKILL is unrecoverable — use only when SIGTERM is ignored. Watch for PID reuse: confirm the target via `pid_status` first.

    View source on GitHub
  • debugging.loadavg exec low Low — read-only or trivially reversible

    Load + memory + uptime snapshot

    Read /proc/loadavg, /proc/meminfo, and /proc/uptime to produce a one-shot system snapshot. Cheap. Use as the very first check when triaging a host alert.

    View source on GitHub
  • debugging.lsof_port exec low Low — read-only or trivially reversible

    Who owns a TCP port?

    Return the PID/process that has a given TCP port open (listening or connected), via `ss -tnp` (not lsof, despite the name). Use to answer "EADDRINUSE: who's on 8080?" or to confirm a stuck connection to an upstream. Read-only.

    View source on GitHub
  • debugging.mem_top exec low Low — read-only or trivially reversible

    Top processes by RSS

    Return the top N processes sorted by resident-set size (RSS). Use when /proc/meminfo or `free` shows pressure and you need the offender. RSS does not double-count shared pages, so a "leak" candidate showing high RSS is worth investigating.

    View source on GitHub
  • debugging.netstat_connections exec low Low — read-only or trivially reversible

    Established connection summary

    Return counts of TCP connections grouped by remote peer + state. Useful for spotting connection storms (single host) or TIME_WAIT pressure. Read-only.

    View source on GitHub
  • debugging.netstat_listen exec low Low — read-only or trivially reversible

    Listening sockets

    Return TCP and UDP listening sockets with the owning process (`ss -tulnp`). Use to confirm whether an expected daemon is actually bound and on which interfaces. Read-only.

    View source on GitHub
  • debugging.pid_argv exec high High — service-affecting

    Process full argument vector

    Show the full argument vector a Linux PID was started with, one argument per line. Arguments often contain tokens, passwords, private URLs, or inline configuration, so this action is high risk and should require explicit approval. Prefer pid_context when executable identity and ancestry are enough.

    View source on GitHub
  • debugging.pid_connections script low Low — read-only or trivially reversible

    Process remote socket ownership

    List a bounded ss inventory of connected TCP and UDP sockets owned by one Linux PID. Each line includes protocol/state, local and remote endpoints, and kernel process ownership. Listening sockets and unrelated processes are omitted.

    View source on GitHub
  • debugging.pid_context script low Low — read-only or trivially reversible

    Process identity and parent chain

    Show a compact identity record for one Linux PID: owner, start time, command name, executable, cwd, and a parent chain capped at sixteen entries. Command arguments and environment values are deliberately omitted.

    View source on GitHub
  • debugging.pid_cwd exec low Low — read-only or trivially reversible

    Process cwd + exe

    Return the working directory and executable path of a PID. Use before drawing conclusions from a process name — `nginx` could be any of several binaries depending on PATH order.

    View source on GitHub
  • debugging.pid_env_keys script low Low — read-only or trivially reversible

    Process environment variable names

    List only the valid environment variable names inherited by one Linux PID, one per line. Values are never emitted, so this can safely establish that variables such as CONSUL_HTTP_TOKEN or NOMAD_ADDR reached an unexpected process. Use the high-risk pid_environ action only when values are essential.

    View source on GitHub
  • debugging.pid_environ exec high High — service-affecting

    Process environment

    Show the full set of environment variables a PID was started with — reads /proc/<pid>/environ and turns NULs into newlines. This deliberately surfaces the process's entire environment, which commonly carries injected secrets (DB URLs, API keys, cloud credentials); scope it by policy and prefer pid_status / pid_limits when you don't need the values. The runner's redaction is a fail-closed backstop, not a guarantee — it is pattern-bound and can miss a bespoke secret whose name and value match no rule.

    View source on GitHub
  • debugging.pid_fd_summary script low Low — read-only or trivially reversible

    Process open-file count and limit

    Count one process's currently open file descriptors and show its soft and hard open-files limits plus soft-limit usage. Use after a process ranking, socket alert, or known PID to confirm whether it is actually near EMFILE. Refresh the PID from the service or process listing before retrying: a stale PID may have exited or been reused. Read-only.

    View source on GitHub
  • debugging.pid_fds exec low Low — read-only or trivially reversible

    Process open file descriptors

    List what each FD in /proc/<pid>/fd points at. Sockets show up as `socket:[N]`, files as their path. Use to spot FD-leak candidates or confirm a daemon has the log file you expect.

    View source on GitHub
  • debugging.pid_io exec low Low — read-only or trivially reversible

    Process I/O accounting

    Show `/proc/<pid>/io` for one process — bytes read and written (logical and physical), syscall counts, and cancelled writes. Use to find which process is driving disk I/O.

    View source on GitHub
  • debugging.pid_limits exec low Low — read-only or trivially reversible

    Resource limits applied to a PID

    Read /proc/<pid>/limits — every ulimit applied (open files, stack size, NPROC, memlock, msgqueue, niceness). Use to confirm whether a process is actually getting the higher limits its service file requested.

    View source on GitHub
  • debugging.pid_stack exec low Low — read-only or trivially reversible

    Process kernel stack

    Read /proc/<pid>/stack — the current kernel-side stack trace. Use to find what syscall a stuck process is hung in (futex, read, write, lock_kernel). This pack requires the runner service to run as root because the read crosses kernel, process, and file-permission checks.

    View source on GitHub
  • debugging.pid_status exec low Low — read-only or trivially reversible

    Process status block

    Show `/proc/<pid>/status` for one process — uid/gid, capability set, signal masks, RSS and peak RSS, voluntary and involuntary context switches, and OOM score. More detail than `ps` for one PID.

    View source on GitHub
  • debugging.pid_threads exec low Low — read-only or trivially reversible

    Per-thread stats for one PID

    List every thread of one PID with CPU%, policy, priority, comm name. Use when one PID's CPU is high but it's unclear which thread inside it is hot. Read-only.

    View source on GitHub
  • debugging.ping_host exec low Low — read-only or trivially reversible

    Ping a host

    Send N ICMP echo requests to a target. Use to confirm L3 reachability when a higher-layer probe (TCP/HTTP) fails. Target is pattern-restricted to safe hostname/IPv4 characters to prevent argument injection. Needs CAP_NET_RAW on the runner identity where the host does not open net.ipv4.ping_group_range: the runner's no_new_privs setting blocks ping's own setuid or file-capability elevation.

    View source on GitHub
  • debugging.processes_top exec low Low — read-only or trivially reversible

    Top processes by CPU

    Return the top N processes sorted by CPU%. Standard `ps` output — pid, user, %cpu, %mem, rss, command. Use as a first-touch check before going deeper with per-PID inspection.

    View source on GitHub
  • debugging.sar_recent exec low Low — read-only or trivially reversible

    sar 3-sample CPU snapshot

    Sample CPU utilization with `sar -u 5 3` — three 5-second samples. Surfaces user/system/iowait/steal/idle split with statistical smoothing. Use when `vmstat`'s noise hides the signal. Requires the sysstat package.

    View source on GitHub
  • debugging.slabtop exec low Low — read-only or trivially reversible

    Kernel slab cache top consumers

    List the top 40 kernel slab caches by cache size (`slabtop -o -s c | head -40`). Use when /proc/meminfo shows high `Slab` but no userland process accounts for the memory. dentry / inode pressure is the usual answer.

    View source on GitHub
  • debugging.socket_summary exec low Low — read-only or trivially reversible

    Socket counts by family + state

    Show socket usage totals with `ss -s` — aggregate counts: TCP/UDP/raw/frag, timewait, by state. Cheaper than the per-connection enumeration. Use as a one-shot "are we close to a port-tuple exhaustion?" check.

    View source on GitHub
  • debugging.swap_status exec low Low — read-only or trivially reversible

    Swap usage summary

    `swapon --show` plus the per-process swap usage (top 20). Read-only. Use to spot a host actively swapping — swap-in I/O is the slowest userland-visible memory tier.

    View source on GitHub
  • debugging.sysctl_set exec high High — service-affecting

    sysctl -w <key>=<value>

    Change a runtime kernel parameter. Change is not persistent — reverts at next boot unless mirrored in /etc/sysctl.d/. Wrong values can crash the kernel (net.* tunables especially). Read the current value first.

    View source on GitHub
  • debugging.tcp_retrans_top exec low Low — read-only or trivially reversible

    TCP connections with retransmits

    List established TCP flows with retransmission counters (`ss -i state established`, filtered). Use when network latency is high — surfaces which peers are seeing TCP loss without a tcpdump. Read-only.

    View source on GitHub
  • debugging.tcp_summary exec low Low — read-only or trivially reversible

    TCP state counts

    Return the count of TCP sockets in each state (ESTAB, TIME-WAIT, CLOSE-WAIT, FIN-WAIT-*, SYN-*). High CLOSE-WAIT usually means the application isn't close()ing; high TIME-WAIT means short-lived client connections. Read-only.

    View source on GitHub
  • debugging.top_open_files script low Low — read-only or trivially reversible

    Top processes by open-file count

    Count `/proc/<pid>/fd` entries and return the top 20 readable processes. This reads directory entries without resolving every descriptor target, so a process with tens of thousands of sockets does not turn the diagnostic into another global lsof crawl. Use debugging.pid_fd_summary on a candidate to compare its count with its own open-files limit. Without the process-inspection host-access recipe it sees only permitted processes. Do not retry an unchanged timeout; use the targeted summary when you already know the PID. Read-only.

    View source on GitHub
  • debugging.vmstat exec low Low — read-only or trivially reversible

    vmstat sample

    Run `vmstat 1 N` to capture N one-second samples. Shows run/block queues, free memory, swap pressure, context switches, and per-CPU user/system/iowait. Use to spot a CPU-bound vs IO-bound vs context-switch-storm problem.

    View source on GitHub
← Back to registry
Pack reference  Publish your own