Kafka cluster operations

v0.1.13

Topic introspection, consumer-group lag, broker config, ACLs, and narrow mutators (reset_offsets, alter_topic_retention, delete_consumer_group, preferred_leader_election). Auth via KAFKA_BOOTSTRAP env var on the runner host plus optional KAFKA_COMMAND_CONFIG (jaas / SASL / SSL).

15 allowed by default 3 need approval by default 2 denied by default

Pack ID

kafka

Vendor

emisar

linux

Actions

Required binaries. Install these on the host before relying on the pack. An action that calls a missing binary fails at run time.


              kafka-topics.sh

Install

emisar pack install fetches this pack, re-validates it, and verifies its content hash against the --hash below before copying it into the runner's packs dir. That hash covers the exact bytes this page was built from, so a tampered copy is rejected. The command reloads a running daemon itself, so you don't need to restart it manually.

on the runner host

sudo emisar pack install kafka --hash sha256:6771e713d679ac3f5ecd1f96b6c624fd2c7f86ed747f1e2143ce8fb1c4fa9957

Setup

The kafka-*.sh tools take the broker list from the KAFKA_BOOTSTRAP env var on the runner host, which every action passes through as --bootstrap-server.

Environment

Set these on the runner host, then add each name to execution.inherit_env so the value reaches the action.

KAFKA_BOOTSTRAP required

Comma-separated bootstrap broker list (host:port).

Notes

KAFKA_BOOTSTRAP only reaches an action when the runner allowlists it in execution.inherit_env — the action env is scrubbed to PATH/LANG/LC_ALL/TERM by default. There is NO fallback: unset, every action fails to reach a broker, and the error names a connection problem rather than the missing allowlist entry.
These actions pass only --bootstrap-server — none reference a --command-config / KAFKA_COMMAND_CONFIG properties file, so as written they assume a listener reachable without per-client SASL/TLS credentials.
Mutators (reset_offsets_*, alter_topic_retention, delete_consumer_group, preferred_leader_election) need a principal with cluster/topic/group Alter rights if the cluster enforces ACLs.

Verify it works

Runs kafka.cluster_info, a low-risk read that confirms the pack can reach its target. Run it on the host once the pack is installed; pack install runs it for you.

on the runner host

sudo emisar pack verify kafka

Install and configure a pack walks through the whole sequence on a host.

Actions 20 total

View on GitHub

kafka.alter_topic_retention exec high

Set topic retention.ms

Set `retention.ms` on a topic. Lower values cause old data to be deleted in the next log roll.
View source on GitHub
kafka.broker_api_versions exec low

Broker API versions

Show the API protocol versions supported by each broker. Useful to confirm rolling upgrade.
View source on GitHub
kafka.broker_configs exec low

Broker configs

List all broker-level configs (dynamic + static + defaults).
View source on GitHub
kafka.cluster_info exec low

Cluster metadata

Show broker IDs, controller, cluster ID. Read-only.
View source on GitHub
kafka.consumer_lag exec low

Consumer lag for a group

Show per-partition lag (log-end-offset minus committed offset) for one consumer group. The canonical "is consumption keeping up?" check.
View source on GitHub
kafka.delete_consumer_group exec high

Delete consumer group

Remove a consumer group from the coordinator; its committed offsets are permanently dropped, so a re-created group restarts from auto.offset.reset. The group must be empty (no active members).
View source on GitHub
kafka.describe_group exec low

Describe consumer group

Show member list, partition assignment, host for one consumer group.
View source on GitHub
kafka.describe_topic exec low

Describe topic

Show partition layout, leaders, ISR, replicas for one topic.
View source on GitHub
kafka.leader_partitions exec low

Partition leaders by broker

Count partitions where each broker is leader. Uneven distribution → preferred-leader election.
View source on GitHub
kafka.list_acls exec low

List ACLs

List every ACL binding on the cluster.
View source on GitHub
kafka.list_consumer_offsets exec low

Committed offsets per partition

Show the same view as consumer_lag but emphasizes committed-offset reading. Useful before reset_offsets.
View source on GitHub
kafka.list_groups exec low

List consumer groups

List all consumer groups in the cluster.
View source on GitHub
kafka.list_topics exec low

List topics

List all topics in the cluster.
View source on GitHub
kafka.preferred_leader_election exec high

Trigger preferred-leader election

Force preferred-leader election cluster-wide. Use after a broker restart leaves leaders skewed.
View source on GitHub
kafka.reassign_status exec low

Partition reassignment status

Show the status of any in-flight partition reassignments.
View source on GitHub
kafka.reset_offsets_to_earliest exec critical

Reset group offsets to earliest

Rewind a consumer group's offsets to the start of every partition. Causes a full replay — can be hours to days of duplicate processing.
View source on GitHub
kafka.reset_offsets_to_latest exec critical

Reset group offsets to latest (skip all)

Fast-forward a consumer group past every existing message — they will NOT be processed. Data loss for that consumer.
View source on GitHub
kafka.topic_configs exec low

Topic configs

List topic-level overrides (retention, compaction, segment size).
View source on GitHub
kafka.unavailable_partitions exec low

Unavailable partitions

List partitions with no leader. These mean writes are failing.
View source on GitHub
kafka.under_replicated exec low

Under-replicated partitions

List topics + partitions with ISR < replication factor. The canonical "is replication healthy?" check.
View source on GitHub

← Back to registry

Pack reference Publish your own