- 1 year ago
Configuring Kafka with jmx-exporter in windows
Configuring Kafka with jmx-exporter
on Windows involves several steps. jmx-exporter
is used to export JMX metrics from Kafka so that they can be scraped by a Prometheus server for monitoring purposes. Here's a step-by-step guide to configure jmx-exporter
with Kafka on Windows:
Download and Install jmx-exporter:
Download the
jmx-exporter
JAR file from the official GitHub repository: https://github.com/prometheus/jmx_exporter/releasesCreate a directory for
jmx-exporter
, and place the downloaded JAR file in that directory.
Configure jmx-exporter YAML file:
Create a new YAML configuration file (e.g.,
kafka-jmx-config.yaml
) forjmx-exporter
with the following content:yamlstartDelaySeconds: 0
rules:
- pattern: 'kafka.(\S+):type=(\S+),name=(\S+)'
name: kafka_$1_$2
help: Kafka $1 $2
type: GAUGE
attrNameSnakeCase: true
This configuration tells
jmx-exporter
to export Kafka JMX metrics with the specified pattern.Configure Kafka to Enable JMX Exporter:
Open the
kafka-run-class.bat
script located in thebin
directory of your Kafka installation.Modify the
set KAFKA_JMX_OPTS=...
line to include the following options:batchset KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.rmi.port=9999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
This enables JMX on port 9999 without SSL or authentication.
Start Kafka Server:
- Start your Kafka server using the modified
kafka-run-class.bat
script.
- Start your Kafka server using the modified
Start jmx-exporter:
Open a command prompt and navigate to the directory where you placed the
jmx-exporter
JAR file and the configuration file.Run the following command to start
jmx-exporter
:shelljava -javaagent:./jmx_prometheus_javaagent-X.X.X.jar=9999:./kafka-jmx-config.yaml -jar jmx_prometheus_javaagent-X.X.X.jar 9999 kafka-jmx-config.yaml
Replace
X.X.X
with the version number of thejmx-exporter
JAR file.Verify Metrics:
- Open your web browser and navigate to
http://localhost:9999/metrics
. You should see the exported Kafka metrics in Prometheus format.
- Open your web browser and navigate to
Scrape Metrics with Prometheus:
- If you haven't already, install and configure Prometheus to scrape the Kafka metrics from
jmx-exporter
. Refer to the Prometheus documentation for installation and configuration steps.
- If you haven't already, install and configure Prometheus to scrape the Kafka metrics from
Visualize Metrics:
- Use Grafana or other monitoring tools to visualize the scraped Kafka metrics.
That's it! You should now have Kafka metrics exported and available for monitoring using jmx-exporter
and Prometheus on Windows.