The Kafka Helm chart provides an easy way to deploy Apache Kafka, a distributed event streaming platform. This chart allows you to manage Kafka instances on Kubernetes with customizable resource allocation, persistence, and scaling options. It includes built-in monitoring with Kafka Exporter and Prometheus alerts.

Dependencies

Before installing the chart, you need to download the required dependencies. Run the following command in the chart directory:

This command will:

1. Read the dependencies from Chart.yaml

2. Download the required charts (ZooKeeper) from the specified repositories

3. Store them in the charts/ directory

4. Create or update the Chart.lock file with the exact versions

If you encounter any issues with the dependencies, you can try:

This chart requires the following dependencies to be installed:

ZooKeeper

• Chart: zookeeper

• Version: 0.0.1

• Repository: https://helm.zop.dev

• Condition: zookeeper.enabled

• Purpose: Provides distributed coordination and configuration management for Kafka

To install this dependency automatically, ensure the following in your values.yaml:

The dependency will be automatically installed when you deploy the Kafka chart. You can customize its configuration through the respective section in your values.yaml file.

Add Helm Repository

Before deploying the Kafka chart, add the Helm repository to your local setup:

For more details, refer to the Helm Repository Documentation.

Install Helm Chart

To install the Kafka Helm chart, use the following command:

helm install [RELEASE_NAME]

Replace [RELEASE_NAME] with your desired release name. For example:

To customize configurations, provide a values.yaml file or override values via the command line.

See Helm Install Documentation for more information.

Uninstall Helm Chart

To remove the Kafka deployment and all associated Kubernetes resources, use the following command:

helm uninstall [RELEASE_NAME]

For example:

See Helm Uninstall Documentation for additional details.

Configuration

The Kafka Helm chart includes several configuration options to tailor the deployment to your needs. Below is a summary of the key configurations:

Default Kafka Configuration

The chart includes several pre-configured Kafka settings:

• Replication Factor: 3

• Number of Partitions: 3

• Min In-Sync Replicas: 2

• Log Retention: 168 hours (7 days)

• Log Segment Size: 1GB

• Message Max Bytes: ~1MB

• Auto Create Topics: Disabled

• Delete Topic: Enabled

  
  

Example values.yaml File

To use this configuration, save it to a values.yaml file and apply it during installation:

Features

• High Availability: Deploy a 3-node Kafka cluster with proper replication and fault tolerance.

• Built-in Monitoring: Includes Kafka Exporter for Prometheus metrics collection.

• Comprehensive Alerts: Pre-configured Prometheus alerts.

• Security:

  • Read-only root filesystem

  • Non-root user execution

  • Dropped capabilities

  • Configurable security protocols

• Resource Management:

  • Configurable CPU and memory limits

  • Persistent volume storage

  • JVM heap optimization

• Networking:

  • Internal and external listeners

  • Headless service for pod discovery

  • Service monitor for Prometheus integration

• Operational Features:

  • Rolling updates with configurable strategy

  • Pod disruption budget

  • Parallel pod management

  • Configurable pod affinity

Monitoring

The chart includes a Kafka Exporter container that exposes metrics on port 2121. These metrics are automatically collected by Prometheus when using the Prometheus Operator.

Key metrics include:

• Broker status

• Topic and partition information

• Consumer group lag

• Replication status

• JVM metrics

Contributing

We welcome contributions to improve this Helm chart. Please refer to the CONTRIBUTING.md file for contribution guidelines.

Code of Conduct

To maintain a healthy and collaborative community, please adhere to our Code of Conduct.

License

This project is licensed under the LICENSE. Please review it for terms of use.