Kafka

Apache Kafka is a distributed event streaming system built around durable logs and consumer groups. It is commonly used for asynchronous service-to-service communication, ingest pipelines, audit streams, and other workloads where multiple consumers need to process the same sequence of events.

On Viduli, Kafka is available through Surge as a managed broker resource. You choose the cluster layout and Kafka settings in the console, and Viduli handles provisioning, lifecycle actions, and the cluster management surface around the resource.

Launching a New Kafka Cluster

To launch a Kafka cluster:

1. Open your project overview page.
2. Click New Resource.
3. Select Surge from the resource creation dialog.
4. Choose Apache Kafka.
5. Configure the cluster layout, node resources, region, topic defaults, and other Kafka settings. For a field-by-field reference, see the configuration guide.
6. Review the cost estimate and click Confirm.

Provisioning starts immediately after creation. The cluster enters a launching state while Viduli provisions the broker resource and applies the selected configuration.

Starting a Kafka Cluster

If a Kafka cluster is stopped, you can start it again from the project overview or from the resource details view.

To start a cluster:

1. Select the Kafka resource in your project overview.
2. Confirm the resource controls are visible.
3. Click the start icon.
4. Wait for the cluster to return to a running state.

Stopping a Kafka Cluster

Stopping a Kafka cluster takes the broker resource offline while preserving the resource itself.

To stop a cluster:

1. Select the Kafka resource in your project overview.
2. Confirm the resource controls are visible.
3. Click the stop icon.
4. Wait for the shutdown action to complete.

Deleting a Kafka Cluster

Deleting a Kafka cluster permanently removes the resource and its data.

To delete a cluster:

1. Select the Kafka resource in your project overview.
2. Open the resource controls.
3. Click the delete icon.
4. Enter the resource name in the confirmation dialog.
5. Click Confirm.

Configuring a Kafka Cluster

Kafka configuration is available in the resource details view under the Configuration tab.

Use this tab when you need to:

• review the current node layout
• change supported CPU, memory, disk, or node-count settings
• adjust topic defaults
• tune leader behavior
• tune Kafka process settings

Some settings are fixed after creation, including the engine, Kafka version, deployment region, replication mode, and node layout. See the configuration guide for details.

Managing Topics

Kafka clusters include a dedicated Topics tab in the resource details view. From there you can list, create, update, and delete topics for the selected cluster.

For the full workflow and a field reference, see Kafka topic management.

Cluster Details View

The Kafka resource details view can be opened from the project overview. For the shared behavior of resource tabs, see Resources.

Kafka clusters expose these tabs:

• Instances
• Monitoring
• Logs
• Shell
• Actions
• Environment
• Configuration
• Topics
• Costs

Monitoring

See resource monitoring.

Logs

See resource logs.

Shell

See resource shell access.

Actions

See resource actions.

Environment

See resource environment.

Costs

See resource costs.