How do I test a Kafka consumer without running a broker?

Record your application once against a real Kafka with Keploy. It captures the fetch responses and offset commits the broker sent, then replays them in order during tests — so the consumer processes real records with no broker and no poll timeout.

Is Keploy an EmbeddedKafka alternative for Spring Boot?

For replay, yes. EmbeddedKafka starts a broker per test class, adding election and topic-creation time to every run. Keploy records a real broker once and replays its responses deterministically, with no broker in CI at all.

Does Keploy test Kafka message serialization?

Yes. Keploy captures the produce request after your serialiser has run, so the recorded value is the actual bytes on the wire. A misconfigured serialiser or wrong partition key shows up as a diff instead of passing a MockProducer assertion.

Can Keploy make Kafka tests less flaky?

Yes, because replay removes the asynchrony. Recorded exchanges are served in the order they were captured, so there is no polling, no timeout to tune, and no consumer group rebalance to race — the same code produces the same result every run.

Does Keploy work with kafkajs, sarama, and confluent-kafka?

Yes, and identically. Capture happens at the Kafka wire protocol beneath the client, so the Java client, kafkajs, sarama, kafka-go, and confluent-kafka all record and replay the same broker exchanges.

Does Keploy support Kafka Streams and Connect?

Not yet. Plain producers and consumers are fully supported, and Schema Registry lookups are captured as HTTP. Kafka Streams internal topics and Kafka Connect worker traffic are on the roadmap rather than supported today.

Keploy as a Apache Kafka testing framework

Keploy tests Kafka producers and consumers by recording the broker protocol beneath your client, capturing produce acknowledgements, fetch responses, and offset commits, then replaying them during tests. No broker or ZooKeeper runs in CI and no embedded cluster is started.

Generate Apache Kafka tests free

keploy record -c "java -jar orders.jar"

18.4K+

1.2M+300M+mocks createdSlack

What Keploy gives a Apache Kafka team

Captured at the Kafka broker protocol

Keploy reads the protocol beneath your client, recording produce requests with the broker's acknowledgement and fetches with the records returned and the offset committed.

Below KafkaProducer, kafkajs, sarama, confluent-kafka
Produce acknowledgements with real offsets
Fetch responses and offset commits
Partition assignment preserved

Try Keploy for free Explore the docs

The problem

Why Apache Kafka integration tests slow teams down

The friction is rarely the assertions. It is everything around them — spinning up dependencies, keeping mocks honest, and repairing tests after every refactor.

Book a call Explore the docs

Three ways to do it

Testing Apache Kafka: by hand, with EmbeddedKafka / Testcontainers, or with Keploy

An embedded or containerised broker gives real Kafka semantics and makes every test timing-dependent, because it must poll for messages that arrive asynchronously. Replay removes the asynchrony.

By hand

EmbeddedKafka / Testcontainers

With Keploy

Select any row for the full comparison, with code.

Get started Book a call

Same coverage, three costs

What you write for Apache Kafka vs what Keploy records

All three produce the same assertion. Only the third still passes after the next refactor without anyone editing it.

By hand2–4 hours

OrderEventsFakeTest.java

hand-written

// Hand-written fake: nothing is serialised or partitioned.

package com.acme.orders;

import java.util.*;

import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

class FakeEventPublisher implements EventPublisher {

final List<Object> published = new ArrayList<>();

@Override

public void publish(String topic, Object event) {

// No serialiser runs. No partition is chosen. No ack returns.

published.add(event);

}

class OrderEventsFakeTest {

@Test

void publishesAConfirmedEvent() {

var publisher = new FakeEventPublisher();

new OrderService(publisher).confirm("ord_1");

assertEquals(1, publisher.published.size());

}

This asserts that a method was called. A serialiser misconfiguration or a wrong partition key would pass this test and break in production.

EmbeddedKafka / Testcontainers1–2 hours

OrderEventsEmbeddedTest.java

tool-assisted

// EmbeddedKafka: a real broker, started per test class.

package com.acme.orders;

import java.time.Duration;

import org.junit.jupiter.api.Test;

import org.springframework.kafka.test.context.EmbeddedKafka;

@EmbeddedKafka(partitions = 1, topics = "orders.confirmed")

class OrderEventsEmbeddedTest {

@Test

void publishesAConfirmedEvent() {

service.confirm("ord_1");

// Poll until it shows up, or give up. This is the flaky part.

var records = consumer.poll(Duration.ofSeconds(10));

assertEquals(1, records.count());

var record = records.iterator().next();

assertEquals("ord_1", record.key());

}

Real serialisation and real partitions, at the cost of a broker start per class and a ten-second poll window that turns CI load into test failures.

With Keploy~5 minutes

mocks.yaml

auto-generated

# Recorded with: keploy record -c 'java -jar orders.jar'

# The broker exchange, captured at the wire protocol.

version: api.keploy.io/v1beta1

kind: Kafka

name: mock-5

spec:

produce:

topic: orders.confirmed

partition: 0

key: "ord_1"

# The bytes your configured serialiser actually produced.

value_serialized: "eyJpZCI6Im9yZF8xIiwic3RhdGUiOiJDT05GSVJNRUQifQ=="

headers:

content-type: "application/json"

# The real acknowledgement from the broker.

response:

base_offset: 4127

error_code: 0

fetch:

topic: orders.confirmed

partition: 0

committed_offset: 4128

The serialised bytes and the broker's real offset acknowledgement are both recorded, so serialiser and partition-key bugs surface as diffs — with no polling and no timeout.

Times are estimates for authoring one endpoint’s coverage from scratch, not measurements.

Explore the docs Request a trial

Keploy vs the alternatives

Apache Kafka testing tools, compared

The options a team on message brokers actually reaches for, and where each one genuinely wins. Select a row for the full comparison.

Keploy

Testcontainers

Embedded Kafka

MockConsumer / MockProducer

LocalStack

Postman

SmartBear ReadyAPI

Best in classStrongPartialNot covered

Assessments reflect each tool’s documented behaviour, not benchmark measurements.

Request a trial Book a call

How it works

Record your Apache Kafka app once, replay it forever

Keploy sits below your Apache Kafka process at the network layer. It watches the calls your app already makes, then serves them back on replay so tests run with no dependencies attached.

Your Apache Kafka appGET/api/v1/orders/{id}200

RecordingKeploy proxyeBPF · userspace

JavaKafkaProducer, KafkaConsumer, and Spring Kafka

Node.jskafkajs

Pythonkafka-python and confluent-kafka-python

Gosarama and segmentio/kafka-go

+2 more Apache Kafka clients

An example shape of a captured call. Your own endpoints and dependencies come from your real traffic, so nothing here has to be written by hand.

Try Keploy for free Request a trial

Mock coverage

Apache Kafka clients Keploy records, driver by driver

Keploy captures Apache Kafka at the wire protocol, so 5 of these 8 clients need no adapter, no test double, and no Apache Kafka instance in CI.

Auto-mocked
JavaView page
KafkaProducer, KafkaConsumer, and Spring Kafka
Auto-mocked
Node.jsView page
kafkajs
Auto-mocked
PythonView page
kafka-python and confluent-kafka-python
Auto-mocked
GoView page
sarama and segmentio/kafka-go
Auto-mocked
Spring BootView page
@KafkaListener and KafkaTemplate
Partial
Schema Registry
Avro and Protobuf schema lookups over HTTP
On the roadmap
Kafka Streams
Internal topic and changelog traffic
On the roadmap
Kafka Connect
Connector REST API and worker traffic

Explore the docs Book a call

Quick start

Your first Apache Kafka test suite in under five minutes

Every command below runs against your existing Apache Kafka service. Nothing in your source tree changes.

1
1Install the Keploy CLI
One binary on the recording machine. Nothing is installed into the broker and no client wrapper is needed.
curl -sSL https://keploy.io/install.sh | bash
2
2Record with your development broker reachable
Run the app as usual. Produce requests, acknowledgements, fetches, and offset commits are all captured in order.
keploy record -c "java -jar orders.jar"
3
3Exercise both sides of the topic
Trigger the endpoint that produces, then let your consumer process the message so both halves are recorded.
curl -X POST localhost:8080/orders/ord_1/confirm -d '{"paymentToken":"tok_123"}'
4
4Replay with the broker stopped
Shut Kafka down and run the suite. Replay is deterministic, so there is no poll timeout to tune.
keploy test -c "java -jar orders.jar" --delay 20