Skip to main content
Quick navigation

How about the throughput impact?

View the full demo in realtime​

You can either follow all the steps manually, or watch the recording

Review the docker compose environment​

As can be seen from docker-compose.yaml the demo environment consists of the following services:

  • gateway1
  • gateway2
  • kafka-client
  • kafka1
  • kafka2
  • kafka3
  • schema-registry
cat docker-compose.yaml

Starting the docker environment​

Start all your docker processes, wait for them to be up and ready, then run in background

  • --wait: Wait for services to be running|healthy. Implies detached mode.
  • --detach: Detached mode: Run containers in the background
docker compose up --detach --wait

Creating topic physical-kafka on kafka1​

Creating on kafka1:

  • Topic physical-kafka with partitions:10 and replication-factor:1
kafka-topics \
    --bootstrap-server localhost:9092,localhost:9093,localhost:9094 \
    --replication-factor 1 \
    --partitions 10 \
    --create --if-not-exists \
    --topic physical-kafka

Let's use kafka-producer-perf-test that comes bundled with Kafka​

throughput is set to -1 to disable throttling and create the maximum pain

kafka-producer-perf-test \
    --topic physical-kafka \
    --throughput -1 \
    --num-records 2500000 \
    --record-size 255 \
    --producer-props bootstrap.servers=localhost:9092,localhost:9093,localhost:9094

Creating topic via-gateway on gateway1​

Creating on gateway1:

  • Topic via-gateway with partitions:10 and replication-factor:1
kafka-topics \
    --bootstrap-server localhost:6969 \
    --replication-factor 1 \
    --partitions 10 \
    --create --if-not-exists \
    --topic via-gateway

Let's use kafka-producer-perf-test that comes bundled with Kafka​

throughput is set to -1 to disable throttling and create the maximum pain

kafka-producer-perf-test \
    --topic via-gateway \
    --throughput -1 \
    --num-records 2500000 \
    --record-size 255 \
    --producer-props bootstrap.servers=localhost:6969 \
    --producer.config ${KAFKA_CONFIG_FILE}

Tearing down the docker environment​

Remove all your docker processes and associated volumes

  • --volumes: Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers.
docker compose down --volumes

Conclusion

Gateway is fast enough for all use cases!