Kafka recap

  1. Consumer group is a logical name of consumers. Messages consumption is load balanced by all consumers under one group.
  2. Topic is a logical name of data feed. Producer publishes and consumer consumers messages in terms of topic.
  3. Messages are stored in partitions. Partition has replicas. One partition is elected as leader to serve request.
  4. Kafka cluster has the ability to enforce quotas of producer and consumer in case saturate one node’s resource.
  5. Producer delivery guarantee. 1). Async. Least latency, no need to wait for message being committed. 2) wait for message committed to the leader. 3) wait for message committed to leader and all followers.
  6. Consumer delivery guarantee. 1) at least once 2) at most once 3) exactly once. transaction commit(2 phase commit).

reference: https://kafka.apache.org/documentation/#semantics