Event Streaming Recommendations for Clients/Tenants

This page contains our recommendations for various client configurations. These recommendations are unchanged from V1 — the underlying query parameters and consumer-per-partition model they're based on are the same in V2.

Category 1: A load not greater than 150k events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 10
   • batch_flush_timeout: 2 seconds
   • commit_timeout: 5 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 50
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
3. Balanced approach
   • batch_limit: 20
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 20 seconds
   • Recommended consumers: 3 (1 per partition)

Category 2: A load between 150k and 350k events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 15
   • batch_flush_timeout: 3 seconds
   • commit_timeout: 10 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 100
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 90 seconds
   • stream_timeout: 70 minutes Recommended consumers: 3 (1 per partition)
3. Balanced approach
   • batch_limit: 50
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 30 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)

Category 3: A load between 350k and 550k events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 20
   • batch_flush_timeout: 2 seconds
   • commit_timeout: 5 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 100
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
3. Balanced approach
   • batch_limit: 50
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 30 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)

Category 4: A load between 750k and 1M events/notifications per 40 minute

1. Real-time low-latency processing
   • batch_limit: 25
   • batch_flush_timeout: 3 seconds
   • commit_timeout: 10 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 6 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 200
   • batch_flush_timeout: 15 seconds
   • commit_timeout: 120 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 6 (1 per partition)
3. Balanced approach
   • batch_limit: 100
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 6 (1 per partition)

Category 5: A load between 1M and 2M events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 50
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 10 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 8 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 300
   • batch_flush_timeout: 20 seconds
   • commit_timeout: 180 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 8 (1 per partition)
3. Balanced approach
   • batch_limit: 150
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 8 (1 per partition)

note

V2's fixed consumer-per-subscription model (typically 4, 6, or 8 consumers, no longer dependent on event type count) may shift which of these recommendations applies to a given subscription — see Differences Between Streaming API V1 and V2.