Event Streaming Recommendations for Clients/Tenants

This page contains our recommendations for various client configurations.

Category 1: A load not greater than 150k events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 10
   • max_uncommitted_events: 100
   • batch_flush_timeout: 2 seconds
   • commit_timeout: 5 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 50
   • max_uncommitted_events: 500
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
3. Balanced approach
   • batch_limit: 20
   • max_uncommitted_events: 200
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 20 seconds
   • Recommended consumers: 3 (1 per partition)

Category 2: A load between 150k and 350k events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 15
   • max_uncommitted_events: 150
   • batch_flush_timeout: 3 seconds
   • commit_timeout: 10 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 100
   • max_uncommitted_events: 1000
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 90 seconds
   • stream_timeout: 70 minutes Recommended consumers: 3 (1 per partition)
3. Balanced approach
   • batch_limit: 50
   • max_uncommitted_events: 500
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 30 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)

Category 3: A load between 350k and 550k events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 20
   • max_uncommitted_events: 200
   • batch_flush_timeout: 2 seconds
   • commit_timeout: 5 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 100
   • max_uncommitted_events: 1500
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)
3. Balanced approach
   • batch_limit: 50
   • max_uncommitted_events: 1000
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 30 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 3 (1 per partition)

Category 4: A load between 750k and 1M events/notifications per 40 minute

1. Real-time low-latency processing
   • batch_limit: 25
   • max_uncommitted_events: 300
   • batch_flush_timeout: 3 seconds
   • commit_timeout: 10 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 6 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 200
   • max_uncommitted_events: 3000
   • batch_flush_timeout: 15 seconds
   • commit_timeout: 120 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 6 (1 per partition)
3. Balanced approach
   • batch_limit: 100
   • max_uncommitted_events: 2000
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 6 (1 per partition)

Category 5: A load between 1M and 2M events/notifications per 40 minutes

1. Real-time low-latency processing
   • batch_limit: 50
   • max_uncommitted_events: 500
   • batch_flush_timeout: 5 seconds
   • commit_timeout: 10 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 8 (1 per partition)
2. High-throughput, less frequent processing
   • batch_limit: 300
   • max_uncommitted_events: 5000
   • batch_flush_timeout: 20 seconds
   • commit_timeout: 180 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 8 (1 per partition)
3. Balanced approach
   • batch_limit: 150
   • max_uncommitted_events: 3000
   • batch_flush_timeout: 10 seconds
   • commit_timeout: 60 seconds
   • stream_timeout: 70 minutes
   • Recommended consumers: 8 (1 per partition)