Lesson 3: Building a Big Data Infrastructure Part 3

Real-time Processing & Messaging

Messaging is Good For

  1. Long Running Tasks

  2. Managing Bursty Events

  3. Decoupling Services

  4. Denormalizing or Copying Data

Messaging Semantics

  1. Producers or Publishers

  2. Brokers, Queues

  3. Consumers or Workers

  4. First in, First out

Kafka

  1. Distributed - Brokers, Producers, Consumers

  2. Persistent

  3. High Throughput

  4. Queues through Topics

  5. Open Source - Built at LinkedIn

Other Messaging Systems

  1. RabbitMQ

  2. Kestrel

  3. Redis

  4. ZeroMQ

On to the install...

← →

/

#