Design Auction System

March 26, 2023 7:00 PM PDT

---

This document outlines the design considerations and technical discussions related to an auction system. The focus is on the architecture, data flow, and database management for handling bids, auctions, and seller interactions efficiently.

System Design Interview Notes

Key Concepts

• QPS (Queries Per Second)
  • Query QPS
  • Bidding QPS / Writing QPS

High-Level Flow

1. Seller Actions

   • Post auction details:
     • Endpoint: v1/enlist
     • Parameters: productID, quantity, lowestprice, seller
   • Create auction:
     • Endpoint: v1/createauction

2. Bidding Side

   • WebSocket connection to the bid agent.
   • Use of Kafka queue for managing bids.
   • Streaming service (Flink) for calculating the top price.

Technical Components

• Event Bus

  • Options: Amazon Lambda, Google Guava

• Data Management

  • Redis for storing the top price.
  • Two Kafka queues to ensure exactly-once processing.

• API Endpoints

  • Post bid:
    • Endpoint: v1/bid
    • Process:
      • Bid is sent to Kafka.
      • Streaming service compares with the top price.
      • If the bid is higher, it is placed into Kafka.

Database Design

• Tables

  • Product table
  • Inventory table
  • Auction table

• Database Type

  • Wide-column NoSQL database:
    • Advantages:
      • Minimal joins
      • No need for transactions
      • LSM Tree database for faster writes

• Data Handling

  • Support for distributed transactions.
  • Cache write-back mechanism to ensure data consistency.
  • Append-only database structure.

Performance Considerations

• Handling High Throughput
  • Use of HDFS for data storage.
  • Sharding by auction ID to manage load.
  • Archiving cold data.

Discussion Points

• Consumer Requirements

  • Need a consumer to read from Redis.

• Alternatives and Trade-offs

  • Kafka supports persistence but requires calculations.
  • Redis can handle top-K but is more complex for persistence.

• Idempotency and Two-Phase Commit

  • Importance of ensuring "exactly once" processing for bids.
  • Use of an idempotency key in the first Kafka queue.
  • Need for a two-phase commit to handle bid processing reliably.

• Bid Management

  • Considerations for handling simultaneous bids.
  • Use of Redis for sequentializing bids.
  • Sharding Redis by auction ID to manage high QPS.

Design Options

1. Handling Highest Price Only
2. Handling Top-K Bids
   • Redis is single-threaded; consider implications for multiple instances.
   • Key design should focus on auction ID and product ID.

Conclusion

The auction system design emphasizes scalability, reliability, and efficient data management. Key architectural decisions revolve around the use of NoSQL databases, streaming services, and message queues to ensure high performance and consistency in bid processing.