June 12, 2024 6:00 PM PDT
The meeting focused on the system design for a real-time commenting feature in a newsfeed application. Key discussions included functional and non-functional requirements, data entities, API design, and high-level architectural choices. The goal is to create a scalable and efficient system capable of handling a high volume of comments in real-time.
Functional Requirements
- Feed Characteristics:
- Real-time display of comments
- Show the 10 latest comments
Non-Functional Requirements
- Scalability:
- Initial scope to support 1 million reads per minute
- Targeting 100 million users with 20 million active users
- Each post can have up to 10 comments
- Each comment is approximately 100 bytes
- Expecting 20 comments per day per post
- High Availability
- High Throughput
- Low Latency
API Design
- POST
/comment/create- Parameters:
feedId,comment
- Parameters:
- GET
/comments/feedId- Returns a list of historical comments
Data Entities
- Comment:
- Attributes:
comment ID,feed ID,userId,comment
- Attributes:
- PostMetaData:
- Attributes:
ID,userId
- Attributes:
- User:
- Attributes:
userId
- Attributes:
High-Level Design Considerations
- Both SQL and NoSQL databases are viable options due to the low number of comments.
- NoSQL may be preferred for high throughput requirements.
- A queue system may be necessary to manage comment submissions and retrieval.
Real-Time Comment Retrieval
- Polling Mechanism:
- Comments can be polled to retrieve updates in real-time.
- Handling High Read Throughput:
- Separate read and write operations to optimize performance.
- Implement a real-time service that subscribes to a queue for comment updates.
Queue Implementation
- Technologies:
- Options include Kafka or Redis for managing comment queues.
- Topic Management:
- Users can be hashed to specific topics to manage load.
- Consider partitioning topics based on user ID or post ID for better distribution.
Kafka Overview
- Kafka supports multiple readers for topics, enhancing scalability and performance in handling real-time comments.
This structured format provides a clear understanding of the discussions and decisions made during the meeting regarding the real-time commenting feature for the newsfeed application.