April 14, 2024 7:00 PM PDT
This presentation provides a comprehensive overview of advertising systems, discussing their importance, use cases, engineering design, and challenges faced in the industry. It aims to educate the audience on the fundamental aspects of ads systems, including their architecture and operational considerations.
Presenter: L, Tech Lead at FAANG
Topics Covered
Importance of Ads Systems
- Connects products with consumers.
- Contributes significantly to the economy (e.g., 2% of US GDP).
- Major revenue source for companies (e.g., 76% of Google’s gross revenue).
- Rapid growth in advertising revenues (e.g., 27% year-over-year growth for Amazon ads).
Use Cases of Ads Systems
- Users encounter ads during website visits, seeking an enjoyable experience while minimizing ad exposure.
- Business owners aim to deliver targeted ads based on demographics and interests, manage budgets, and track ad performance.
- Platform owners focus on increasing demand from advertisers and supply from active users.
Simplified Flow of Ads System
- Business owner creates an ad and adds it to the ads inventory.
- Ads platform indexes the ads inventory.
- User visits a website that loads ads from the ads ranking system.
- The website sends user activity data (e.g., ad impressions, user ID, ad ID, placement).
- User interacts with the ad, and the website reports the action back to the platform.
- Business owner receives reports on impressions and purchases, allowing budget adjustments based on feedback.
Engineering Design
- Requirements: Strong consistency, durability, scalability (millions of ads).
- Solutions:
- Use of relational databases for strong consistency.
- Sharding for availability.
- Replication for durability.
- Goal: Increase speed for retrieving ads.
- Techniques:
- Skip list and B+ tree in relational databases.
- LSM tree in key-value databases.
- Inverted index in document search systems to map keywords to ad IDs.
- Objective: Select 30 relevant ads from millions of inventory quickly.
- Solutions:
- 1st layer ranking reduces millions of ads to thousands.
- 2nd layer ranking narrows down to hundreds.
- A heavier model selects the final 30 ads.
- Goal: Ensure high availability and end-to-end consistency.
- Solutions:
- Asynchronous actions for high availability.
- Idempotent operations to ensure exactly once processing.
- Use of message queues for exactly once delivery.
- Global unique IDs for deduplication.
Measurement Service
- Aggregation reporting of ad impressions and actions.
- Supports near real-time reporting using message queues and streaming technologies.
- Sharding to manage workload and batch actions.
Challenges
- Error handling and infrastructure resilience.
- Privacy challenges due to platform and regulatory limitations.
- Impact of third-party data restrictions on tracking and targeting.
Audience Questions
- Discussion on the impact of cross-app tracking limitations on ad tracking.
- Insights on billing system latency and sharding strategies.
- Clarification on the role of memory and databases in maintaining consistency.
- Challenges related to user and ad interactions, including auto-scaling and budget management.
Recommendations
- Emphasize the need for big data solutions and robust billing systems to handle financial transactions with high throughput and exact once guarantees.