Machine learning for Facebook Newsfeed

April 17, 2022 7:00 PM PDT

---

This document summarizes a mock system design interview focused on machine learning applications for the Facebook Newsfeed. The discussion covered various aspects of system requirements, model architecture, and challenges associated with implementing a recommendation system. The interview aimed to assess the candidate's understanding of machine learning concepts, system design, and practical applications in a social media context.

Interview Details

• Topic: Machine Learning for Facebook Newsfeed
• Level: L5 (Senior)
• Duration: 45 minutes
• Drawing Tool Used: Excalidraw

Requirements

Functional Requirements

• FB Newsfeed:
  • No queries
  • No ads

Problem Formation

• Improve relevance by recommending diverse content such as images, videos, texts, and live streams.
• Candidate generation to return relevant items.

Data and Metrics

• Labels: Clicks, likes, comments, shares, reposts.
• De-noising of Labels:
  • Offline Metrics: Mean Average Precision (MAP), Area Under Curve (AUC), F1 Score, Mean Reciprocal Rank (MRR).
  • Online Metrics: Number of clicks, comments, shares, and reposts per user session.

Feature Engineering

• User Features: User ID, preferences/tags, user embedding.
• Author Features: Author ID, preferences/tags, embedding.
• Newsfeed Features: Newsfeed ID, content type (text, image, video), engagement metrics over various timeframes.
• Context Features: Time of day, day of the week, device, holidays.
• Cross Features: Similarity between user and newsfeed, user and author.

Model Architecture Possibilities

Candidate Generation

• Two-Tower Model:
  • Dense model to create embeddings.
  • Minimize loss by ensuring positive pairs of user and item are ranked higher than unrelated pairs.

Ranking Models

• Logistic Regression: Limited in processing categorical features.
• Gradient Boosted Decision Trees (GBDT)/Neural Networks: For preprocessing inputs to logistic regression.
• Wide and Deep Model: Loses sequence information.
• Deep Interest Model: Improves upon wide and deep by preserving sequence information.

Challenges

• Caveats: Positional bias, diversity, cold start problem.
• Probability of Clicking: Influenced by relevance and position rank.
• Diversity Management: Downrank multiple items from the same author to avoid over-representation.

Training and Evaluation

• Train/Test Split: Use a month of data, with the first three weeks for training.
• Cross Validation: Random sampling for negative cases to improve precision.
• A/B Testing: Random assignment and partition strategy for social network division.

Monitoring and Retraining

• Metrics for online and offline performance. Frequency of retraining based on requirements and capacities.

Cold Start Solutions

• Multi-Armed Bandit Approach: Provide initial items to users and reward based on usage.
• Feature Sharing: Train models using features shared between mature and cold start items.

System Design Considerations

• Key Algorithms: Collaborative filtering, semantic-based filtering.
• Performance Measurement: Likes, comments, shares, reposts, and metadata analysis.

Interviewer and Audience Feedback

Interviewer Comments

• Clear outline but some solutions were overly complex.
• Emphasis on high-level system design rather than deep dives into complex solutions.
• Suggested more research on the product and its specific features.

Audience Insights

• Highlighted the importance of understanding the cold start problem and its implications.
• Noted the need for a clear connection between the ML system and larger business goals, such as active users and click-through rates.

Conclusion

The interview provided valuable insights into the candidate's approach to machine learning system design for Facebook's Newsfeed. It highlighted the importance of balancing complexity with clarity and the need for practical solutions to real-world challenges in recommendation systems.