Home Mixer

Orchestration layer that assembles the For You feed using the Candidate Pipeline framework

Overview

Home Mixer is the primary orchestration service that powers the For You feed on X. It coordinates multiple subsystems to retrieve, rank, and filter posts, returning a personalized feed for each user.

The service leverages the Candidate Pipeline framework to compose a multi-stage recommendation pipeline, combining both in-network content (from Thunder) and out-of-network content (from Phoenix Retrieval).

Architecture

Home Mixer exposes a gRPC endpoint (ScoredPostsService) that processes feed requests through a series of pipeline stages:

pub struct HomeMixerServer {
    phx_candidate_pipeline: Arc<PhoenixCandidatePipeline>,
}

Pipeline Stages

The PhoenixCandidatePipeline implements the complete feed generation flow:

1. Query Hydration

   Fetch user context before retrieving candidates:

   • User Action Sequence: Recent engagement history (likes, replies, reposts)
   • User Features: Following list, preferences, language, country
2. Candidate Sourcing

   Retrieve candidates from multiple sources in parallel:

   • Thunder: In-network posts from followed accounts
   • Phoenix Retrieval: Out-of-network posts discovered via ML-based similarity search
3. Hydration

   Enrich candidates with additional metadata:

   • Core post data (text, media, timestamps)
   • Author information (username, verification status)
   • Video duration for video posts
   • Subscription eligibility
4. Pre-Scoring Filters

   Remove ineligible candidates:

   • Duplicates and reposts of same content
   • Posts older than retention threshold
   • Self-posts (user’s own tweets)
   • Blocked/muted authors
   • Muted keywords
   • Previously seen/served posts
5. Scoring

   Apply multiple scorers sequentially:

   • Phoenix Scorer: ML predictions from transformer model
   • Weighted Scorer: Combine predictions into relevance score
   • Author Diversity Scorer: Attenuate repeated authors
   • OON Scorer: Adjust out-of-network content scores
6. Selection

   Sort by final score and select top K candidates

7. Post-Selection Processing

   Final validation and cleanup:

   • Visibility filtering (spam, violence, gore detection)
   • Conversation deduplication
8. Side Effects

   Cache request information for future optimizations

Implementation

The pipeline is constructed using the builder pattern, assembling all components:

home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs

impl PhoenixCandidatePipeline {
    async fn build_with_clients(
        uas_fetcher: Arc<UserActionSequenceFetcher>,
        phoenix_client: Arc<dyn PhoenixPredictionClient>,
        phoenix_retrieval_client: Arc<dyn PhoenixRetrievalClient>,
        thunder_client: Arc<ThunderClient>,
        // ... other clients
    ) -> PhoenixCandidatePipeline {
        // Query Hydrators
        let query_hydrators = vec![
            Box::new(UserActionSeqQueryHydrator::new(uas_fetcher)),
            Box::new(UserFeaturesQueryHydrator { strato_client }),
        ];

        // Sources
        let sources = vec![
            Box::new(PhoenixSource { phoenix_retrieval_client }),
            Box::new(ThunderSource { thunder_client }),
        ];

        // ... filters, scorers, selectors
    }
}

Request Flow

When a user requests their For You feed:

home-mixer/server.rs

#[tonic::async_trait]
impl ScoredPostsService for HomeMixerServer {
    async fn get_scored_posts(
        &self,
        request: Request<ScoredPostsQuery>,
    ) -> Result<Response<ScoredPostsResponse>, Status> {
        let query = ScoredPostsQuery::new(
            proto_query.viewer_id,
            proto_query.client_app_id,
            proto_query.country_code,
            proto_query.language_code,
            proto_query.seen_ids,
            proto_query.served_ids,
            proto_query.in_network_only,
            // ...
        );
        
        let pipeline_result = self.phx_candidate_pipeline.execute(query).await;
        
        let scored_posts: Vec<ScoredPost> = pipeline_result
            .selected_candidates
            .into_iter()
            .map(|candidate| ScoredPost { /* ... */ })
            .collect();
            
        Ok(Response::new(ScoredPostsResponse { scored_posts }))
    }
}

Key Components

Sources

Two primary candidate sources run in parallel:

Thunder Source

Retrieves recent posts from accounts the user follows (in-network)

Phoenix Source

ML-based retrieval of relevant posts from the global corpus (out-of-network)

Filters

Home Mixer applies extensive filtering to ensure feed quality:

Filter	Purpose
DropDuplicatesFilter	Remove duplicate post IDs
CoreDataHydrationFilter	Remove posts missing core metadata
AgeFilter	Remove posts older than threshold
SelfTweetFilter	Remove user’s own posts
RetweetDeduplicationFilter	Dedupe reposts of same content
IneligibleSubscriptionFilter	Remove paywalled content user can’t access
PreviouslySeenPostsFilter	Remove already seen posts
PreviouslyServedPostsFilter	Remove posts served in current session
MutedKeywordFilter	Remove posts with muted keywords
AuthorSocialgraphFilter	Remove blocked/muted authors
VFFilter	Remove spam/violence/gore
DedupConversationFilter	Deduplicate conversation threads

Scorers

Multiple scorers combine to produce the final relevance score:

1. Phoenix Scorer

   Calls the Phoenix ranking model to get ML predictions for each engagement type (like, reply, repost, etc.)

2. Weighted Scorer

   Combines multiple predictions into a single relevance score using weighted sum

3. Author Diversity Scorer

   Attenuates scores for posts from the same author to ensure feed diversity

4. OON Scorer

   Adjusts scores for out-of-network content based on user preferences

Configuration

Key parameters controlling pipeline behavior:

// Maximum results from each source
pub const THUNDER_MAX_RESULTS: u32 = 1000;
pub const PHOENIX_MAX_RESULTS: usize = 500;

// Final result size
pub const RESULT_SIZE: usize = 100;

// Post age limits
pub const MAX_POST_AGE: u64 = 86400 * 2; // 2 days

Performance Characteristics

The pipeline is highly optimized for low latency:

• Parallel execution: Sources and hydrators run concurrently
• Sub-millisecond lookups: Thunder provides in-memory access to recent posts
• Batch prediction: Phoenix scores all candidates in a single inference call
• Streaming responses: gRPC with Zstd compression for efficient transport

Typical request latency: 100-300ms end-to-end

Monitoring

Home Mixer emits detailed metrics at each pipeline stage:

#[xai_stats_macro::receive_stats]
async fn get_scored_posts(...) -> Result<...> {
    // Automatic instrumentation tracks:
    // - Request count and latency
    // - Candidates from each source
    // - Filtered vs kept candidates
    // - Final result size
}

Related Components

Candidate Pipeline

Framework that powers Home Mixer’s orchestration

Phoenix

ML component for retrieval and ranking

Thunder

In-memory post store for in-network content

Example Request

grpcurl -d '{
  "viewer_id": 12345,
  "client_app_id": "web",
  "country_code": "US",
  "language_code": "en",
  "seen_ids": [],
  "served_ids": [],
  "in_network_only": false
}' \
  home-mixer.example.com:443 \
  xai.ScoredPostsService/GetScoredPosts

Set in_network_only: true to disable Phoenix retrieval and only return posts from followed accounts.