Designing a Scalable Feed with DynamoDB and AWS Lambda: Lessons from Leemo

At whereweup, we build mobile-first platforms on AWS serverless. For Leemo, our social discovery app, one of the hardest features to get right was the feed. Like Instagram or TikTok, Leemo’s feed mixes posts from followed profiles and recommendations, and it needs to be fast, real-time, and scalable.

In this post, we’ll break down the feed architecture we built in Golang, AWS Lambda, DynamoDB, and SQS, focusing on the fan-out pattern with batching and parallelism.

The Core Problem

Our backend stack is fully serverless:

• AWS Lambda (Golang) → all compute
• DynamoDB → main datastore
• Cognito → identity and auth
• Elasticsearch (OpenSearch) → search for media & users

We store data in separate DynamoDB tables:

• Media → user posts (PK = mediaId)
• Relation → follow graph (PK = userId, SK = followedUserId)
• Feed → fan-out results (PK = userId, SK = timestamp)
• MediaCounters, ProfileCounters → likes, comments, plays, etc.

The naive approach—fetching posts from every followed user at feed load time—fails in DynamoDB because you can’t query across partitions efficiently. You’d need 1,000 queries if you follow 1,000 users, and results wouldn’t be globally ordered.

Fan-In vs. Fan-Out

There are two patterns for building feeds:

1. Fan-In (pull)

Build the feed dynamically when the user opens the app. Cost scales with number of followed accounts. High latency, expensive, doesn’t scale.

2. Fan-Out (push)

On post creation, push a copy of that post into each follower’s Feed partition. Reading a feed = one query on Feed table:

Query Feed where PK = :userId ORDER BY timestamp DESC

We chose fan-out.

The Fan-Out Challenge

What happens when an influencer with 1 million followers posts?

• A Lambda can’t write 1M items in one go (timeouts, retries).
• DynamoDB partitions can’t take 1M writes in a burst to the same key.
• SQS message limits (256 KB) mean we can’t send all followers in one message.

The solution: batched fan-out with parallelism.

Go Code: Batched Fan-Out to SQS

func FanOutToFollowers(ctx context.Context, sqsClient *sqs.Client, queueURL string, followers []string, mediaID, postedBy string, timestamp int64, batchSize int) error {
    total := len(followers)
    if total == 0 {
        return nil
    }

    batchCount := (total + batchSize - 1) / batchSize
    for i := 0; i < total; i += batchSize {
        end := i + batchSize
        if end > total {
            end = total
        }

        msg := FanoutMessage{
            FollowerIDs:  followers[i:end],
            MediaID:      mediaID,
            PostedBy:     postedBy,
            Timestamp:    timestamp,
            BatchNumber:  (i / batchSize) + 1,
            TotalBatches: batchCount,
        }

        body, _ := json.Marshal(msg)
        _, err := sqsClient.SendMessage(ctx, &sqs.SendMessageInput{
            QueueUrl:    aws.String(queueURL),
            MessageBody: aws.String(string(body)),
        })
        if err != nil {
            return err
        }
    }
    return nil
}
  

DynamoDB Considerations

• On-Demand Capacity → handles unpredictable spikes.
• Hot Partition Avoidance → writes are spread by followerId, not posterId.
• BatchWriteItem → workers insert 25 items at a time.

Key Takeaways

1. Fan-out > Fan-in for social feeds at scale.
2. Batching + parallelism are mandatory for >10k followers.
3. DynamoDB schema design is the backbone: Feed(userId, timestamp).
4. Serverless + Go + SQS = cost-effective, scalable feed pipeline.

At whereweup, this architecture powers Leemo’s feed today, scaling from new users to influencers with millions of followers. If you’re building a social feed on AWS, fan-out with batching is a battle-tested approach.