Building Scalable APIs with Go: A Startup's Guide to Not Overengineering

Every startup engineering conversation eventually arrives at: “But what if we need to scale?”

You probably won’t. Not in the way Twitter or Uber scaled. The infrastructure decisions that make sense for 100 million users are actively harmful at 1,000. I inherited a project recently where the team was running a Kubernetes cluster and paying thousands a month in infrastructure costs. I moved it to Lambda + SQS and AWS App Runner and got the same result for about $50/month, with none of the operational headaches of Kubernetes.

This post is about right-sizing your API architecture: the math on when you actually need to worry about scale, why containerization beats serverless for most APIs, and how to build systems that are simple enough to debug at 3 AM but can still grow with you.

The Scale Math Most Startups Ignore

Engineers love talking about “millions of requests,” so let’s break down what that actually means.

AWS Lambda limits (as of 2024):

• 1,000 concurrent executions (default, can request increase)
• 10,000 concurrent executions (typical raised limit)
• 15-minute maximum execution time
• ~$0.20 per million requests + compute time

GCP Cloud Run limits:

• 1,000 concurrent requests per instance (configurable)
• Up to 1,000 instances per service (default)
• No execution time limit for HTTP
• ~$0.40 per vCPU-hour + requests

Now work backwards from “I need more than Lambda can handle”:

With 10,000 concurrent Lambda executions and a 100ms average response time, you can handle:

10,000 concurrent × (1000ms / 100ms) = 100,000 requests/second

That’s 8.6 billion requests per month. Unless you’re building the next TikTok, Lambda can handle your traffic.

I’ve had this conversation with founding engineers where we agreed: if we have to move off Lambda, that’s an earned problem. Until then, we’re fine. Don’t solve problems you don’t have yet.

Serverless vs Containers

Both can handle massive traffic, so I pick based on development workflow and cost structure.

Lambda (or Cloud Functions, Azure Functions) fits event-driven workloads like S3 triggers, queue consumers, and webhooks, plus infrequent bursty traffic like cron jobs and admin tools. It also wins at very low traffic, under 1M requests/month, where pay-per-invocation beats a running container, and for simple single-purpose functions like image resizing or data transformation. If you’re already deep in AWS, GCP, or Azure, the native integrations help too.

Containers (Cloud Run, ECS Fargate, Azure Container Apps) make more sense for user-facing APIs where latency consistency matters, for long-running operations over 15 minutes, websockets, or streaming, and when local development needs to match production exactly. They handle VPC and database access more cleanly, and at predictable traffic above 5M requests/month, reserved capacity gets cheaper than paying per invocation.

In practice I use both: containers for the API, serverless for background jobs, things like S3-triggered processing, scheduled cleanup, and webhook handlers for Stripe or SendGrid.

Why Containers Often Win for APIs

For user-facing APIs, managed containers typically beat serverless functions, for a few concrete reasons.

Lambda cold starts range from 100ms (Go, Node.js) to several seconds (Java, C#, Python with heavy dependencies). For a CLI tool or batch job that’s fine. For an API where p99 latency matters, cold starts show up in your metrics and in user complaints. Managed containers keep instances warm by default, so your p99 stays consistent because you’re not paying the cold start tax on the first request after idle.

A Dockerfile that works locally works in production:

# Local
docker run -p 8080:8080 -e DATABASE_URL=... my-service

# Production (Cloud Run)
gcloud run deploy my-service --image=gcr.io/project/my-service

# Production (ECS)
aws ecs update-service --service my-service --force-new-deployment

# Production (Azure)
az containerapp update --name my-service --image myregistry.azurecr.io/my-service

Lambda requires SAM CLI, LocalStack, or other emulators, and the dev environment never quite matches production. Containers are the same everywhere, which also makes debugging easier: pull the exact image that’s running, run it locally with production environment variables, and reproduce the issue. Lambda’s execution environment is opaque. You’re debugging through CloudWatch logs and educated guesses about what the runtime actually looks like. Containers are also portable: a Cloud Run service can move to ECS, Kubernetes, or a VPS with minimal changes, where Lambda functions need rewriting for each provider’s API.

None of this means Lambda is wrong. It’s still better for event-driven workloads, infrequent bursty traffic, and cost optimization at very low scale. One nuance worth mentioning: at AWS, we don’t consider Lambda statically stable, meaning you have to engineer around the possibility that it can’t scale when you need it to. The fan-out technique helps, or you can use ECR + ECS with smart autoscaling. The key is not setting limits high until you need to, and watching throughput so you don’t overallocate. For most startups this doesn’t matter, but if you’re building on Lambda at scale, it’s worth understanding.

All three major cloud providers offer managed containers:

Cloud Run makes sense if you’re on GCP or starting fresh and want the simplest deployment with scale-to-zero and minimal cold starts. ECS Fargate makes sense if you’re already on AWS and need tight integration with RDS, S3, and the rest, or want more control over networking and security. Azure Container Apps makes sense if you’re on Azure, use .NET heavily, or want Kubernetes-like features without running Kubernetes. All three are good choices. Pick based on your existing cloud provider, not container platform features.

The Simplest Architecture That Works

Here’s what a simple, scalable API architecture looks like:

                    ┌─────────────────┐
                    │  Load Balancer  │
                    │  (Cloud LB/ALB) │
                    └────────┬────────┘
                             │
         ┌───────────────────┼───────────────────┐
         │                   │                   │
         ▼                   ▼                   ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│  Auth Service   │ │ Billing Service │ │   API Service   │
│  (Container)    │ │  (Container)    │ │  (Container)    │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
         │                   │                   │
         └───────────────────┼───────────────────┘
                             │
                    ┌────────▼────────┐
                    │    Database     │
                    │  (Managed SQL)  │
                    └─────────────────┘

That’s it. No Kubernetes yet, no service mesh, no API gateway, no message queues for synchronous operations. Each service is a single Go binary in a Docker container, connects directly to the database, handles its own authentication via shared middleware, and scales independently based on CPU/memory. Start here. Add complexity when metrics force you to, not before.

That inherited Kubernetes cluster from the intro didn’t need any of it. Lambda + SQS + App Runner gave the team the same functionality at a fraction of the cost and complexity. When you need actual K8s features, custom networking, stateful workloads, multi-cloud, you’ll know. Until then, skip it.

The same logic applies down the stack. An API gateway isn’t worth adding until you need centralized policy management; managed containers already give you HTTPS, SSL termination, and basic routing, and auth and rate limiting can live in middleware. A service mesh adds observability and traffic management, but also real complexity, so stick with plain HTTP calls and structured logging until debugging distributed systems gets genuinely painful, usually somewhere past 10 services and 5 engineers. And if Service A needs data from Service B right now, make an HTTP call. Save Pub/Sub or Kafka for actual async work like notifications, analytics, or fan-out, not request/response.

Go Patterns That Actually Matter

Forget the “high-throughput” patterns you read about for building the next trading platform. For a startup API, the one pattern worth implementing properly is bounded concurrency: don’t spawn unbounded goroutines.

// Bad: unbounded goroutine creation
func ProcessRequests(requests <-chan Request) {
    for req := range requests {
        go handleRequest(req) // This will kill you under load
    }
}

// Good: worker pool with bounded concurrency
func ProcessWithPool(requests <-chan Request, numWorkers int) {
    for i := 0; i < numWorkers; i++ {
        go func() {
            for req := range requests {
                handleRequest(req)
            }
        }()
    }
}

For most APIs you won’t need this; HTTP servers handle concurrency for you. But for background processing, batch jobs, or fan-out operations, bounded concurrency keeps you from getting OOM killed.

Every external call needs a timeout. Every database query, every HTTP request, no exceptions.

ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

result, err := db.QueryContext(ctx, query)
if err != nil {
    if errors.Is(err, context.DeadlineExceeded) {
        return fmt.Errorf("query timeout: %w", err)
    }
    return err
}

The default should be aggressive: 5 seconds for HTTP calls, 10 seconds for database queries. If you need longer, make it explicit and justify it.

When calling third-party APIs like Stripe or SendGrid, implement circuit breakers:

type CircuitBreaker struct {
    failures    int
    maxFailures int
    resetAfter  time.Duration
    lastFailure time.Time
    mu          sync.Mutex
}

func (cb *CircuitBreaker) Execute(fn func() error) error {
    cb.mu.Lock()
    if cb.failures >= cb.maxFailures {
        if time.Since(cb.lastFailure) < cb.resetAfter {
            cb.mu.Unlock()
            return ErrCircuitOpen
        }
        cb.failures = 0 // Reset after timeout
    }
    cb.mu.Unlock()

    err := fn()
    if err != nil {
        cb.mu.Lock()
        cb.failures++
        cb.lastFailure = time.Now()
        cb.mu.Unlock()
    }
    return err
}

When Stripe has an outage, you don’t want every request in your system waiting 30 seconds for a timeout. Circuit breakers fail fast after detecting problems.

Structured logging isn’t a performance pattern, but it’s critical for debugging:

logger.Info("request processed",
    "user_id", userID,
    "duration_ms", duration.Milliseconds(),
    "status", status,
)

Use slog (Go 1.21+) or zerolog. JSON output, structured fields. When you’re debugging at 3 AM, grep and jq are your friends.

What Actually Causes Scale Problems

In my experience, scale problems at startups are rarely about request volume. They’re about bad query patterns and synchronous bottlenecks. On a recent consulting engagement, the same project I talk about in my discovery process and 5-minute rule posts, I inherited queries timing out at 30+ seconds. The team had worked around it by hardcoding data into static JSON files. When I wired the frontend to the actual API, pages were unusable.

The biggest culprit was N+1 queries: fetching a list of 100 users, then making 100 more database queries to get their profiles, 101 queries instead of 2.

// Bad: N+1
users, _ := db.Query("SELECT id FROM users LIMIT 100")
for users.Next() {
    var id int
    users.Scan(&id)
    profile, _ := db.Query("SELECT * FROM profiles WHERE user_id = ?", id)
    // ...
}

// Good: Single query with JOIN or IN clause
query := `
    SELECT u.id, p.*
    FROM users u
    LEFT JOIN profiles p ON p.user_id = u.id
    LIMIT 100
`
rows, _ := db.Query(query)

N+1 queries are the number one cause of slow APIs I’ve seen. Use query logging in development to catch them.

Beyond fixing N+1s, you can parallelize queries and split large data processing into smaller chunks, or, for database queries, run a few smaller queries with code-level processing instead of one giant complex JOIN; it’s often easier and faster. On that consulting project, I replaced the massive queries with smaller, decomposed ORM-based ones. Query time dropped from 30+ seconds to ~200ms.

Two other things worth checking: unbounded result sets and missing indexes. If a table can grow unboundedly, every query needs a LIMIT.

// Bad: fetching all records
users, _ := db.Query("SELECT * FROM users")

// Good: always paginate
users, _ := db.Query("SELECT * FROM users LIMIT 100 OFFSET ?", offset)

And if a query is slow, check EXPLAIN ANALYZE and add an index. This alone solves most database performance issues.

-- Check query plan
EXPLAIN ANALYZE SELECT * FROM orders WHERE user_id = 123;

-- Add index if missing
CREATE INDEX idx_orders_user_id ON orders(user_id);

Last one: don’t call Stripe, send emails, or hit external APIs in the request path if you can avoid it. Queue the work, return immediately, process async.

// Bad: synchronous email in request handler
func HandleSignup(w http.ResponseWriter, r *http.Request) {
    user := createUser(r)
    sendWelcomeEmail(user) // This blocks the response
    json.NewEncoder(w).Encode(user)
}

// Good: async via queue
func HandleSignup(w http.ResponseWriter, r *http.Request) {
    user := createUser(r)
    queue.Publish("send-welcome-email", user.ID)
    json.NewEncoder(w).Encode(user)
}

Measuring Success: Metrics That Matter

Track four metrics and ignore everything else until they tell you to optimize.

Request latency: p50 under 100ms is good, p95 under 200ms is good, p99 under 500ms is acceptable for most APIs, and p99 over 1000ms means investigate.

// Middleware to track latency
func LatencyMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        next.ServeHTTP(w, r)
        duration := time.Since(start)
        
        logger.Info("request",
            "path", r.URL.Path,
            "method", r.Method,
            "duration_ms", duration.Milliseconds(),
        )
    })
}

Error rate: 5xx responses as a percentage of total requests. Under 0.1% is excellent, 0.1-1% is good, 1-5% needs attention, and above 5% you’re on fire.

Database query time matters too. Log all queries over 100ms:

func SlowQueryLogger(ctx context.Context, query string, duration time.Duration) {
    if duration > 100*time.Millisecond {
        logger.Warn("slow query",
            "query", query,
            "duration_ms", duration.Milliseconds(),
        )
    }
}

And saturation: CPU above 80% means you’re at capacity and need to scale, memory above 80% means check for leaks.

Once you have 5+ services, add distributed tracing. Once you have 10+ engineers, add service-level SLOs. Once you have paying customers, add business metrics. Start simple and add complexity when the simple metrics stop being enough.

The Migration Path

You don’t have to choose your architecture forever. Most teams start on serverless in month one: deploy Lambda functions, no container knowledge required, pay-per-invocation, fast time to market. That’s enough until cold starts start affecting user experience (p95 over 500ms), local development starts slowing the team down, you need VPC or database access, or you cross 5M requests/month.

From there, most teams move to managed containers somewhere around month six: containerize the services, deploy to Cloud Run, Fargate, or ACA, keep the same autoscaling with a lot less operational burden than Kubernetes. Most startups stay here for years, and that’s success, not failure.

The move to actual Kubernetes, usually year two or later, only makes sense once you have 5+ engineers dedicated to platform, need multi-region active-active, need custom networking or a service mesh, or you’re running 20+ services. It gets you full control at the cost of full complexity, and it requires a dedicated platform team to run. Kubernetes is a tool for specific problems.

Anti-Patterns to Avoid

If your services are in the same monorepo, deploy to the same cloud, and are owned by the same team, ask whether they need to be separate services at all. A single service with good module boundaries is easier to develop, test, and debug than three services that always deploy together.

Don’t add caching before you need it. Redis is operational complexity you’re taking on, and before reaching for it you should check your database queries for indexes and N+1s, check your connection pooling, and actually profile request latency. Caching hides problems; fix the root cause first.

And don’t reach for Kubernetes until you have a platform team to run it. Cloud Run, ECS, and similar managed services give you most of the benefit with a fraction of the operational cost. Revisit that decision once you have dedicated platform engineers and multi-region requirements.

Before You Ship

Run through this once before launch:

• All external calls have timeouts, and database queries are parameterized
• Secrets live in environment variables, not code, and everything is HTTPS only
• Every endpoint except health checks requires authentication
• List endpoints paginate, common queries have indexes, N+1s are gone, and external calls happen async when possible
• Circuit breakers sit in front of third-party services
• There’s a /health endpoint and graceful shutdown for in-flight requests
• Logging is structured with request IDs, errors carry correlation IDs, and the database connection pool is configured
• The full stack runs locally with docker-compose, with a README that explains how, and environment variables documented
• Deployment is automated, with no manual steps, and rollback is documented

If you can check all of these, ship it. Monitor it. Scale when the metrics tell you to, not when your ego does.

Further reading:

• Pilum: my open-source multi-cloud deployment orchestrator. It started as too many Python scripts and grew into a proper CLI in Go, which is a story for another post.
• The Twelve-Factor App
• Golang Best Practices