Architecture¶

cs-lewis-backend is a single-service Python application — Django + Django Ninja + Wagtail — running on AWS ECS Fargate, serving a Flutter mobile app and the editorial CMS from the same codebase. The backing data stores are Postgres (RDS), Redis (ElastiCache), and S3 with CloudFront edge cache for media.

This document describes what is built and how it is wired. The why behind each component choice lives in ADR-0001.

System Overview¶

                 Internet
                    │
              ┌─────▼─────┐
              │ Cloudflare│  (WAF + DDoS + rate limit)
              └─────┬─────┘
                    │
         ┌──────────┴──────────┐
         │                     │
         ▼                     ▼
   ┌──────────┐         ┌────────────┐
   │CloudFront│         │    ALB     │
   │ (media)  │         │            │
   └────┬─────┘         └─────┬──────┘
        │                     │
        ▼                     ▼
   ┌──────────┐    ┌──────────────────────┐
   │    S3    │    │  ECS Fargate         │
   │  audio + │    │  ┌──────────────┐    │
   │  images  │    │  │ Django (web) │    │  ← Django Ninja API
   └──────────┘    │  └──────────────┘    │    + Wagtail Admin
                   │  ┌──────────────┐    │    + Django Admin
                   │  │ Celery (jobs)│    │
                   │  └──────────────┘    │
                   └─────┬───────────┬────┘
                         │           │
                         ▼           ▼
                   ┌──────────┐ ┌────────────┐
                   │   RDS    │ │ElastiCache │
                   │ Postgres │ │   Redis    │
                   └──────────┘ └────────────┘
                                      │
                              ┌───────▼────────┐
                              │   External     │
                              │  • Anthropic   │
                              │  • ElevenLabs  │
                              │  • Substack    │
                              │  • Sentry      │
                              └────────────────┘

A single web tier sits behind Cloudflare (WAF + DDoS + rate limit) for the API and CloudFront for media. CDN-fronting of the API is deferred to Phase 2 — Redis at origin is sufficient at MVP scale. Celery workers run as a separate Fargate service so background workloads cannot starve the API. Postgres holds the content graph and user data; Redis caches API responses and brokers Celery jobs; S3 holds binary media.

Services & Components¶

Django web service (ECS Fargate)¶

• Process: gunicorn + uvicorn workers serving the Django application.
• Surfaces inside this process:
• Django Ninja API at /api/v1/* — Flutter mobile contract, async views, auto-generated OpenAPI.
• Wagtail Admin at /cms/* — editor authoring, StreamField content, scheduled publishing, workflows.
• Django Admin at /admin/* — user management, reflection moderation queue (Phase 2), operational tooling.
• Service layer: Wireup is the DI container wiring the API → service layer. All swappable services (Ranker, AI clients, ElevenLabs, Substack sync) are registered once in the container; views and Celery tasks declare their dependencies by signature and receive them via @inject / @inject_app. This is what makes the Postgres GIN → Meilisearch Ranker swap mechanical — one registration change, zero call-site changes.
• Scaling: horizontal on Fargate; ALB fan-out to multiple tasks.
• Reads from: Postgres (Django ORM), Redis (cache).
• Writes to: Postgres, Redis (cache writes), Celery queue (job enqueue), Sentry (errors).

Celery worker service (ECS Fargate)¶

• Process: Celery workers and a Celery Beat scheduler, deployed as separate Fargate services.
• Jobs handled at MVP:
• AI tag drafting on content ingest (Anthropic SDK).
• ElevenLabs audio pre-generation per published passage.
• Substack sync polling.
• Scheduled publishing for Daily Drop via Celery Beat.
• Phase 2 additions: Meilisearch reindex on publish; receipt validation hooks; semantic-similarity index updates if pgvector is adopted.
• Scaling: horizontal; queue depth drives autoscaling.

Postgres (AWS RDS, managed)¶

• Holds: Wagtail content (pages, snippets, StreamField payloads), Django models (users, reflections, daily drops, audit), tag arrays with GIN indexes for overlap queries.
• Configuration: Postgres 16, multi-AZ in prod, single-AZ in staging, automated daily snapshots, PITR (35-day window in prod).
• Extensions: pg_trgm for fuzzy text matching where useful; planned pgvector for Phase 2 semantic similarity if it earns its place.

Redis (AWS ElastiCache, managed)¶

• Holds: API response cache, Celery broker, Celery result backend (short TTL), JWT denylist and refresh-token revocation, rate-limit counters.
• Configuration: Redis 7, single-shard at MVP, multi-AZ replica in prod.

S3 + CloudFront¶

• Holds: ElevenLabs-generated audio files, AI-generated featured images, editor uploads.
• Access: read-public at MVP (all content is guest-accessible). Signed URLs in Phase 2 for tier-gated audio.
• Cache: CloudFront edge in front of S3; long TTL on immutable content keyed by content hash.

Request Flow¶

Guest read — today's passage¶

1. Mobile client → WAF → ALB → Django Ninja.
2. Redis cache lookup on today:{date}:{register}. HIT returns immediately.
3. MISS: Postgres query for today's DailyDrop joined to its Passage → Pydantic response → cache write → response.
4. Mobile fetches audio + featured image directly via CloudFront URLs.

Guest read — dive deeper from a passage¶

1. Mobile → WAF → ALB → Django Ninja.
2. Ranker.related_to(passage_id, filters) is invoked.
3. At MVP, the ranker is backed by Postgres GIN array overlap:

   SELECT id, label,
          array_length(motif_tags & :seed_motifs, 1) AS overlap
   FROM passage
   WHERE motif_tags && :seed_motifs
     AND id != :seed_id
     AND register = ANY(:register_filter)
   ORDER BY overlap DESC
   LIMIT 20;
   

4. Response cached under related:{passage_id}:{filters_hash}; TTL tuned to publish cadence.
5. Phase 2 swap: the Ranker implementation moves to Meilisearch without changing call-sites.

Editor publish — passage with AI-drafted tags¶

1. Editor authors a passage in Wagtail admin and saves a draft.
2. Post-save signal enqueues draft_tags_for_passage(passage_id) on Celery.
3. Worker calls Anthropic SDK with the passage text + tag taxonomy prompt → returns drafted tags with confidence scores.
4. Worker writes the draft tags to the passage (review_status=draft) and surfaces a "needs review" indicator in the admin.
5. Editor reviews, edits, flips workflow state to reviewed.
6. Wagtail's workflow gate allows publish. On publish, a post-publish signal:
7. Invalidates the relevant Redis cache keys.
8. Enqueues generate_audio_reading(passage_id) → ElevenLabs → S3.
9. (Phase 2) Enqueues reindex_passage_in_search(passage_id).

Data Stores¶

See docs/cms-architecture.md for the content data model and collection schemas.

Network Topology¶

• VPC: dedicated VPC per environment with public + private subnets across two AZs.
• Public subnets: ALB, NAT gateway.
• Private subnets: ECS tasks (web + Celery), RDS, ElastiCache.
• Security groups: ALB → ECS (443); ECS → RDS (5432); ECS → Redis (6379); ECS → S3 via VPC gateway endpoint (no NAT egress for S3).
• Egress: outbound to Anthropic, ElevenLabs, Substack, Sentry via NAT gateway.
• DNS: Route 53; domain to be confirmed with client.
• TLS: ACM-issued certs on ALB and CloudFront; TLS 1.2+ enforced.

Environments¶

Infrastructure as Code¶

Detailed IaC choices land in deployment.md once infrastructure is provisioned.

External Dependencies¶

Security Architecture¶

Scaling Characteristics¶

Performance Targets¶

Open Items¶

• Domain / DNS — to be confirmed with client.
• Account-deletion strategy (V1) — FK on-delete strategies (cascade vs anonymise) to be defined in the first sprint; must be production-tested before App Store submission.
• Passage vs Article modeling — to resolve in docs/cms-architecture.md before first sprint commits.

Related¶

• docs/adr/0001-backend-architecture.md — decision rationale, considered alternatives, production credibility
• docs/cms-architecture.md — content data model the Wagtail/Django models encode
• docs/guides/deployment.md — deployment pipeline, IaC layout, rollback
• docs/api-overview.md — API conventions (auth, errors, pagination, caching)
• Internal working memory (.context/) — product domain, scope, constraints