Inam Baig / projects / content-os

AI / Content Automation

Content OS

Production agentic pipeline that discovers topics, writes scripts, generates scenes, voiceovers, and media, then renders short form video.

Multi agent local LLM orchestration with human approval before publish.

Agentic Video Content GenerationWorldwideactive developmentFeatured
Next.js
React
TypeScript
FastAPI

Role

Agentic Engineer · Product Manager · Technical Lead

Duration

… to Present

Domain

Agentic Video Content Generation · Worldwide

Tech highlights

PythonFastAPINext.jsPostgreSQLRedisRabbitMQOllamaLM StudioOpenAIAnthropicGeminiElevenLabsHeyGenReplicateRemotion

Problem

What was hard

Overview

Running many short form channels requires repetitive discovery, writing, asset creation, editing, and publishing work that does not scale with point tools or single prompt chatbots.

Users affected

Creators, studios, and agencies operating multiple niches and channels.

Why difficult

Each pipeline stage needs different context, tooling, and guardrails, while outputs must stay on brand, reviewable, and composable into final video.

Existing limitations

  • Single LLM prompts cannot reliably run an end to end production pipeline
  • Point tools do not cover discovery through publish
  • Hardcoded niches do not scale across tenants

Constraints

  • Multi tenant Content Sets
  • Human approval before publish
  • Schema validated agent outputs
  • Local and cloud LLM support

Solution

Approach

Overview

A multi tenant Content OS with five specialized local LLM agents orchestrated by FastAPI workers: topic discovery, script writing, scene and media prompt generation, voiceover synthesis, and image/video generation, then Remotion/FFmpeg render and publish.

Architecture rationale

Specialized agents with narrow schemas beat one general agent: each step validates output, passes structured artifacts forward, and keeps cost, quality, and retries under control.

Major components

  • Next.js operator app
  • FastAPI orchestrator
  • Five local LLM agents
  • RabbitMQ job queues
  • Voice and media generators
  • Remotion renderer
  • Object storage

Key capabilities

  • Daily topic discovery
  • Script generation
  • Scene and prompt planning
  • Voiceover generation
  • Image and video asset generation
  • Human approval and publish

Contribution

My contribution

Project outcome

I designed and built a production agentic content platform that automates short form video creation from topic discovery through rendered output.

What I personally owned

  • I owned 100% of product, architecture, and engineering with agentic delivery
  • I designed and implemented five local LLM agents for topics, scripts, scenes, voiceovers, and media
  • I built the FastAPI orchestration layer, worker queues, operator UI, and render pipeline

Features

Key capabilities

Expand a card for implementation detail and user value.

Architecture

Interactive diagram

Zoom, pan, and select nodes to explore components and connections.

Operators configure Content Sets in Next.js. FastAPI enqueues staged jobs to RabbitMQ. Five local LLM agents run in sequence: topics, scripts, scenes/prompts, voiceovers, and media. Workers persist artifacts in PostgreSQL and S3/MinIO, assemble Remotion/FFmpeg renders, and require human approval before publish.

Loading architecture diagram…

Open in Architecture Lab →

Stack

Technology

Grouped by layer. No skill bar percentages.

Frontend

  • Next.js
  • React
  • TypeScript

Backend

  • Python / FastAPI
  • Background workers

Database

  • PostgreSQL
  • Redis

Infrastructure

  • RabbitMQ
  • S3 / MinIO
  • Docker

AI / ML

  • OllamaLocal LLM runtime for specialized pipeline agents
  • LM StudioLocal model hosting and agent prototyping
  • OpenAI APICloud LLM completions, embeddings, and generation fallback
  • Anthropic APIClaude models for script/scene generation and agent tasks
  • Google Gemini APIMultimodal and generation tasks where Gemini fits best
  • ElevenLabsVoiceover and narration synthesis in the voice agent stage
  • HeyGenAI avatar and video generation for media output
  • ReplicateHosted image/video model inference for media generation
  • Agentic orchestrationFive stage agent pipeline with schema validated handoffs

Integrations

  • ElevenLabs APIText to speech for scene voiceovers
  • HeyGen APIAI presenter/video generation

Other

  • Remotion
  • FFmpeg
  • Playwright
  • CursorAgentic engineering and implementation workflow

Decisions

Engineering decisions

  • Five specialized local LLM agents instead of one general agent

    Why

    Each stage needs different prompts, tools, and validation. Topic discovery, script writing, and scene planning fail when collapsed into a single unconstrained chat loop.

    Trade off

    More orchestration code, but clearer retries, observability, and output quality.

    Alternatives: Single monolithic LLM prompt · Fully manual content production

  • Local LLM runtime via Ollama and LM Studio

    Why

    Keeps iteration cost low, supports offline development, and lets each agent use an appropriate local model.

    Trade off

    Requires local GPU/CPU capacity; cloud APIs kept as optional fallback.

    Alternatives: OpenAI only · One shared remote model for all stages

  • Scene and prompt agent feeds both voiceover and media workers

    Why

    One structured scene plan keeps narration and visuals aligned before expensive generation steps.

    Trade off

    Extra schema design up front, but fewer mismatched audio/visual outputs.

    Alternatives: Separate uncoordinated voice and media generation · Manual scene planning

Challenges

Engineering challenges

  • Running a reliable multi agent content pipeline in production

    Why hard

    Each agent depends on validated output from the previous stage, and failures must be retryable without corrupting downstream jobs.

    Solution

    I split the pipeline into five agents with schema validated JSON handoffs, RabbitMQ staged jobs, PostgreSQL job state, and human approval before publish.

    Result

    Topics, scripts, scenes, voiceovers, and media compose into Remotion renders as a repeatable production workflow.

  • Automating many short form niches without forking the codebase

    Why hard

    Point tools do not cover discovery through publish, and hardcoded niches do not scale.

    Solution

    Content Sets model niche, tone, schedule, and agent settings as configuration while FastAPI orchestrates shared workers.

    Result

    One deployment can run many channels with the same agentic pipeline.

  • LLM cost and quality control

    Why hard

    Unconstrained generation is expensive and drifts off brand.

    Solution

    Local LLMs for core agents, narrow schemas per stage, Content Set tone/niche as config, and human approval before publish.

    Result

    Generation stays bounded, reviewable, and cost controlled.

Results

Outcomes

Outcomes

  • Production agentic pipeline with five specialized local LLM agents
  • End to end flow from topic discovery to voiceover, media, and rendered video
  • Multi tenant Content Sets with human approval before publish

Related

Related projects

Also look at

Want the full picture?

Ask me about Content OS, download my CV, or browse the architecture lab.

View CVContact