Inam Baig / projects / content-os
AI / Content Automation
Content OS
Production agentic pipeline that discovers topics, writes scripts, generates scenes, voiceovers, and media, then renders short form video.
Multi agent local LLM orchestration with human approval before publish.
Role
Agentic Engineer · Product Manager · Technical Lead
Duration
… to Present
Domain
Agentic Video Content Generation · Worldwide
Tech highlights
Problem
What was hard
Overview
Running many short form channels requires repetitive discovery, writing, asset creation, editing, and publishing work that does not scale with point tools or single prompt chatbots.
Users affected
Creators, studios, and agencies operating multiple niches and channels.
Why difficult
Each pipeline stage needs different context, tooling, and guardrails, while outputs must stay on brand, reviewable, and composable into final video.
Existing limitations
- Single LLM prompts cannot reliably run an end to end production pipeline
- Point tools do not cover discovery through publish
- Hardcoded niches do not scale across tenants
Constraints
- Multi tenant Content Sets
- Human approval before publish
- Schema validated agent outputs
- Local and cloud LLM support
Solution
Approach
Overview
A multi tenant Content OS with five specialized local LLM agents orchestrated by FastAPI workers: topic discovery, script writing, scene and media prompt generation, voiceover synthesis, and image/video generation, then Remotion/FFmpeg render and publish.
Architecture rationale
Specialized agents with narrow schemas beat one general agent: each step validates output, passes structured artifacts forward, and keeps cost, quality, and retries under control.
Major components
- Next.js operator app
- FastAPI orchestrator
- Five local LLM agents
- RabbitMQ job queues
- Voice and media generators
- Remotion renderer
- Object storage
Key capabilities
- Daily topic discovery
- Script generation
- Scene and prompt planning
- Voiceover generation
- Image and video asset generation
- Human approval and publish
Contribution
My contribution
Project outcome
I designed and built a production agentic content platform that automates short form video creation from topic discovery through rendered output.
What I personally owned
- I owned 100% of product, architecture, and engineering with agentic delivery
- I designed and implemented five local LLM agents for topics, scripts, scenes, voiceovers, and media
- I built the FastAPI orchestration layer, worker queues, operator UI, and render pipeline
Features
Key capabilities
Expand a card for implementation detail and user value.
Architecture
Interactive diagram
Zoom, pan, and select nodes to explore components and connections.
Operators configure Content Sets in Next.js. FastAPI enqueues staged jobs to RabbitMQ. Five local LLM agents run in sequence: topics, scripts, scenes/prompts, voiceovers, and media. Workers persist artifacts in PostgreSQL and S3/MinIO, assemble Remotion/FFmpeg renders, and require human approval before publish.
Stack
Technology
Grouped by layer. No skill bar percentages.
Frontend
- Next.js
- React
- TypeScript
Backend
- Python / FastAPI
- Background workers
Database
- PostgreSQL
- Redis
Infrastructure
- RabbitMQ
- S3 / MinIO
- Docker
AI / ML
- OllamaLocal LLM runtime for specialized pipeline agents
- LM StudioLocal model hosting and agent prototyping
- OpenAI APICloud LLM completions, embeddings, and generation fallback
- Anthropic APIClaude models for script/scene generation and agent tasks
- Google Gemini APIMultimodal and generation tasks where Gemini fits best
- ElevenLabsVoiceover and narration synthesis in the voice agent stage
- HeyGenAI avatar and video generation for media output
- ReplicateHosted image/video model inference for media generation
- Agentic orchestrationFive stage agent pipeline with schema validated handoffs
Integrations
- ElevenLabs APIText to speech for scene voiceovers
- HeyGen APIAI presenter/video generation
Other
- Remotion
- FFmpeg
- Playwright
- CursorAgentic engineering and implementation workflow
Decisions
Engineering decisions
Five specialized local LLM agents instead of one general agent
Why
Each stage needs different prompts, tools, and validation. Topic discovery, script writing, and scene planning fail when collapsed into a single unconstrained chat loop.
Trade off
More orchestration code, but clearer retries, observability, and output quality.
Alternatives: Single monolithic LLM prompt · Fully manual content production
Local LLM runtime via Ollama and LM Studio
Why
Keeps iteration cost low, supports offline development, and lets each agent use an appropriate local model.
Trade off
Requires local GPU/CPU capacity; cloud APIs kept as optional fallback.
Alternatives: OpenAI only · One shared remote model for all stages
Scene and prompt agent feeds both voiceover and media workers
Why
One structured scene plan keeps narration and visuals aligned before expensive generation steps.
Trade off
Extra schema design up front, but fewer mismatched audio/visual outputs.
Alternatives: Separate uncoordinated voice and media generation · Manual scene planning
Challenges
Engineering challenges
Running a reliable multi agent content pipeline in production
Why hard
Each agent depends on validated output from the previous stage, and failures must be retryable without corrupting downstream jobs.
Solution
I split the pipeline into five agents with schema validated JSON handoffs, RabbitMQ staged jobs, PostgreSQL job state, and human approval before publish.
Result
Topics, scripts, scenes, voiceovers, and media compose into Remotion renders as a repeatable production workflow.
Automating many short form niches without forking the codebase
Why hard
Point tools do not cover discovery through publish, and hardcoded niches do not scale.
Solution
Content Sets model niche, tone, schedule, and agent settings as configuration while FastAPI orchestrates shared workers.
Result
One deployment can run many channels with the same agentic pipeline.
LLM cost and quality control
Why hard
Unconstrained generation is expensive and drifts off brand.
Solution
Local LLMs for core agents, narrow schemas per stage, Content Set tone/niche as config, and human approval before publish.
Result
Generation stays bounded, reviewable, and cost controlled.
Results
Outcomes
Outcomes
- Production agentic pipeline with five specialized local LLM agents
- End to end flow from topic discovery to voiceover, media, and rendered video
- Multi tenant Content Sets with human approval before publish
Related
Related projects
Also look at