Module 10 of 18
Prompt Chaining & Orchestration Patterns
Design multi-prompt workflows with routing, branching, and error handling for complex AI systems
Sequential Chaining
Sequential chaining is the most fundamental orchestration pattern: the output of one prompt becomes the input to the next. This simple pattern is remarkably powerful because it allows you to decompose complex tasks into manageable steps, where each step can be optimized independently.
The key to effective sequential chaining is defining clear contracts between steps, specifying exactly what each step produces and what the next step expects. Without these contracts, chains become fragile and break when any step produces unexpected output.
Definition
Sequential Chaining
Input/Output Contracts
Define typed schemas for each step's input and output. Use JSON Schema or TypeScript interfaces to make contracts explicit. Validate between steps to catch errors early.
Step Independence
Design each step to be independently testable and replaceable. If step 2 fails, you should be able to retry it with step 1's output without re-running the entire chain.
Context Accumulation
Decide how much context flows through the chain. Passing everything creates long prompts; passing too little loses important information. Use summaries to compress context between steps.
Checkpointing
Save intermediate results so that if a later step fails, you can resume from the last successful checkpoint rather than starting over. This is essential for long, expensive chains.
A well-designed chain should read like a pipeline: each step has a clear purpose, a defined input, a defined output, and can be understood in isolation. If you cannot explain what a step does without referencing other steps, your chain needs better decomposition.
Parallel Execution
Not all steps in a workflow need to run sequentially. When tasks are independent, executing them in parallel dramatically reduces total latency. Parallel execution is the key to building prompt workflows that are both thorough and fast.
Sub-Task Decomposition
Break a complex task into independent sub-tasks that can run simultaneously. For example, analyzing a document for legal compliance, financial impact, and technical feasibility can happen in three parallel prompts, each with its own specialized instructions.
Pattern: Decompose, fan-out to parallel prompts, fan-in to merge results. The merge step synthesizes findings from all parallel analyses.
Independence Verification
Before parallelizing, verify that tasks are truly independent. If task B needs information from task A, they must run sequentially. A dependency graph helps identify which tasks can safely run in parallel.
Result Merging
After parallel execution, a merge step combines results. Design the merge prompt to handle conflicting findings, prioritize by source expertise, and produce a unified output. The merge step is often the most critical part of a parallel workflow.
Pattern: "You have received analyses from three specialists: [legal], [financial], [technical]. Synthesize their findings into a unified recommendation. Where analyses conflict, note the disagreement and explain the trade-offs."
Routing and Branching
Routing patterns use a classifier prompt to analyze the input and direct it to the most appropriate handler. This is how you build systems that can handle diverse inputs with specialized processing for each type, similar to how a receptionist routes calls to the right department.
Classifier Design
Define clear, mutually exclusive categories with descriptions and examples. The classifier prompt should output category, confidence score, and brief reasoning for audit purposes.
Confidence-Based Routing
Set confidence thresholds for automatic routing vs. human review. High-confidence predictions go directly to handlers; low-confidence predictions are flagged for human oversight.
Specialized Handlers
Each category gets a handler prompt optimized for that specific type of input. Handlers can use different models, temperatures, and output formats based on the category's needs.
Fallback Strategy
Design a general-purpose handler for inputs that do not fit any category. This prevents the system from failing silently when it encounters unexpected input types.
Map-Reduce Patterns
Map-reduce is a powerful pattern for processing large inputs that exceed a single prompt's effective capacity. The input is split into chunks (map), each chunk is processed independently, and the results are combined into a final output (reduce). This pattern is essential for tasks like summarizing long documents, analyzing large codebases, or processing datasets.
Map Phase: Chunking Strategy
How you split the input matters enormously. Split at natural boundaries (paragraphs, sections, functions) rather than at arbitrary token counts. Include overlap between chunks to prevent losing context at boundaries. Each chunk should be self-contained enough for meaningful processing.
Pattern: "Analyze this section of the document. Previous section summary: {prev_summary}. Current section: {chunk}. Extract: [specific data points]. Note any references to other sections."
Reduce Phase: Synthesis
The reduce step must do more than concatenate results. It should synthesize, deduplicate, resolve conflicts, and produce a coherent output. For hierarchical reduce, process results in batches (e.g., reduce 10 chunks to 3 summaries, then reduce 3 to 1).
Pattern: "Synthesize these {n} section analyses into a comprehensive summary. Remove duplicates, resolve any contradictions by favoring later sections, and maintain the document's logical flow."
Hierarchical Map-Reduce
For very large inputs, use multiple levels of reduction. First reduce chunks into section summaries, then reduce sections into chapter summaries, then reduce chapters into a final summary. Each level uses a different prompt optimized for its granularity.
The most common map-reduce failure is losing important details during the reduce phase. To prevent this, include specific instructions about what must be preserved (key numbers, names, dates, conclusions) and what can be safely summarized or omitted.
Recursive and Iterative Patterns
Recursive patterns apply the same prompt logic repeatedly, with each iteration building on the previous result. The most common application is self-refining loops, where a generate-critique-revise cycle iteratively improves output quality until convergence criteria are met.
The power of recursive patterns is that they can achieve quality levels that single-pass prompts cannot. By explicitly separating generation from evaluation, you leverage the model's ability to critique text (which is easier than generating perfect text from scratch).
Definition
Self-Refining Loop
Generate-Critique-Revise Cycle
The generator produces initial output, the critic evaluates it against specific dimensions with actionable feedback, and the reviser applies the feedback. Each role has its own optimized prompt.
Convergence Criteria
Define clear stopping conditions to prevent infinite loops: target quality score achieved, maximum iterations reached, improvement between iterations drops below threshold, or no remaining issues above a minimum severity.
Progressive Refinement
Structure the critique to focus on different aspects in each iteration. First iteration: factual accuracy. Second iteration: structure and flow. Third iteration: style and polish. This prevents the model from trying to fix everything at once and potentially introducing new issues.
Error Handling in Chains
Prompt chains are only as reliable as their weakest link. Without robust error handling, a single malformed output can cascade through the entire chain, producing garbage results or causing hard failures. Designing for failure is not optional; it is what separates prototype chains from production systems.
Input Validation
Before each step, validate that the previous step's output matches the expected schema. Check for required fields, correct data types, and reasonable value ranges. Reject malformed input immediately.
Retry with Repair
When a step produces malformed output, retry with an augmented prompt that includes the malformed output and a description of the issue: "Your previous output had [issue]. Please correct and reformat."
Graceful Degradation
Design chains that can produce partial results when some steps fail. If the analysis step fails but extraction succeeded, return the extracted data without analysis rather than failing entirely.
Dead Letter Queues
When a step fails after maximum retries, save the input and error context to a dead letter queue for human review. This prevents data loss and provides debugging information.
Error Handling Decision Tree
Schema Validation Fails: Retry with repair prompt (max 2 retries), fall back to simpler output format, route to dead letter queue.
API Error (timeout/rate limit): Exponential backoff retry, switch to fallback model, queue for delayed processing.
Quality Below Threshold: Run self-refining loop (max 3 iterations), accept with quality warning flag, route for human review.
Unexpected Category/Route: Send to general handler, log for classifier retraining, alert human operator.
Design chains with the assumption that any step can fail. The question is not "will this chain fail?" but "when this chain fails, will it fail gracefully?" Input validation, structured retries, and graceful degradation are the three pillars of reliable prompt orchestration.
Activities
Activity 1
40-50 min · Advanced
Prompt Chain Design
Design a 4-step prompt chain for a real task. Define the input/output contract for each step. Test with edge cases.
Activity 2
45-55 min · Advanced
Routing Prompt System
Implement a routing prompt that classifies inputs into 5 categories and routes to specialized handlers. Measure classification accuracy.
Activity 3
40-50 min · Advanced
Self-Refining Loop
Build a self-refining loop that iteratively improves a piece of writing through critique-and-revise cycles. Define convergence criteria.
Real-World Applications
Document Analysis Pipelines
Multi-step workflows that extract, classify, analyze, and summarize documents from diverse sources with specialized processing at each stage.
Content Creation Workflows
End-to-end content pipelines from research to outline to draft to review to publication, with quality gates between each step.
Customer Intent Routing
Intelligent routing systems that classify customer inputs and direct them to specialized handlers for optimal response quality and efficiency.
Data Processing Pipelines
Extract-transform-analyze workflows that process raw data through multiple AI-powered stages to produce structured insights.
Multi-Agent Coordination
Systems where multiple AI agents with different specializations collaborate on complex tasks through structured communication protocols.
Quality Assurance Loops
Self-improving systems that generate, evaluate, and refine outputs iteratively until quality thresholds are met.