```dv dv.paragraph(("https://jimr.fyi/" + dv.currentFilePath.replace(/\.md$/, "").replace(/ /g, "+"))) ``` | Approach | Best For | Example Use Cases | Strengths | Limitations | Approx. Cost | Time to Implement | Key Techniques | | --------------------------------- | ---------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------ | ----------------- | ------------------------------------------------------------------------------------------------ | | **Prompt Engineering** | - Quick prototyping<br>- General knowledge tasks<br>- Low-stakes applications | - Simple Q&A bots<br>- Content summarization<br>- Ideation tools | - Fast iteration<br>- Zero training costs<br>- Flexible to change | - Limited by base model capabilities<br>- Can't learn domain-specific patterns<br>- Limited context window | $100s | Days | - Few-shot learning<br>- Chain-of-thought<br>- ReAct prompting | | **Prompt + Evaluation System** | - Interactive systems<br>- Process-driven workflows<br>- Feedback collection | - Your civic participation facilitator<br>- Survey tools<br>- Guided learning systems | - Semi-automated improvement<br>- Low ongoing costs<br>- Captures user feedback | - Evaluator quality bottleneck<br>- Limited by base models<br>- Needs initial human feedback | $1,000s | Weeks | - Hierarchical prompting<br>- Synthetic data generation<br>- Constitutional AI concepts | | **Prompt + Fine-Tuned Evaluator** | - Quality-sensitive applications<br>- Domain-specific evaluation<br>- Specialized feedback | - Content moderation<br>- Specialized coaching<br>- Technical writing assistance | - Better evaluation accuracy<br>- Can learn nuanced criteria<br>- More consistent judgments | - Training resources needed<br>- Domain expertise required<br>- Periodic retraining needed | $10,000s | Weeks to Months | - LoRA (1K-10K examples)<br>- QLoRA (1K-10K examples)<br>- Adapter tuning | | **Fine-tuned Model** | - Domain expertise needed<br>- Consistent output required<br>- Specific style/tone important | - Legal document analysis<br>- Medical information systems<br>- Technical support | - Better domain performance<br>- More consistent outputs<br>- Can learn specialized patterns | - Training resources required<br>- Less flexible to change<br>- Ongoing maintenance needs | $100,000s | Months | - Full fine-tuning (10K-100K examples)<br>- PEFT methods<br>- Instruction tuning | | **Full Retraining** | - Novel domain adaptation<br>- High-stakes applications<br>- Performance-critical systems | - Financial analysis<br>- Scientific research assistance<br>- Complex decision support | - Deeper customization<br>- Better performance than fine-tuning<br>- More control over model behavior | - High compute requirements<br>- Significant ML expertise needed<br>- Extensive dataset required | $1M-10M | 3-6 Months | - Continued pre-training (100K-1M+ examples)<br>- Domain adaptation<br>- SFT on specialized data | | **Custom Model** | - Mission-critical applications<br>- Unique data characteristics<br>- Specialized requirements | - Autonomous systems<br>- Safety-critical applications<br>- Highly regulated industries | - Maximum customization<br>- Optimized for specific use case<br>- Best performance potential | - Extremely expensive<br>- Requires vast datasets<br>- Long development cycles | $10M-100M+ | 6+ Months | - Pre-training from scratch (1M-1B+ examples)<br>- Novel architecture design<br>- RLHF |