Jason Liu – Systematically Improving RAG Applications

Turn your RAG system from demo-ready to production-grade with a proven, data-driven framework.
Master systematic evaluation, multimodal retrieval, and measurable optimization to deliver mission-critical search performance in weeks—not months.

Jason Liu – Systematically Improving RAG Applications

Stop building RAG systems that impress in demos but disappoint in production

Transform your retrieval from “good enough” to “mission-critical” in weeks, not months. Most RAG systems stall in prototype purgatory: they demo well, but fail on complex queries—eroding trust and wasting engineering time. The difference isn’t just better tech, but a systematic mindset.

With the RAG Flywheel, you’ll:
✅ Pinpoint failures with synthetic evals
✅ Fine-tune embeddings for 20–40% gains
✅ Collect 5x more user feedback
✅ Segment queries to target high-impact fixes
✅ Build multimodal indices for docs, tables, images
✅ Route queries to the best retriever automatically

Week by week, you move from vague “make it better” to clear metrics, focused improvements, and compounding value. Real-world results include +20% accuracy from re-ranking, +14% with cross-encoders, and $50M revenue boosts from better search.

Join 400+ engineers applying this framework in production. Instructor Jason Liu has built multimodal retrieval and recommendation systems at Facebook, Stitch Fix, and through consulting—experience that shaped this practical, battle-tested approach.

What you’ll learn

Follow a repeatable process to continually evaluate and improve your RAG application

Analyze and Diagnose RAG System Performance

• Evaluate retrieval quality using precision, recall, and MRR metrics to identify system weaknesses
• Differentiate between leading metrics (experiments run) and lagging metrics (customer satisfaction) to drive actionable improvements
• Design synthetic data generation pipelines that enable rapid experimentation without waiting for user data

Construct Data-Driven Improvement Frameworks

• Create comprehensive evaluation datasets using LLMs to generate realistic query-answer pairs
• Establish baselines using tools like LanceDB to benchmark different retrieval implementations

Design Specialized Search Systems

• Develop multimodal retrieval systems that handle documents, images, tables, and structured data
• Synthesize lexical (BM25), semantic (embeddings), and metadata-based search for optimal results

Optimize Query Understanding & Routing

• Extract structured information from diverse data sources to enable precise filtering
• Classify queries using domain expertise and few-shot classifiers to improve routing accuracy

Learn directly from Jason

Jason Liu

Staff machine learning engineer, currently working as an AI consultant

Jason has built search and recommendation systems for the past 8 years. He has consulted and advised a dozens startups in the last year to improve their RAG systems. He is the creator of the Instructor Python library.

Prerequisites

• Deployed a RAG System

  The goal of this course is not just to share with you a how-to guide, but rather how to systematically improve these architectures.

• Optional (Python)

  We have over 20 iPython notebooks that you can explore, run code to be more hands-on with the experiments that we plan to run.

What’s included

Live sessions

Learn directly from Jason Liu in a real-time, interactive format.

6 Prerecorded Lectures

Short, focused videos that unpack the full RAG-improvement framework that you can rewatch anytime.

6+ Office Hour Q&As

Open office hours for deep dives, debugging help, and personalized feedback.

12 Hands-On Python Notebooks

Ready-to-run notebooks & walkthrough videos so you can practice every concept instantly.

Lifetime Slack Community

Private Slack for peer reviews, job leads, and ongoing support forever.

Expert Speaker Library

Curated talks from builders running large-scale RAG systems in production.

$2K+ in Cloud & AI Credits

Test vector DBs, LLM APIs, and infra with over $2,000 in partner credits.

Free Future Re-Enrollment

Join any future cohort at no cost and get updated content and live coaching again whenever you need it.

Certificate of completion

Showcase your advanced RAG skills to clients, employers, and your LinkedIn network.