Understanding DAAF

What is DAAF?

DAAF is an AI-powered research assistant that helps you go from a research question to a completed analysis (including data acquisition, cleaning, statistical analysis, visualizations, and a written report) while keeping you in control of every decision. It runs inside a tool called Claude Code (Anthropic's AI coding assistant) on your computer. You interact with it by typing instructions in plain English, and DAAF handles the technical work while checking in with you at key decision points. Claude Code is a powerful general-purpose AI coding assistant, but it wasn't designed specifically for research: DAAF adds the structure, the domain knowledge, and the safety guardrails that turn it into a rigorous research tool.

Ignoring the fancy terms for a moment (agents, subagents, skills, orchestrators) DAAF is just a series of pre-cooked "recipes" of context that get handed to Claude at exactly the right moments. Every single design decision in DAAF comes down to telling Claude exactly what it needs to know, when it needs to know it, so it does what you want more often and with higher quality on average -- transparently and rigorously and reproducibly, like a scientist would prefer.

Why that works, and why it matters so much, becomes much clearer once you understand how these AI systems actually operate under the hood. So that's where we'll start. You need to know this before you do anything else with DAAF, because it will fundamentally shape how you use and interact with it, as well as what you should expect from it.

How LLMs Actually Work

Every large language model (LLM) assistant (e.g., Claude, ChatGPT, Gemini) is, at its core, an autocomplete engine. It takes a sequence of words and tries to come up with the next best, most sensical word in that sequence -- then the next, then the next. I like to describe modern AI as autocomplete with an extremely fancy hat: the hat keeps getting fancier, but the mechanic underneath hasn't changed. Everything crazy you've seen an AI do like writing code, building slide decks, and searching the web, all fundamentally stems from this one simple but extremely flexible mechanic.

That mechanic is also the technology's fundamental flaw. Predicting the next plausible word is not a process grounded in truth or correctness. When an LLM gets something wrong and "hallucinates," it isn't malfunctioning: it's just making stuff up the way it always does, predicting a word that happens to imply an untruth or an unfact. To see how this works (and why there's still real hope for making these tools useful), take a sequence of words:

The model turns the words it's given into mathematical points in space and uses complex math to predict the next work (to wildly oversimplify, you can almost imagine it plots a sort of best-fit prediction/regression line) then picks somewhat randomly among the top candidates above the threshold of likelihood.

That random selection among top candidates is intentional, by the way: model providers add it deliberately to produce variety and something resembling creativity. But it also means the same question can get different answers on different days: pizza today, cheese next week, ice cream tomorrow. It'll also change based on exactly how we phrase the question itself. If we're asking these systems for anything factual or truth-based, we are going to be disappointed.

Two Kinds of "Memory"

So if these systems aren't grounded in truth, how could they possibly be useful for science -- which requires truth and fact and rigor and care? Here's a mental model that helps (it's not the most accurate technical description, but it's a substantively useful one): an LLM almost draws on two kinds of "memory" when it fills in that blank.

Its long-term memory is the training data -- any given LLM model "reads" essentially every publicly available book, code repository, Wikipedia article, and Reddit thread to form a baked-in, fuzzy understanding of language and the world when it is first created. When a model relies solely on this long-term memory, it's piecing together a bunch of fuzzy information all at once without really getting any of it quite right -- it will confidently make stuff up, and it will often be wrong in the details. You can imagine at least in part because it has so much data to influence it, and because a lot of that information may conflict. Its short-term memory is information that is provided directly to it in something called the context window -- the instructions, documents, and conversation you provide directly, sitting presently in its "mind" as you chat with it. When the model works from material in its short-term memory, its informational recall and specific topical synthesis are far more precise and far more useful.

The practical implication: whenever we can shift an LLM's work from relying on its long-term memory to instead relying on what we carefully curate and provide directly to it in its short-term memory, we get a lot better performance out of it, and it becomes much, much more useful. To see what that looks like, return to our "favorite food" sentence from earlier: this time with a rich paragraph of context placed directly into the model's short-term memory:

A sufficiently advanced model recognizes that Janice is a distinct person whose preferences stand in opposition to the speaker's -- and the entire prediction shifts accordingly. Nothing about the model changed; only the words it was given, and how it then leverages that important provided context to shift how it thinks the rest of the words should look.

This single move is the key: nearly everything that has happened in AI over the past year is, one way or another, just a version of leveraging this "short term memory" in increasingly creative and robust ways to get better outputs from LLMs for specific tasks. If we give it domain expertise in the context window, if we give it core data documentation info, if we give it data analysis API references, we start to get less noise and more quality, grounded output. To be really clear, though: it is still a probabilistic engine underneath, not a truth-based one, and so will always be at risk of getting it wrong. No amount of finagling with short-term memory can fully guarantee success.

Context Windows and Context Rot

While all of that is really valuable and important, different models can only "digest" a certain amount of context before predicting the next word. That capacity is known as the model's context window. (Capacity is technically measured in tokens -- word fragments, with one token roughly 3/4 of a word -- but thinking in words is close enough.) For perspective: GPT-3 could only really "read" about 1,500 words -- any more than that was ignored or would break it. Claude Opus 4.6 handles roughly 150,000 words by default, with context windows of ~750,000 words in beta at time of writing (mid-2026). Expanding this capacity is one major frontier of model advancement, because the more context a model can hold, the more expertise and framing it can use for the task at hand.

But more context isn't automatically better, because not all context is treated equally: models weigh different parts of their context in complex, sometimes unpredictable ways. Fill the window with scattered, conflicting, or poorly structured material, and the model becomes confused, erratic, and forgetful. This failure mode is called context rot, and it needs to be avoided at all costs.

One subtlety that catches many people: in a chat, everything accumulates. Your first message and the model's response become part of the context that shapes its next response, and so on. Ask for help revising a syllabus, then drafting a letter to your mom, then picking a TV show, and by the third request the model is working from a weird middle-ground synthesis of all three. The generalizable advice: have focused conversations. One task per conversation; when you switch gears, start fresh.

From Prompt Engineering to Context Engineering

The craft of deciding exactly how to talk to an LLM and ask it to do what you want in clear, structured ways as you chat with it is known colloquially as "prompt engineering." It's more of an art than a science in most circumstances -- people are largely making things up as they go and seeing what works, which means new expertise is forming in really informal communities like Reddit and X.

The most recent wave of model advancement goes a step further. Models can now read many documents you provide to it directly, and moreover interpret what you're asking for and assemble their own context dynamically: reading files they judge relevant to your request beyond what you've suggested, searching the web for current information, even delegating side-conversations to separate AI assistants whose findings flow back into the main conversation. Designing the systems, instructions, and processes that help an LLM intelligently manage and piece together its own short-term memory is called context engineering. It's a step beyond prompt engineering (which is about crafting individual prompts well) into something more architectural: how do you set up an entire system so the right information gets loaded at the right time, every time?

Three Dimensions of AI Capability

One useful way to think about where AI is right now -- and why people seem to disagree so strongly about how capable it is -- is to think about AI capability as having three interdependent dimensions:

1. The Mind -- the base model's raw intelligence and reasoning ability. This is what Anthropic, Google, and OpenAI are competing on with each new model release.
2. The Body -- the orchestration frameworks and tooling that let the model actually do things: read files, run code, search the web, delegate tasks to other models. Claude Code is the "body" that lets Claude's "mind" interact with your computer. DAAF adds a much more structured and capable body on top of that.
3. The Instructions -- your skill in communicating what you want, plus whatever pre-built instructions the system provides. This covers everything from how you phrase a question to how an entire orchestration system like DAAF structures its instructions behind the scenes.

Each dimension is necessary but insufficient on its own. A brilliant model with no tools can only chat. Powerful tools connected to a weak model will produce sophisticated-looking garbage. A strong model with great tools but vague instructions will go confidently in the wrong direction. The real capability of any AI system is a product of all three working together -- which is why blanket statements about "what AI can and can't do" are so often wrong. It depends enormously on the configuration.

Notice that the second and third dimensions are exactly what we just covered: the Body is tooling that lets a model act on the world and curate its own context, and the Instructions are context engineering made concrete. That's what DAAF is, at its core -- a context engineering framework designed specifically for research workflows. Everything in DAAF -- the skills, the agents, the orchestrator, the progressive loading of reference files -- is fundamentally an answer to the question: "What context does Claude need right now to do this specific research task well?" The DAAF Field Guide ↗ has more detail on these concepts.

This framework also explains why people have such wildly different experiences with AI. Someone chatting casually with a basic web interface is experiencing one narrow slice of what's possible. Someone using Claude Code with DAAF and well-crafted prompts is operating in a genuinely different capability regime -- not because the underlying model is different, but because the other two dimensions are dramatically more developed. The information gradient here is steep: a lot of people are still having the "AI is dumb, it can't count the Rs in strawberry" experience, while people who have invested in tooling and instruction quality are seeing capabilities that are invisible to casual users, and vice versa. This is a significant part of why discourse around AI can feel so polarized -- people are often talking past each other because they're working with very different combinations of these three dimensions.

From Intuition to Design: The Three Challenges

With that shared understanding in place, the design problem comes into focus. If we want an autocomplete engine -- however fancy the hat -- to support real research, there are three big challenges that anyone building (or evaluating) such a system will need to grapple with in some way, shape, or form:

1. Teaching it to think like a researcher. If we want that short-term memory filled with really useful material, what does it actually contain? It's worth pausing on the underlying question: what are the core scientific principles we hope and expect human researchers to be trained to protect and embody -- and what would those look like written down as instructions and reference files? It's not a perfect one-to-one translation, but it tells us what we're trying to preserve.
2. Context engineering across the entire research workflow. Research isn't one task; it's dozens of very different ones. What a careful researcher needs to know while profiling an unfamiliar dataset is completely different from what they need while specifying a regression, or interpreting complicated, conflicting results. If the research process were easy to teach, it wouldn't take a PhD to learn it -- but it does, because it's really hard. Dynamically adapting what context an AI receives at each stage is a huge engineering challenge.
3. Proactively mitigating harms, costs, and risks. There's the big stuff -- deleted data, leaked credentials, a misdirected email. And then there's the small stuff that matters most for research: a coding error that seems right but isn't, an interpretation that seems right but isn't, a statistical method that seems right but isn't. What permissioning, oversight, and review processes let us work confidently with a collaborator we know will sometimes be wrong?

Notice that these questions sound less like configuring software and more like teaching a new colleague and setting up a research lab. Those are exactly the design considerations behind everything that follows.

The rest of this page is, in effect, DAAF's answer sheet. The skills and agent protocols are the answer to teaching it to think like a researcher. The engagement modes and orchestrator workflow are the answer to context engineering across the entire research process. And the dual-layer validation system is the answer to proactively mitigating harms, costs, and risks.

The Mental Model: Orchestrator, Agents, Skills

You don't need to understand the architecture to use DAAF -- you can absolutely just type a research question and let it run. But if you want to understand why DAAF does what it does, why it pauses when it pauses, and why the output is structured the way it is, this section will hopefully make all of that click.

Thoughtfully shaping context is how we move Claude from what I lovingly describe as an over-eager recent MBA graduate -- terrible memory, very quick to please you -- into something much closer to a careful research colleague. Even after all that shaping, they're still an over-eager MBA grad at heart, so you'll still want to review their output before it goes out into the world. So I'm going to use an analogy that I think captures DAAF's architecture well: DAAF is intended to mirror the workflows of a well-run research lab with you as the lead researcher -- the PI, in lab terms.

The Orchestrator: Your Lab Director

When you type a message to DAAF, you're talking to the orchestrator. Think of the orchestrator as a lab director -- the person who takes your research question, figures out what needs to be done, decides who on the team should do each piece, coordinates the whole effort, and reports back to you at key milestones.

The orchestrator should NOT be doing the hands-on work itself, because its primary value-add and contribution is coordination and workflow management. It doesn't write analysis scripts, it doesn't clean data, it doesn't run regressions. What it does is:

• Classify your request into an engagement mode
• Delegate tasks in proper sequence to specialized agents
• Enforce quality gates
• Report progress to you and pause for your approval at key junctures
• Keep the big picture in mind

The orchestrator is the most important part of DAAF, because it needs to simultaneously understand the whole workflow and the broad context of the work, while also being technical enough and specific enough to give precise, targeted instructions and context to every other LLM assistant involved in the process. (This is also why model choice matters so much for DAAF: DAAFBench measures precisely how well different models handle this orchestration role -- classifying requests, delegating properly, following protocol.)

Specialized Agents: Your Research Team

Trying to get Claude to do everything equally well within a single context window is impossible -- overloading one assistant with every responsibility and every reference file will ultimately confuse it and cause the dreaded context rot we covered earlier. This means we need to split responsibilities across "versions" of Claude provided very different instructions and behavioral protocols, each working with a lean, focused context of its own.

Skills: Your Team's Reference Library

If agents define behavior ("how should I work?"), then skills define knowledge ("what do I need to know?"). Skills are structured knowledge documents that agents load into their own context on demand. Think of them as specialized reference manuals that your research team pulls off the shelf when they need domain-specific information.

Skill categories include:

• Data source skills -- Out of the box, we include education-data-explorer and education-data-query for finding and fetching data from the Urban Institute Education Data Portal, plus one skill for each major education data source, covering what endpoints exist, what variables mean and how they're coded, what caveats and limitations to watch for, what suppression rules apply, and historical context about data collection (Note: these are exemplar skills that should inform how you can make your own data skills via the Data Onboarding mode; DAAF is not restricted to the education data domain at all).
• Technical skills -- data-scientist for methodology and rigor principles, polars for data manipulation, plotnine for static publication-quality plots, plotly for interactive visualizations, marimo for reactive Python notebooks
• Meta skills -- skill-authoring and agent-authoring for extending DAAF itself

The key insight: in DAAF, skills are generally intended to be loaded by agents, not by the orchestrator. When the orchestrator delegates a task to the research-executor, it tells the agent which skills to load. The agent pulls up the relevant reference material, uses it to guide its work, and returns findings to the orchestrator. This keeps the orchestrator's context lean (it doesn't need to hold the full contents of every skill in memory) and ensures each agent gets exactly the knowledge it needs for its specific task.

Click to expand

DAAF's multi-agent architecture: The user communicates with the orchestrator, which dispatches specialized agents. Each agent loads the skills it needs -- structured knowledge documents with optional sub-references for detailed guidance.

×

The Nine Engagement Modes

DAAF first classifies every request you make into one of nine engagement modes. Each mode triggers a fundamentally different workflow, different outputs, and different expectations. Understanding these modes is the single most useful thing you can do to work with DAAF effectively, because it helps you frame your questions in the way most likely to get you what you actually want, and better understand what's going on behind the scenes.

Before doing anything else, DAAF will tell you which mode it's classifying your request into, explain why, and ask you to confirm. This is intentional. You should always have the chance to say "actually, I just wanted a quick lookup" or "actually, let's go deeper on this."

Expand any mode below for trigger words, detailed descriptions, and guidance on when to use it -- and when not to:

Mode Transitions

DAAF supports clean transitions between any pair of modes where the shift makes sense. You don't need to memorize the transitions -- DAAF will suggest the right mode at natural breakpoints and wait for your confirmation. It should never silently switch modes on you.

Try It Yourself: A Guided Progression

Rather than try to jump in with a complete Full Pipeline Analysis at once, I strongly recommend testing out the simpler features and engagement modes first. The whole premise of this project is that DAAF is surprisingly robust, but the right way to build confidence is to start small and work your way up. Here's a concrete progression I'd recommend, designed to let you assess DAAF's knowledge and capabilities at each level of complexity:

Anatomy of a Completed Analysis

When a Full Pipeline analysis completes, your project folder will look something like this:

Each artifact serves a specific purpose. The Plan.md is the single most important artifact in the project -- it captures everything about what was done and why. If the scripts are the "what," Plan.md is the "why." It includes the research question (verbatim), Research Outcomes (specific, measurable topics the analysis must investigate -- these define what must be examined, not what the answer should be), data sources with rationale, methodology with justification, a risk register, and a key decisions log. If any outcomes read like hypotheses (predicting a direction), flag them. Plan_Tasks.md contains the detailed machine-readable task specifications with the exact transformation sequence, dependencies, wave assignments for parallel execution, and input/output file paths.

The scripts/ directory is the real work product -- not the notebook. Get a sense for how these scripts are actually written and run: this is where DAAF's value lives. Without the core engine of data analysis being transparent, rigorous, and reproducible, nothing else that comes out of this process is valuable. Spend time here. Each script reads top-to-bottom like a lab notebook -- no functions, no classes, no jumping around -- with clear section headers, inline audit trail comments explaining intent and reasoning, embedded validation assertions, and an appended execution log showing exactly what happened when the script ran. When a script fails QA and needs revision, the original keeps its output (it's part of the audit trail), and the revised version gets a letter suffix: 01_task.py → 01_task_a.py → 01_task_b.py. The cr/ subdirectory contains the code-reviewer's independent QA inspection scripts for every analysis script.

The Notebook.py is a Marimo notebook assembled from the completed scripts -- it's the presentation layer, not where analysis was done. What you won't see in the notebook: new analysis code, interactive dashboards, filter widgets, or additional transformations. This ensures what you see is exactly what was executed and validated, with nothing added or changed. The Report.md is a stakeholder-ready narrative synthesizing key findings, methodology, limitations, visualizations, and a references section (data sources, methodological references, software, and reporting standards -- DAAF tracks these automatically, though you should verify accuracy).

All data files use Apache Parquet format rather than CSV. Parquet preserves data types (integers stay integers, dates stay dates), compresses efficiently, and is fast to read. CSV files lose type information -- everything becomes a string -- which introduces subtle bugs. Parquet prevents an entire category of data quality issues.

STATE.md tracks the current state of the analysis -- which tasks are completed, which are pending, and any issues encountered. This is critical for multi-session work. LEARNINGS.md captures data source quirks, surprising findings, and methodological notes that might help future analyses.

Sample Projects

The DAAF repository includes sample projects in the research/ folder to illustrate what DAAF produces. The College Graduation Rate & Selectivity Analysis ↗ demonstrates a complete Full Pipeline run -- browse the Report ↗, the Plan ↗, a data fetch script ↗, or a statistical analysis script ↗ to see real artifacts. We've also created an intuitive, interactive, and highly-curated walk-through of this main sample project at See How it Works. A companion Reproducibility Verification ↗ shows what independent re-execution and verification looks like.

These projects are presented warts and all -- some of the interpretation is arguably overblown, and some analytical choices could be questioned. That's the point: DAAF produces work that is worth reviewing, not work that can be trusted blindly.

Dual-Layer Validation

This is where DAAF most directly confronts that third challenge -- and honestly, where the biggest gap exists in most ad-hoc, LLM-assisted analysis today. Remember the core mechanic: a probabilistic engine can never guarantee correct output, only better or worse odds of it. So DAAF assumes errors will happen and builds two independent layers of validation to catch them before they reach you.

Layer 1: Primary Validation (CP1-CP4)

Layer 2: Secondary Validation (QA1-QA4b)

If a primary checkpoint fails, execution stops. Period. DAAF doesn't try to power through -- it reports the failure and either attempts a fix or escalates to you. Every script gets both layers of review, no exceptions, and the code-reviewer approaches each script with an adversarial mindset and inspects it immediately after execution, not batched at the end -- because an error in script 1 that goes undetected will silently propagate through scripts 2, 3, and 4, compounding into a mess that's far harder to diagnose and fix.

Why two layers? Because they catch different types of errors. Primary validation catches operational failures -- the data is empty, the types are wrong, something clearly broke. Secondary QA catches methodological errors -- the code runs fine and produces output, but the methodology is wrong, the join logic is subtly off, or the interpretation doesn't match what the data actually shows. These are the insidious errors that humans regularly miss when reviewing LLM-generated code, and they're exactly the errors that matter most for research integrity.

The Full Pipeline Flow

The orchestrator coordinates several specialized agents for different tasks. You don't need to memorize who does what -- DAAF manages the team automatically. The important thing is understanding the overall flow and where your review points are.

1. You ask a research question
2. The orchestrator classifies the request as Full Pipeline and confirms with you
3. The orchestrator delegates data exploration to a subagent
4. The orchestrator delegates source deep-dives to source-researcher agents
5. A synthesis agent consolidates all findings
6. The orchestrator pauses for your review (Phase Status Update 1) ← your checkpoint
7. A planning agent creates a detailed Plan, validated by a plan-checking agent
8. The orchestrator pauses for your review again (Phase Status Update 2) ← your checkpoint
9. An execution agent works through each task, with a code-reviewing agent inspecting each script
10. The orchestrator pauses twice more for your review (Phase Status Updates 3 and 4) ← your checkpoints
11. A notebook assembly agent compiles all scripts
12. A report-writing agent creates the stakeholder report
13. A verification agent performs adversarial final verification
14. The orchestrator delivers everything to you

That's the core loop. Every piece has a job. Every job has a quality check. Every quality check has consequences (stop, revise, or proceed). And you get four mandatory check-in points where DAAF pauses and waits for your explicit approval before continuing.

Session Management

Real research analyses take time -- often more time than a single Claude Code session can handle. DAAF monitors its own context utilization to handle this gracefully:

When context runs low (or you choose to stop), DAAF writes a comprehensive STATE.md that captures exactly where work stands and provides a restart prompt -- a pre-written message capturing exactly what has been done and what needs to happen next.

How to Restart a Session

1. Type /clear in the Claude Code terminal to reset the session (this clears the context window but does not affect any files on disk)
2. Paste the restart prompt that DAAF provided
3. DAAF reads STATE.md, picks up where it left off, and continues working

If you closed your terminal entirely or the session crashed, start a new Claude Code session and point DAAF to the project folder -- it will read STATE.md and figure out where to resume.

Reproducibility Verification mode note: Reproducibility Verification mode uses Reproduction_Report.md as its session state document instead of STATE.md.

Where Things Live

You don't need to know what's in most of these directories. The two that matter to you are research/ (where your analyses live) and user_reference/ (where documentation lives). Everything else is DAAF's internal machinery.

Everything you need to review, share, or reproduce is inside the project folder. You can copy the entire folder to a colleague and they'd have everything needed to understand and verify the analysis -- that's the whole point of reproducibility.

Browsing and Viewing Your Work

DAAF includes convenience scripts for viewing your files, notebooks, and session logs outside of the terminal:

• Browse and edit project files: bash run_vscode.sh (macOS/Linux) or .\run_vscode.ps1 (Windows) -- opens a browser-based VS Code editor
• View interactive notebooks: bash view_notebooks.sh (macOS/Linux) or .\view_notebooks.ps1 (Windows) -- opens Marimo at localhost:2718
• View session logs: bash view_logs.sh -- opens the DAAF Log Explorer, an interactive timeline that shows orchestrator actions, subagent dispatches, and tool calls in your browser at localhost:2719