Extending DAAF

This guide covers how to make DAAF your own: onboarding new datasets, authoring custom skills and agents, customizing your Python environment, and even creating entirely new workflow modes. Everything here is for your own use -- but please do see sharing extensions if you're interested in sharing what you build back with the broader DAAF community.

The Extension Model -- Skills, Agents, and Data Onboarding

Here's the fundamental insight behind DAAF's extensibility: the framework is intended to separate what it knows from how it behaves. This is a really important distinction that makes the whole extension model work.

• Skills are structured knowledge documents. They tell DAAF's agents what they need to know about a specific topic -- a data source, a Python library, a visualization framework, a domain of expertise. Think of skills as extremely thorough, well-organized reference guides that an agent loads into its context when it needs specialized knowledge, and that can be easily shared or transferred across multiple agents.
• Agents are behavioral protocols. They tell a subagent how to behave -- what steps to follow, what to validate, when to stop, how to format output. Think of agents as detailed job descriptions that define a specific role in the pipeline (the code reviewer, the data planner, the report writer, etc.).

This separation is what makes DAAF extensible without being fragile. When you want DAAF to work with a new dataset, you generally don't need to touch the workflow, the validation logic, or the agent protocols at all. You just add a new skill that teaches the existing agents about the new data. The agents already know how to fetch, clean, transform, and analyze data -- they just need to be told the specifics of your data.

The Three Extension Paths

The most common extension path by far is adding new data sources. DAAF has a dedicated engagement mode for this: Data Onboarding Mode, which orchestrates a thorough profiling protocol and generates the skill documentation automatically. For methodology and domain expertise skills, the process is lighter-weight -- you point DAAF at documentation or literature and it drafts a skill for review.

Data Onboarding Mode (Step-by-Step)

Data Onboarding Mode is DAAF's built-in workflow for turning a raw dataset (or online data source) into a comprehensive data source skill. It automates the tedious but critical work of profiling every column, detecting coded values, checking data quality, and reconciling documentation against actual data. A 10-minute video tutorial ↗ gives you the intuition and overview for how Data Onboarding works.

Before You Start

1. A data file or API access -- either a file in a supported format (Parquet, CSV, Excel, or TSV) or access to an API that serves the data. Public data sources are strongly preferred. If working with proprietary or sensitive data, be careful to abide by your organization's AI policy and data protection standards -- Claude will be examining the actual contents of the data, and analytical outputs (sample rows, statistics, summaries) will be sent to Anthropic's servers as part of the conversation. For data protected by FERPA, HIPAA, or other regulations, consult your institution's compliance team and review your specific Anthropic license terms before proceeding.
2. Any available documentation -- codebooks, data dictionaries, README files, or documentation website URLs. Not strictly required, but they dramatically improve the resulting skill because the agent can cross-reference what the documentation says against what the data actually shows.
3. A sense of how the data will be used -- what research questions it might inform, what domain it belongs to, and which columns are most important.

Running Data Onboarding Mode

Just ask DAAF to ingest or profile a new dataset conversationally -- it will classify the request as Data Onboarding Mode automatically. Something like:

The Four Profiling Phases

1. Structural Discovery -- Basic shape of the data (rows, columns, memory footprint, column types) and initial column-level profiling. This gives the agent a bird's-eye view including null rates, unique value counts, and basic distributions. It identifies which columns uniquely identify each row and which could be used to link to other datasets.
2. Statistical Deep Dive -- Full distribution analysis for numeric columns, category enumeration for categorical columns, temporal pattern analysis, outlier detection. If the data has date/year columns or geographic identifiers, this phase also analyzes temporal coverage gaps and entity coverage.
3. Relational Analysis -- Identifying potential key columns (high uniqueness suggests an identifier), foreign keys (naming patterns like _id suffixes), hierarchical relationships, cross-column dependencies, and detection of coded values -- those suspicious negative numbers like -1, -2, -9 that often mean "missing" or "suppressed" rather than being real values.
4. Interpretation & Reconciliation -- The agent uses column names, value patterns, and domain conventions to make educated guesses about what each column means. Every interpretation is explicitly marked as [PRELIMINARY] -- the agent knows it's hypothesizing, not asserting. Column named fips? Probably a FIPS geographic code. Column with values 0 and 1? Probably a binary indicator, but is 1 "Yes" or "Male" or "Urban"? The agent will flag the ambiguity.

If documentation was provided, the profiling also runs Documentation Reconciliation: it parses the codebook or data dictionary, extracts every claim it can find, and verifies each claim against the actual data. Documentation says there are 50 columns? The agent checks. Codebook says state_code should be a string? The agent confirms or flags the mismatch. This catches the disturbingly common case where documentation is outdated or describes a different version of the data.

Reviewing the Profile Output

The agent returns a structured report with:

• Structural summary -- row/column counts, memory size, format
• Column summary -- type, null rate, unique count, and notes for every column
• Coded values detected -- which columns have potential coded values, and whether documentation confirms their meaning
• Quality assessment -- scores for completeness, documentation accuracy, and coded value coverage
• Preliminary interpretations -- the agent's best guesses for what columns mean, each flagged with a confidence level and basis
• Discrepancies found -- every case where documentation contradicted observed data, with evidence for both sides
• User review requested -- explicit questions for you to answer: which interpretations are correct, how to handle undocumented values, whether missing columns are expected

Once you've provided feedback, the agent finalizes the skill and writes it to .claude/skills/[skill-name]/. From there, you can start a fresh session and ask DAAF to analyze it alongside other datasets. Running it through some simple paces first (see Testing) is strongly recommended.

Registering Your New Skill

Skills are automatically discovered via their YAML frontmatter -- every skill with a SKILL.md file in .claude/skills/{skill-name}/ appears in the system message at conversation start. No manual registration is needed. The key to good discoverability is writing a clear, descriptive description field in your skill's YAML frontmatter.

Authoring Other Types of New Skills

Methodology Skills

For adding knowledge about a statistical method, Python library, or analytical technique, use Framework Development Mode and mention the skill-authoring skill directly. It may be helpful to refer DAAF to existing skills that yours will be most like -- a Python library skill? Reference the polars or plotnine skills as models. Something more methodological? Reference the data-scientist skill.

Domain Expertise Skills

Domain expertise skills capture knowledge about a content area rather than tooling -- the nuances of interpreting graduation rate data, the policy context around school funding formulas, or the methodological considerations for panel data in education research:

Adding a New Agent

Adding a new agent is more complex than adding a skill, because agents define behavior rather than knowledge. Framework Development Mode with the agent-authoring skill guides the process through five phases:

1. Design -- Get crystal clear on fundamentals before writing anything. The agent-authoring skill ensures you can answer five critical questions:
   1. What does this agent do and why does it exist? (one sentence)
   2. Which pipeline stage(s) does it operate in?
   3. Which existing agents are most similar, and how does yours differ?
   4. Does it need file-write access (general-purpose) or is it read-only (Plan)?
   5. Will it need to invoke any skills?
2. Author -- Write the agent definition following the canonical 12-section template (defined in agent_reference/AGENT_TEMPLATE.md). Required sections: Identity, Inputs, Core Behaviors, Protocol, Output Format, Boundaries, STOP Conditions, Anti-Patterns, Quality Standards, Invocation, References, and Consumers. The skill provides a self-validation checklist and targets 400-700 lines.
3. Integrate -- Register the agent across the DAAF ecosystem. This is where the most things can go wrong. Integration follows three tiers:
   • Tier 1 (Mandatory): Register in .claude/agents/README.md (Agent Index, When to Use, Coordination Matrix, and Agent catalog)
   • Tier 2 (Conditional): Additional updates if the agent maps to a specific pipeline stage
   • Tier 3 (Conditional): Additional updates if the agent affects specific workflow areas
4. Validate -- Verification checks to confirm cross-agent consistency and completeness.
5. Human Review -- This is non-negotiable. Review the agent file yourself for accuracy, intention, completeness, and value before it's considered done.

Key Resources

Agent definitions are stored in .claude/agents/. See the existing agents on GitHub ↗ for reference. For changes to existing agents, see the Contributing guide.

Testing Your New Extension End-to-End

Five test types to verify your extension works correctly, ordered from lightest to heaviest:

1. Data Discovery Test -- Can DAAF find your new skill and understand what it's for?
   What data sources does DAAF know about? Can you tell me about [your new data source]?

   If DAAF can't find the skill, verify the YAML frontmatter has a clear description field and that SKILL.md is in .claude/skills/{skill-name}/. Note: this may also be a non-deterministic loading issue -- LLMs don't always load what they're told to load. Try the query again in a fresh session before assuming the skill content is wrong.

2. Fetch Test -- Test that DAAF can actually retrieve and load the data.
   Can you fetch [your data source] for [year] and show me the first few rows and basic summary statistics?

   This tests the data access pathway. The fetch should complete with a CP1 validation (shape, types, missingness checks). If CP1 fails, the dataset path in your skill may not match what's actually available.

3. Context Test -- Test whether coded value mappings, missing data codes, and caveats are applied correctly during cleaning.
   Can you fetch and clean [your data source] for [year], making sure to handle any coded missing values and apply the source-specific caveats documented in the skill?

   Watch the cleaning script that DAAF produces. It should reference the specific coded values and pitfalls documented in your skill.

4. Full Pipeline Test -- The gold standard: run a simple research question through the entire pipeline.
   Using [your new data source], can you analyze [simple, well-defined research question]? Keep the scope narrow -- just verify the data flows through correctly.

   Pick a deliberately simple question. You're testing integration, not analytical sophistication.

5. Methodology/Domain Skill Test -- For non-data-source skills, test that DAAF references your skill's guidance correctly.
   I'd like to run a [method from your new skill] analysis on [some existing DAAF data]. Can you walk me through the approach?

Sharing Extensions with the Community (Optional)

Everything above works entirely locally -- your extensions are yours, and they don't need to go anywhere else. But if you've created something useful and want to share it with other DAAF users, this section covers how.

Before You Submit

• At least a Discovery Test and a Fetch Test pass for your extension
• You've thoroughly reviewed Data Onboarding output and corrected any preliminary interpretations -- skills with [PRELIMINARY] markers still in place aren't ready for sharing
• The skill follows the appropriate template, has at least 2 decision trees, and a substantive Common Pitfalls section
• The skill references only publicly accessible data -- if it was built from proprietary data, make sure it doesn't leak confidential information
• The extension follows DAAF's naming conventions

How to Submit

To share your extension with the DAAF community: fork the repository, add your skill files, update the registration entries, and submit a pull request. See the Contributing guide ↗ for the full workflow, quality standards, and review process.

If you're not comfortable with the pull request process, you can also open an issue ↗ describing your new skill and sharing the files, or post in the DAAF community channels ↗ -- the community can help get it integrated.

LEARNINGS.md: The Other Way to Contribute

Even without creating new skills, there's a contribution path that requires almost zero effort: sharing your LEARNINGS.md files. Every time DAAF completes a Full Pipeline project, it produces a LEARNINGS.md documenting everything it learned about data quirks, process issues, and methodology edge cases. These learnings are written to be immediately actionable.

You can also incorporate learnings directly into your own DAAF instance: start a new session and say "incorporate learnings" -- Framework Development mode will scan your project LEARNINGS.md files, present a consolidated backlog of framework improvements, and walk you through implementing them.

To share learnings with the broader community, open an issue ↗ with your LEARNINGS.md content. This is genuinely one of the most impactful things you can do -- every project run generates practical knowledge that benefits every future project.

Customizing Your Python Environment