Skip to content

Package Overview

← Back to Home

This page frames Mountainash TreeSpec as a course of study: who it's for, what you need to know before you start, what you'll be able to do afterward, and how the material is sequenced. For a narrative introduction to the library itself, see Home; for authorship, licensing, and how this manual relates to the code repository, see About.

Course Summary

TreeSpec has a small public API but a lot of design behind it — child dispatch modes, foreign-key pre-indexing, format-specific schema introspection, hook pipelines, streaming for large files. This course walks through that design in order, so that by the end you're not just calling flatten() and build() correctly, you understand why the engine is built the way it is and can extend it — new format handlers, custom hooks, tuned specs for awkward real-world data — with confidence.

Intended Audience

This course is aimed at Python developers and data engineers who already work with tree-structured data — deeply nested JSON from REST APIs, regulatory XML filings, Protobuf messages from partner systems, Pydantic models in application code, or Avro records from legacy pipelines — and need to move that data into relational tables (and often back again) as part of an ETL pipeline, a data warehouse ingestion path, or a system that bridges hierarchical and relational representations.

Prerequisites

Before starting, you should be comfortable with:

  • Intermediate Python — dataclasses, protocols, decorators, and type annotations should be familiar, not new material.
  • Hierarchical vs. relational data modelling — you should already understand, conceptually, why a nested XML document and a set of normalised database tables encode the same information differently.
  • At least one tree-structured format — XML, JSON, or Protobuf. You do not need prior exposure to all five formats TreeSpec supports; the format-specific chapter will bring you up to speed on the others.
  • Basic DataFrame familiarity — Polars or PyArrow. You don't need to be an expert, just know what a DataFrame is and how to read one.

Learning Outcomes

By the end of this course, you will be able to:

  1. Write TreeSpec configurations (TreeSpec, NodeSpec, ChildSpec) that flatten complex hierarchies into normalised tables while keeping referential integrity intact.
  2. Rebuild original tree structures from flat tables using that same spec, with full round-trip fidelity.
  3. Choose the right child dispatch mode — inline, separate table, or attribute-routed — for a given data shape.
  4. Process entire directories of files in batch mode, with automatic deduplication across shared reference tables and cascade dedup for dependent tables.
  5. Attach Polars transforms via the hooks system to normalise, enrich, or reshape data during the flatten or build step, without modifying the spec itself.
  6. Stream large XML files into Parquet output without holding the entire document in memory.
  7. Read and extend a format handler that implements the SchemaHandler protocol, and understand how the type system bridges Python, xsdata, and PyArrow types.

What the Course Does Not Cover

Consistent with the library's scope, this course deliberately does not cover database storage or query optimisation, ETL/ELT pipeline orchestration, cloud deployment and infrastructure, data visualisation and reporting, or the authoring of XML Schema (XSD) or Protobuf IDL definitions themselves. TreeSpec consumes those schemas; it doesn't teach you to write them.

Syllabus

The course is organised as 9 chapters, progressing from foundational concepts through the full engine internals:

Chapter Focus
1. Foundation Concepts Why hierarchical and relational data need a bridge, and the vocabulary this manual uses throughout.
2. Spec Model TreeSpec, NodeSpec, ChildSpec, MetaConfig, and SettingsConfig — the declarative contract for tree-table decomposition.
3. Flatten Engine How TreeFlattener walks dataclass trees top-down to produce flat relational tables, including inline, separate, and routed child dispatch.
4. Type System The UniversalType enum and type_mapping, bridging Python types, xsdata types, and PyArrow types.
5. Schema Handling The SchemaHandler protocol, registry-based format dispatch, and FieldInfo extraction.
6. Build Engine How TreeBuilder reverses the flatten process, reconstructing dataclass trees from flat tables using FK pre-indexing.
7. Format Handlers The five format handlers — JSON/Dataclass, XML/xsdata, Protobuf, Pydantic, and Avro — and their format-specific conventions.
8. Batch Processing BatchFlattener and BatchBuilder for multi-file workloads: source columns, concat, dedup, and cascade dedup.
9. Hooks System HookRegistry and the condition, field_transform, pre_flatten, and post_build decorator-based hooks.

Each chapter builds on the ones before it — the spec model chapter is worth reading closely even if you only care about, say, format handlers, since every later chapter assumes fluency with TreeSpec, NodeSpec, and ChildSpec.

How to Use This Course

Read chapters in order if you're new to TreeSpec — the spec model and flatten engine chapters establish vocabulary and mental models that later chapters depend on. If you already have a working TreeSpec configuration and want to go deeper on one topic (streaming, hooks, a specific format handler), jump directly to the relevant chapter via the chapter list; each chapter is written to stand reasonably well on its own once you know the spec model.