Skip to content

← Back to Ecosystem

Mountainash TreeSpec

Flatten any hierarchical data structure into clean relational tables, then build it back with full fidelity.

Why a Guided Manual?

TreeSpec has a small API surface but a lot going on beneath it: child dispatch modes, FK pre-indexing, format-specific schema introspection, hook pipelines, streaming for large files. The auto-generated API reference tells you what each class and method does. This manual explains why the library is designed the way it is, when to use each feature, and how the pieces fit together so you can make good decisions for your own data.

What's Inside

  • Chapters --- 9 chapters covering everything from the spec model and flatten/build engines through format handlers, batch processing, and hooks.

Who This Is For

Python developers and data engineers who work with tree-structured data --- XML, JSON, Protobuf, Pydantic models, Avro --- and need to move it into relational tables for analysis, storage, or validation. If that sounds like you, start with About for prerequisites and a reading guide, or jump straight into Chapter 1.

Package Overview

Mountainash TreeSpec is a Python library for bidirectional hierarchical data transformation. It flattens tree-structured data --- XML, JSON, Protobuf, Pydantic models, Avro --- into normalised relational tables, and rebuilds the original structure from those tables with round-trip fidelity. A declarative spec describes how each node maps to table rows, how children are dispatched, and how keys propagate.

Target Audience

Python developers and data engineers who need to flatten tree-structured data (XML, JSON, Protobuf, Pydantic, Avro) into clean relational tables and rebuild the original structure with round-trip fidelity.

Who This Is For

This manual is for Python developers and data engineers who deal with hierarchical data --- deeply nested JSON from REST APIs, regulatory XML filings, Protobuf messages from partner systems, Pydantic models in application code, or Avro records from legacy pipelines. You want to flatten that data into clean relational tables for analytics, validation, or storage, and sometimes rebuild the original structure for submission or interoperability.

You might be building an ETL pipeline that ingests complex XML, normalising API responses for a data warehouse, or bridging between hierarchical and relational representations in a healthcare or financial system. TreeSpec handles the transformation layer; this manual shows you how to use it well.

What You Should Already Know

You should be comfortable with:

  • Python at an intermediate level --- dataclasses, protocols, decorators, and type annotations should be familiar, not new.
  • Hierarchical vs. relational data --- you understand why a nested XML document and a set of database tables represent the same information differently.
  • At least one tree-structured format --- XML, JSON, or Protobuf. You don't need to know all five formats TreeSpec supports.
  • DataFrames --- basic familiarity with Polars or PyArrow. You don't need to be an expert, but you should know what a DataFrame is.

What You'll Get Out of This

After working through this manual, you'll know how to:

  • Write TreeSpec configurations that flatten complex hierarchies into normalised tables with referential integrity intact.
  • Rebuild original tree structures from flat tables using the same spec --- full round-trip fidelity.
  • Process entire directories of files in batch mode with automatic deduplication across shared reference tables.
  • Attach Polars transforms via hooks to normalise, enrich, or reshape data during the flatten or build step.
  • Stream large XML files into Parquet output without holding the whole document in memory.
  • Choose the right format handler and child dispatch mode for your data shape.

Prerequisites

  • Intermediate Python (dataclasses, protocols, decorators, type annotations)
  • Understanding of hierarchical vs. relational data models
  • Familiarity with at least one tree-structured format (XML, JSON, Protobuf)
  • Basic knowledge of Polars or PyArrow DataFrames

What This Manual Covers

  1. Spec Model --- TreeSpec, NodeSpec, ChildSpec, MetaConfig, and SettingsConfig: the declarative specification for tree-table decomposition.
  2. Flatten Engine --- TreeFlattener walks dataclass trees to produce flat relational tables with inline, separate, and routed child dispatch.
  3. Build Engine --- TreeBuilder reconstructs dataclass trees from flat tables using FK pre-indexing.
  4. Batch Processing --- BatchFlattener and BatchBuilder for multi-file flatten with source columns, concat, dedup, and cascade dedup.
  5. Schema Handling --- The SchemaHandler protocol with registry-based format dispatch, FieldInfo extraction, and construct/serialize methods.
  6. Format Handlers --- JSON/Dataclass, XML/xsdata, Protobuf, Pydantic, and Avro schema handlers.
  7. Type System --- UniversalType enum bridging Python types, xsdata types, and PyArrow types via type_mapping.
  8. Hooks System --- HookRegistry with condition, field_transform, pre_flatten, and post_build decorator-based hooks.
  9. DataPackage Bridge --- Converting TreeSpec and flat tables to Frictionless DataPackage with foreign keys and primary keys.
  10. Spec Derivation --- Automatic TreeSpec derivation from class hierarchies, YAML/JSON loading, and validation.

What This Manual Does Not Cover

  • Database storage and query optimization
  • ETL/ELT pipeline orchestration
  • Cloud deployment and infrastructure
  • Data visualization and reporting
  • XML Schema (XSD) or Protobuf IDL authoring

Key Capabilities

Flatten any hierarchy into clean tables --- Walk a tree of any depth and complexity into normalised relational tables with referential integrity intact. Auto-derive a spec from the schema, or hand-tune it for proprietary formats. Cascade dedup maintains integrity across tables.

Build it back with round-trip fidelity --- The same spec that flattened the tree rebuilds it. Relational tables become hierarchical structure again --- the original nesting, relationships, and format. When a regulatory filing needs a response in the same XML format, TreeSpec rebuilds it from the relational data.

Five major hierarchical formats --- JSON, XML, Protobuf, Pydantic, and Avro, each with an independently installable format handler tailored to its conventions. XML has streaming for gigabyte files. Protobuf has proto-to-spec derivation. Pydantic introspects class hierarchies.

Streaming for large files --- The streaming flattener processes large XML files through Parquet batch output without holding the whole document in memory. Multi-gigabyte regulatory feeds are a normal part of the pipeline, not a special case.

Hooks for custom transforms --- TableHook lets you attach Polars transforms to specific tables during flatten or build. Type coercion, column renaming, derived fields, data enrichment --- applied declaratively as the data flows through the spec.

Code generation from schemas --- Generate Python dataclasses from XSD, JSON Schema, Avro, or Protobuf definitions. The generated classes work directly with TreeSpec's flatten and build operations, giving you type-safe code without manual mapping.

Context

Mountainash TreeSpec provides a declarative, bidirectional bridge between hierarchical data and relational tables. A TreeSpec YAML/JSON file describes how each node in a tree maps to table rows, how children are dispatched (inline, separate table, or routed by attribute), and how keys propagate. The TreeFlattener walks the tree top-down producing flat tables; the TreeBuilder reverses the process using FK pre-indexing. Five format handlers (JSON/dataclass, XML/xsdata, Protobuf, Pydantic, Avro) abstract over schema differences via the SchemaHandler protocol. Batch processing handles multi-file workloads with source column injection, vertical/horizontal concat, and cascade dedup. A hooks system allows runtime customisation without modifying specs.