Data Catalog

MicrosoftOfficialPublished by Microsoft

Skill
Content generator
Data and AI
CC-BY-4.0
Low risk
Has scripts

From the author

Create and enrich durable data catalogs using the native DS_CATALOG_V1 Markdown contract, declared entity relationships, privacy citation fields, and stable relationship IDs. Use when inventorying engagement data, recording semantic relationships, or preparing a catalog for ERD rendering.

— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The data-catalog skill provides a standardized procedure for inventorying business entities and their relationships. It utilizes the DS_CATALOG_V1 Markdown contract to ensure that data catalogs are both human-readable and machine-validatable. The workflow emphasizes data integrity by requiring stable identifiers, explicit confidence levels for relationships, and clear separation between semantic facts and narrative descriptions. It is designed to support data science and engineering engagements by providing a reliable, versioned record of data assets, lineage, and privacy classifications without embedding sensitive credentials or duplicating profile data.

What it does

Defines a structured Markdown and YAML format for data catalogs.
Requires stable, unique identifiers for all entities and relationships.
Enforces explicit confidence levels (confirmed, inferred, assumed) for all semantic links.
Validates catalog structure against a predefined JSON schema.
Separates metadata and facts from narrative documentation.
Provides a mechanism for tracking privacy standards and lineage.

When to use it

When you need to inventory engagement data for a data science or engineering project.
When recording semantic relationships between business entities for ERD rendering.
When preparing a durable, versioned data catalog that must be machine-readable.
When you need to document privacy classifications and lineage in a standardized way.

Advantages

Ensures consistency across different data catalogs using a strict contract.
Prevents the accidental leakage of credentials by prohibiting embedded secrets.
Maintains high data quality by requiring explicit evidence for relationships.
Supports automated validation to catch errors before review.
Provides a clear, audit-ready history of data relationships.

Usage examples

“Create a new data catalog for the customer engagement project.”: The agent initializes a new Markdown file using the DS_CATALOG_V1 template and guides the user through inventorying entities and defining their relationships.
“Validate the current data catalog file.”: The agent executes the validation script to ensure the YAML frontmatter adheres to the DS_CATALOG_V1 schema and reports any structural issues.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/microsoft/data-catalog/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip data-catalog.zip -d .agents/skills

Documentation by Microsoft

Data Catalog Workflow

Goal

Produce a customer-readable Markdown catalog whose YAML frontmatter is a valid DS_CATALOG_V1 machine contract. Preserve uncertainty explicitly so inferred or assumed relationships never appear confirmed.

Flow

Confirm the engagement name and the caller-approved durable output path.
Inventory entities at business grain. Record source access, tier, volume, profile pointer, classification, lineage, and open questions without copying column-level profile data.
Assign every relationship a stable rel-* identifier. Record endpoints, maximum cardinality, both endpoint minimums, one or more paired join-key fields, confidence, and evidence basis.
Reconcile coverage counts with the entity and relationship records.
Render the human-readable sections from the YAML facts, ending with the canonical Data Science and Engineering Coaching disclaimer footer. Narrative can explain facts but cannot redefine them.
Validate the artifact with scripts/validate_catalog.py before treating it as ready for review.

Inputs

Engagement context and a caller-approved output path
Data source inventory and access status
Business entity names, grain, and declared relationships
Existing per-dataset profile paths, when available
Privacy classifications or standards citations produced by the owning privacy workflow

Success criteria

The frontmatter declares exactly catalog_version: DS_CATALOG_V1 and validates against assets/ds-catalog-v1.schema.json.
Entity IDs and relationship IDs are unique and stable. Every endpoint and lineage reference resolves.
Every relationship declares cardinality as its maximum multiplicity plus from_minimum and to_minimum as zero or one.
Join keys use one string on both sides or paired arrays of equal length, and record field names only without primary-key, foreign-key, or uniqueness roles.
Relationship confidence is one of confirmed, inferred, or assumed, and every relationship records its basis.
Classification uses the privacy-standards citation-field names. The catalog does not invent standards identifiers.
Column statistics and feature metadata remain behind profile_ref rather than being copied into the catalog.
Every customer-facing catalog ends with the canonical Data Science and Engineering Coaching disclaimer footer.

Constraints

Treat catalog enrichment as user-driven. Offer enrichment when a source is missing, but do not change the catalog silently.
Record source locations as paths or connection references, never embedded credentials.
Keep DCAT alignment non-binding. The catalog is native YAML, not RDF, and makes no DCAT conformance claim.
Keep tier recording separate from tier behavior. data-catalog records the tier; dataops owns what that tier means.
Keep rendering separate from semantic authority. Diagram tools consume declared relationships and do not infer new ones.
End every customer-facing catalog with the canonical Data Science and Engineering Coaching disclaimer from the disclaimer-language skill.

Stop rules

Stop and request clarification when an entity grain, relationship endpoint, join-key pairing, or endpoint minimum is ambiguous.
Stop and retain inferred or assumed confidence when evidence does not support confirmed.
Stop before a durable write when the caller has not confirmed the destination.
Stop and route privacy interpretation to privacy-standards or the Privacy Planner when citation values or DPIA status are unknown.

Package resources

Resource	Use
catalog-contract.md	Read for the authoritative field, identity, multiplicity, and body-generation rules
dcat-crosswalk.md	Read when planning a DCAT, DCAT-AP, or catalog-platform export
provenance.md	Read for standards selection, licensing, and non-conformance boundaries
ds-catalog-v1.md	Copy when starting a new catalog
northwind-catalog.md	Read as a complete valid example with scalar and composite join keys
`assets/ds-catalog-v1.schema.json`	Use as the structural JSON Schema for parsed frontmatter
`scripts/validate_catalog.py`	Execute with `uv run python scripts/validate_catalog.py <catalog.md>` to validate a catalog

Attribution

The native contract, workflow, schema, template, examples, and validator are repository-original content licensed CC BY 4.0.

The crosswalk paraphrases selected concepts from W3C DCAT 3, W3C PROV-O, and Frictionless Table Schema v2. Standard names and term identifiers are factual citations, the crosswalk prose and the paired-array join-key contract are independently authored, and no upstream schema, example, table, or substantial excerpt is reproduced. Because no upstream expression is redistributed, the package remains solely CC BY 4.0. See provenance.md.

Files

ATTRIBUTION.mdAdded by AutoGineer975 B
LICENSEAdded by AutoGineer16.6 KiB
SKILL.md6.1 KiB
assets/ds-catalog-v1.schema.json5.9 KiB
examples/northwind-catalog.md8.4 KiB
pyproject.toml698 B
references/catalog-contract.md11.1 KiB
references/dcat-crosswalk.md5.4 KiB
references/provenance.md3.1 KiB
scripts/validate_catalog.pyScript15.0 KiB
templates/ds-catalog-v1.md5.3 KiB
tests/corpus/0_valid_frontmatter90 B
tests/corpus/1_empty_frontmatter8 B
tests/corpus/2_unclosed_sequence20 B
tests/corpus/3_duplicate_key24 B
tests/corpus/4_no_frontmatter15 B
tests/corpus/README.md1.2 KiB
tests/fuzz_harness.pyScript1.2 KiB
tests/test_validate_catalog.pyScript24.0 KiB
uv.lock107.7 KiB

references/dcat-crosswalk.md

title: DS_CATALOG_V1 DCAT crosswalk description: Non-binding mapping from the native catalog contract to selected DCAT, DCTERMS, and PROV terms

Status

This crosswalk supports future export design. DS_CATALOG_V1 is YAML, while DCAT conformance requires an RDF description. The mapping does not rename native fields, vendor an upstream shape, or claim DCAT, DCAT-AP, DCMI, or PROV conformance.

Mapping

Native field	Nearest external term	Fidelity	Export note
Catalog document	`dcat:Catalog`	exact	One engagement catalog maps to one catalog resource
`engagement`	`dcterms:title`	exact	Catalog title
`generated_at`	`dcterms:issued`	exact	Initial publication time
`last_enriched`	`dcterms:modified`	exact	Latest enrichment time
`entities[]`	`dcat:Dataset`	exact	One entity maps to one dataset resource
`entities[].id`	IRI and `dcterms:identifier`	exact	Export must mint a stable IRI
`entities[].name`	`dcterms:title`	exact	Customer-readable name
`entities[].description`	`dcterms:description`	exact	Description
`entities[].source.system`	`dcterms:source` or `dcterms:publisher`	partial	Source system is not always a publishing organization
`entities[].source.location`	`dcat:accessURL` or `dcat:downloadURL`	partial	Requires a distribution and may not be a URL
`entities[].source.format`	`dcterms:format` or `dcat:mediaType`	exact	Applies to a distribution
`entities[].source.access_confirmed`	No direct term	none	Engagement-operational state remains native
`entities[].tier`	No direct term	none	DataOps tier remains native
`entities[].grain`	No direct term	none	Retain in description or an export extension
`entities[].volume.period_covered`	`dcterms:temporal`	exact	Temporal coverage
`entities[].volume.update_frequency`	`dcterms:accrualPeriodicity`	exact	Native readable name remains unchanged
`entities[].profile_ref`	`dcat:qualifiedRelation` with an export role	partial	Related artifact pointer
`entities[].classification.sensitivity`	`dcterms:accessRights`	partial	Export requires an appropriate controlled vocabulary
`entities[].lineage.derived_from`	`prov:wasDerivedFrom`	exact	Provenance relation
`entities[].lineage.transform_ref`	`prov:wasGeneratedBy`	partial	Native value is a code reference rather than a PROV activity
`relationships[]`	`dcat:qualifiedRelation` and `dcat:Relationship`	partial	Association-node pattern fits, but payload fields remain native
Relationship endpoints	`dcterms:relation`	partial	Export profile must define endpoint roles
Relationship cardinality, keys, confidence	No direct term	none	Load-bearing native fields require an export profile or extension
`coverage`	No direct term	none	Engagement progress remains native

The from_minimum and to_minimum endpoint values share the cardinality row: they are load-bearing native facts with no direct external term, so an export profile or extension must carry them.

Sources

W3C, Data Catalog Vocabulary (DCAT) Version 3, W3C Recommendation, 2024-08-22
W3C, PROV-O: The PROV Ontology, W3C Recommendation, 2013-04-30
DCMI, Metadata Terms, DCMI Recommendation

Requirements and compatibility

Written for: GitHub Copilot, Claude Code

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Claude Code frontmatter, Needs Python

Security

What our scanner noticed· Low risk

Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

Microsoft

Publisher

Microsoft (Official)

Source

microsoft/hve-core · .github/skills/data-science-engineering/data-catalog · commit f425e94

Licence

Creative Commons Attribution 4.0 International (redistribution allowed)

LICENSE (written by AutoGineer)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download data-catalog.zip