> For the complete documentation index, see [llms.txt](https://docs.sdv.dev/sdv/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sdv.dev/sdv/integration/metadata/metadata-json.md). # Metadata JSON This guide describes the metadata JSON spec.

An example of multi table data. The tables are connected to each other through primary/foreign keys.

Click to see the metadata JSON file This is an example of a JSON file describing a multi table schema. ```json { "METADATA_SPEC_VERSION": "V2", "tables": { "hotels": { "primary_key": "hotel_id", "columns": { "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" }, "city": { "sdtype": "city", "range_is_nullable": false }, "rating": { "sdtype": "numerical", "range_min": 0, "range_max": 5, "decimal_places": 1 } }, "column_relationships": [] }, "guests": { "primary_key": "guest_email", "columns": { "guest_email": { "sdtype": "email", "range_is_nullable": false }, "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" }, "checkin_date": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "01 Jan 1970", "range_max": "31 Dec 2050" }, "checkout_date": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "01 Jan 1970", "range_max": "31 Dec 2050" }, "room_type": { "sdtype": "categorical" } }, "column_relationships": [] } }, "relationships": [{ "parent_table_name": "hotels", "parent_primary_key": "hotel_id", "child_table_name": "guests", "child_foreign_key": "hotel_id" }] } ```
{% hint style="success" %} **Create your metadata programmatically.** Use the [Python API ](broken://pages/70dzL3ZTDgj8HBQH5gTc)to automatically detect the metadata based on your data. {% endhint %} ## Overview The metadata for a single table contains the following elements: * (required) `"METADATA_SPEC_VERSION"`: The version of the metadata. If you are using SDV 2.0, the metadata version will be `"V2"` * (required) `"tables"`: A dictionary that maps the table names to the table-specific metadata such as primary keys, column names and data types * (required) `"relationships"`: A list of dictionaries that specify the connections between the tables ## Tables The tables dictionary maps each table name to the table-specific metadata. This includes: * (required) `"columns"`: A dictionary that maps the column names to the data types they represent and any other attributes. * `"primary_key"`: The column name that is the primary key in the table. This must be an ID or PII sdtype.\ ＊ *If the table has a composite key, this will be a list of column names instead; in this case, at least 1 of the columns must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.* * `"alternate_keys"`: A list of column names that can act as alternate keys in the table\ ＊ *If the table has a composite key, this will be a list of a list of column names instead; in this case, at least 1 of the columns must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.* ### Table Columns When describing a column, you will provide the column name and the data type, known as the **sdtype**. The common sdtypes are: `"categorical"`, `"ordinal"`, `"numerical"`, `"datetime"`, `"boolean"` and `"id"`. Click on the type below to learn more about the type and how to specify it in the metadata. {% tabs %} {% tab title="categorical" %} Categorical columns represent discrete, unordered data. ```json { "tax_payer_status": { "sdtype": "categorical", "range_values": ["single", "married_filing_jointly", "married_filing_separately", "head_of_household", "surviving_spouse"], "range_is_nullable": false }, "tax_location_code": { "sdtype": "categorical", "high_cardinality": true } } ``` **Properties:** * `range_values`: A list of all possible non-null values that can appear in the categorical column. Because categorical data is unordered, this list can appear in any order. *This list may include category values that are not present in your training data.* * (default) If this is not provided, then SDV assumes that the full list of possible values is not known. It may look at your training data to determine the comprehensive list. * `high_cardinality`: A true/false value describing whether the categorical column has a large number of possible category values that cannot be fully and meaningfully listed in `range_values`. We recommend setting this when there are >500 unique values. If this is set to true, then the `range_values` should not be present. * `range_is_nullable`: Whether the missing values are allowed in the column. * (default) If this is not provided, then SDV looks at your training data as the main example that determines whether nulls should be allowed. {% endtab %} {% tab title="ordinal" %} Ordinal columns represent discrete, ordered data. ```json { "likert_responses": { "sdtype": "ordinal", "range_values": ["strongly_disagree", "disagree", "neither_agree_nor_disagree", "agree", "strongly_agree"] } } ``` **Properties:** * `range_values`: A list of all possible non-null values that can appear in the ordinal column. Because ordinal data is ordered, this list should appear in order from lowest to highest. *This list may include ordinal values that are not present in your training data.* * (default) If this is not provided, then SDV assumes that the full list of possible values is not known. It may look at your training data to determine the comprehensive list. * `high_cardinality`: A true/false value describing whether the ordinal column has a large number of possible values that cannot be fully and meaningfully listed in `range_values`. We recommend setting this when there are >500 unique values. If this is set to true, then the `range_values` should not be present. * `range_is_nullable`: Whether the missing values are allowed in the column. * (default) If this is not provided, then SDV looks at your training data as the main example that determines whether nulls should be allowed. {% endtab %} {% tab title="numerical" %} Numerical columns represents continuous numerical ranges of values. ```json { "age": { "sdtype": "numerical", "range_min": 18, "range_max": 120, "decimal_places": 0 }, "transaction_amt": { "sdtype": "numerical", "range_is_nullable": false, "decimal_places": 2 } } ``` **Properties** * `range_min`: The absolute lowest value that is possible for this column. *This might be lower than what your training data contains.* * (default) If this is not provided, then SDV assumes that the lowest value is not known. It may look at your training data to determine the lowest value. * `range_max`: The absolute highest value that is possible for this column. *This might be higher than what your training data contains.* * (default) If this is not provided, then SDV assumes that the highest value is not known. It may look at your training data to determine the lowest value. * `range_is_nullable`: Whether the missing values are allowed in the column. * (default) If this is not provided, then SDV looks at your training data as the main example that determines whether nulls should be allowed. * `decimal_places`: An integer representing the number of decimal places that the numerical values should be rounded to, for example `2` for 2 decimal digits. Note that a designation of `0` could mean either that there are whole numbers (integers) or that all values contain `.0` at the end such as `1.0`, `2.0`, etc. {% endtab %} {% tab title="datetime" %} Date columns represent a point in time ```json { "start_date": { "sdtype": "datetime", "datetime_format": "%Y-%m-%d", "range_min": "1970-01-01", "range_max": "2050-12-31" } } ``` **Properties** * (required) `datetime_format`: A string describing the format as defined by [Python's strftime module](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes). * `range_min`: The absolute earliest datetime that is possible for this column. *This might be earlier than what your training data contains.* * (default) If this is not provided, then SDV assumes that the earliest value is not known. It may look at your training data to determine the earliest value. * `range_max`: The absolute latest value that is possible for this column. *This might be later than what your training data contains.* * (default) If this is not provided, then SDV assumes that the latest value is not known. It may look at your training data to determine the latest value. * `range_is_nullable`: Whether the missing values are allowed in the column. * (default) If this is not provided, then SDV looks at your training data as the main example that determines whether nulls should be allowed. {% endtab %} {% tab title="id" %} ID columns represent identifiers that do not have any special mathematical or semantic meaning ```json "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" } ``` **Properties** * `regex_format`: A string describing the format of the ID as a [regular expression](https://docs.python.org/3/library/re.html) {% endtab %} {% tab title="boolean" %} Boolean columns represent True or False values. ```json "is_active" : { "sdtype": "boolean" } ``` **Properties**: * `range_is_nullable`: Whether the missing values are allowed in the column. * (default) If this is not provided, then SDV looks at your training data as the main example that determines whether nulls should be allowed. {% endtab %} {% tab title="other" %} You can input any other data type such as `'phone_number'`, `'ssn'` or `'email'`. See the [Sdtypes Reference](/sdv/integration/metadata/sdtypes.md#conceptual-sdtypes) for a full list. ```json "guest_email": { "sdtype": "email", "pii": true } ``` **Properties** * `pii`: A boolean denoting whether the data is sensitive * (default) `true`: The column is sensitive, meaning the values should be anonymized * `false`: The column is not sensitive, meaning the exact set of values can be reused in the synthetic data * `range_is_nullable`: Whether the missing values are allowed in the column. * (default) If this is not provided, then SDV looks at your training data as the main example that determines whether nulls should be allowed. {% endtab %} {% endtabs %} ### Column Relationships Annotate groups of columns that represents higher level concepts. Denote the concept using the `"relationship_type"` keyword, followed by `"column_names"` with the list of columns involved. The column names can be present in any order. Each relationship type supports different types of columns. Browse the table below to explore different options. {% tabs %} {% tab title="address" %} An address is defined by 2 or more columns that have the following sdtypes: `country_code`, `administrative_unit`, `state`, `state_abbr`, `city`, `postcode`, `street_address` and `secondary_address`. ```python { "type": "address", "column_names": ["addr_line1", "addr_line2", "city", "state", "zipcode"] } ``` {% endtab %} {% tab title="gps" %} A GPS coordinate pair is defined by 2 columns: * sdtype `latitude` & * sdtype `longitude` ```python { "type": "gps", "column_names": ["location_lat", "location_lon"] } ``` {% endtab %} {% tab title="More coming soon!" %} Additional column relationships coming soon! *Do you have a request for a type of column relationship? Please* [*file a feature request*](https://github.com/sdv-dev/SDV/issues/new/choose) *describing your use case.* {% endtab %} {% endtabs %} {% hint style="info" %} While anyone can add column relationships to their data, SDV Enterprise users will see the highest quality data for the relationships. To learn more about the SDV Enterprise and its extra features, [visit our website](https://datacebo.com/pricing/). {% endhint %} ## Relationships A list of dictionary objects that describe the relationship between 2 connected tables, parent and child. The parent table contains the primary key references while the child table has rows that refer to its parent. Multiple child rows can refer to the same parent row. * `"parent_table_name"`: The name of the parent table * `"parent_primary_key"`: The primary key column in the parent table. This column uniquely identifies each row in the parent table. The column must be an ID or another PII sdtype.\ ＊ *If the table has a composite key, this will be a list a list of column names instead; in this case, at least 1 of the columns must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.* * `"child_table_name"`: The name of the child table that refers to the parent * `"child_foreign_key"`: The foreign key column in the child table. The values in this column contain a reference to a row in the parent table\ ＊ *If the table has a composite key, this will be a list of column names instead. The length of this list should match the parent primary key. Only SDV Enterprise users can create synthesizers with composite keys.* Use multiple dictionaries to represent multiple relationships. ```json "relationships": [{ "parent_table_name": "users", "parent_primary_key": "user_id", "child_table_name": "sessions", "child_foreign_key": "user_id" }, { "parent_table_name": "sessions", "parent_primary_key": "session_id", "child_table_name": "transaction", "child_foreign_key": "transacted_session_id" }] ``` ## Multi Sequence Data In some cases, you may have a table that describes multiple, ordered sequences of data such as the one shown below.

An example of sequential data. There are multiple sequences (one for each Patient ID). Within each sequence is an ordered set of rows.

You can annotate the sequences within a table by applying the `sequence_key` and `sequence_index` keywords.
Click to see the metadata JSON ```json { "METADATA_SPEC_VERSION": "V2", "tables": { "patients": { "sequence_key": "Patient ID", "sequence_index": "Time", "columns": { "Patient ID": { "sdtype": "id", "regex_format": "ID_[0-9]{3}" }, "Address": { "sdtype": "physical_address", "pii": true, "range_is_nullable": false }, "Smoker": { "sdtype": "boolean" }, "Time": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "Jan 01 1970", "range_max": "Dec 31 2050" }, "Heart Rate": { "sdtype": "categorical", "range_values": ["LOW", "RESTING", "ELEVATED"] }, "Systolic BP": { "sdtype": "numerical", "range_min": 50, "range_max": 200 } } } } } ```
**`"sequence_key"`**: A column name of the sequence key, if you have multi-sequence data {% hint style="info" %} The **sequence key** is a column that identify which row(s) belong to which sequences. This is usually an ID column but it may also be a PII sdtype (such as `"phone_number"`). This is important for tables that contain multiple sequences. In our example, the sequence key is `'Patient ID'` because this column is used to break up the sequences. If you don't supply a sequence key, the SDV assumes that your table only contains a single sequence. *Note: The SDV sequential models do not fully support single sequence data.* {% endhint %} **`"sequence_index"`**: A column name of the sequence index, if you have sequential data {% hint style="info" %} The **sequence index** determines the spacing between the rows in a sequence. Use this if you have an explicit index such as a timestamp. If you don't supply a sequence index, the SDV assumes there is equal spacing of an unknown unit. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.sdv.dev/sdv/integration/metadata/metadata-json.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.