# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
This site has an agentic ask interface you may use to query the documentation dynamically by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.sdv.dev/sdv/index.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

---

# Welcome to the SDV!

The **Synthetic Data Vault** (SDV) is your one-stop shop for creating tabular synthetic data.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FOBG7qwnU9TkQQpIwga5i%2FSDV%20Intro.png?alt=media&amp;token=3bf4ade2-2943-44d8-8048-bbc1fba6d690" alt=""><figcaption></figcaption></figure>

## Key Features

:brain: **Train your own generative AI model.** Choose from a variety of AI algorithms designed for tabular data — single table, sequential, or multi-table (relational) data. Train your own synthesizer using your real data, and create any amount of synthetic data on-demand. SDV is designed to work on-prem, with standard CPUs.

:bar\_chart: **Evaluate & visualize synthetic data.** Measure the statistical quality of your synthetic data and diagnose problems. For even more insight, create visualizations that compare your synthetic data with your real data.

:gear: **Customize your synthesizer.** The SDV platform offers powerful features for creating higher quality synthetic data. You can add constraints, adjust the data preprocessing, and selecting anonymization options for any SDV synthesizer.

## Take synthetic data to the next level with SDV Enterprise

**SDV Enterprise** is available to licensed users. With SDV Enterprise, you'll have access to everything in SDV Community *plus the ability to ...*

:white\_check\_mark: Create synthetic data for large numbers of complex, interconnected data tables using scalable synthesizers

:white\_check\_mark: Improve the quality of your synthetic data with more advanced data preprocessing, deeper data understanding, and enhanced AI algorithms

:white\_check\_mark: Easily integrate data sources and deploy synthetic data applications enterprise-wide

To learn more, [**visit our website**](https://datacebo.com/pricing/).

{% hint style="info" icon="user" %}
SDV Enterprise is available to licensed users. Sign up now to get your license key. You can then complete your SDV Enterprise installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

## Owned & Maintained by DataCebo

The SDV library is a part of the greater [Synthetic Data Vault Project](https://sdv.dev/), first created at MIT's Data to AI Lab in 2016. After 4 years of research and traction with enterprise, we created DataCebo in 2020 with the goal of growing the project.

Today, [DataCebo](https://datacebo.com/) is the proud developer of the SDV, the largest ecosystem for synthetic data generation & evaluation.


# Get Started

## How it works?

SDV is designed to learn statistical patterns from your training data and then emulate those patterns when creating synthetic data. The patterns are saved in a synthesizer object. You can think of your synthesizer as a model that contains a condensed representation of your data.

{% stepper %}
{% step %}

### Data Integration

Load your data into SDV. You can connect your database or load in data from a local file. Then, create a metadata object that contains a description of your data for SDV to use.\
[More Information → ](/sdv/integration/overview)
{% endstep %}

{% step %}

### Modeling

Create a synthesizer and use it to learn patterns from your training data. Pick from a variety of synthesizers that incorporate various different ML modeling techniques.\
[More Information → ](/sdv/modeling/single-table-synthesizers)
{% endstep %}

{% step %}

### Sampling

Use your synthesizer to generate realistic synthetic data on-demand. Or supply instructions to your synthesizer to create specific types of data. When you're done, export the synthetic data into a different file or database.\
[More Information→](/sdv/sampling/sample-realistic-data)&#x20;
{% endstep %}

{% step %}

### Evaluation

Evaluate the synthetic data by comparing it against the real data. Check for basic diagnostic stats, compare the statistical quality, and visualize any differences.\
[More Information→](/sdv/evaluation/diagnostic)&#x20;
{% endstep %}
{% endstepper %}

## Tutorials

Use SDV Community to explore synthetic data, and create synthesizers for simple, proof-of-concept datasets.&#x20;

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td><a href="https://colab.research.google.com/drive/1MCTkTj9-93Ei-cLDQoj9AXaqPhpue7a3?usp=sharing"><strong>Synthesize a Table (Gaussian Copula)</strong></a></td><td>A fast, customizable and transparent way to synthesize data.</td><td><a href="https://colab.research.google.com/drive/1MCTkTj9-93Ei-cLDQoj9AXaqPhpue7a3?usp=sharing">https://colab.research.google.com/drive/1MCTkTj9-93Ei-cLDQoj9AXaqPhpue7a3?usp=sharing</a></td><td></td></tr><tr><td><a href="https://colab.research.google.com/drive/15iom9fO8j_gHg4-NlGkzWF5thMWStXwv?usp=sharing"><strong>Synthesize a Table (CTGAN)</strong></a></td><td>A GAN-based approach to creating synthetic data with high fidelity.</td><td><a href="https://colab.research.google.com/drive/15iom9fO8j_gHg4-NlGkzWF5thMWStXwv?usp=sharing">https://colab.research.google.com/drive/15iom9fO8j_gHg4-NlGkzWF5thMWStXwv?usp=sharing</a></td><td></td></tr><tr><td><a href="https://colab.research.google.com/drive/1YLk2uwn8yrSRPy0soEeJwu8Hdk_tGTlE?usp=sharing"><strong>Synthesize Sequences (PAR)</strong></a></td><td>A neural network-based approach to creating synthetic data.</td><td><a href="https://colab.research.google.com/drive/1YLk2uwn8yrSRPy0soEeJwu8Hdk_tGTlE?usp=sharing">https://colab.research.google.com/drive/1YLk2uwn8yrSRPy0soEeJwu8Hdk_tGTlE?usp=sharing</a></td><td></td></tr><tr><td><a href="https://colab.research.google.com/drive/1L6i-JhJK9ROG-KFcyzT9G-8FC3L8y8Lc?usp=sharing"><strong>Synthesize Multi Tables (HMA)</strong></a></td><td>Synthesize data across multiple, connected tables in a database.</td><td><a href="https://colab.research.google.com/drive/1L6i-JhJK9ROG-KFcyzT9G-8FC3L8y8Lc?usp=sharing">https://colab.research.google.com/drive/1L6i-JhJK9ROG-KFcyzT9G-8FC3L8y8Lc?usp=sharing</a></td><td></td></tr><tr><td><a href="https://colab.research.google.com/drive/1DlUlHNohxBhxN6GA5C_C4Fd9wx5AR11Y?usp=sharing"><strong>Use Your Own Data</strong></a></td><td>Prepare your own data for SDV:  Load your raw data and write a metadata description</td><td><a href="https://colab.research.google.com/drive/1DlUlHNohxBhxN6GA5C_C4Fd9wx5AR11Y?usp=sharing">https://colab.research.google.com/drive/1DlUlHNohxBhxN6GA5C_C4Fd9wx5AR11Y?usp=sharing</a></td><td></td></tr></tbody></table>

### Customizations

Using SDV Community, you can control the full workflow when preprocessing data, anonymizing sensitive values, adding constraints and more.

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="https://colab.research.google.com/drive/1Z9Rff5G3eDI-LEc1LXiew9E3hF5Ob-on?usp=sharing"><strong>Custom Processing &#x26; Anonymization</strong></a></td><td>Improve your synthesizer by customizing the data processing workflow.</td><td><a href="https://colab.research.google.com/drive/1Z9Rff5G3eDI-LEc1LXiew9E3hF5Ob-on?usp=sharing">https://colab.research.google.com/drive/1Z9Rff5G3eDI-LEc1LXiew9E3hF5Ob-on?usp=sharing</a></td></tr><tr><td><a href="https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing"><strong>Constraint-Augmented Generation</strong></a></td><td>Input business rules into your synthesizer using constraints. This ensures high-quality, valid synthetic data, 100% of the time.</td><td><a href="https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing">https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing</a></td></tr></tbody></table>


# SDV Community

Use **SDV Community** as an educational resource to learn about synthetic data and create it for small applications.

## Access the powerful SDV platform

SDV Community uses features throughout SDV platform ecosystem. The platform includes a suite of libraries and features that work together to form a one-stop shop for your synthetic data needs. You can also browse and use the platform features in a standalone way.

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="https://docs.sdv.dev/rdt"><strong>RDT</strong></a></td><td>Preprocess and anonymize your data using reversible data transformers.</td><td><a href="https://docs.sdv.dev/rdt">https://docs.sdv.dev/rdt</a></td></tr><tr><td><a href="https://docs.sdv.dev/sdmetrics"><strong>SDMetrics</strong></a></td><td>Evaluate synthetic data for quality and privacy. Visualize &#x26; share results.</td><td><a href="https://docs.sdv.dev/sdmetrics">https://docs.sdv.dev/sdmetrics</a></td></tr><tr><td><a href="https://docs.sdv.dev/sdgym"><strong>SDGym</strong></a></td><td>Benchmark synthetic data generators across SDV and other libraries.</td><td><a href="https://docs.sdv.dev/sdgym">https://docs.sdv.dev/sdgym</a></td></tr></tbody></table>

## Installation

Get started with the publicly available [**SDV Community**](/sdv/explore/sdv-community), distributed under the [Business Source License](https://github.com/sdv-dev/SDV/blob/main/LICENSE).&#x20;

SDV Community is available as a Python SDK that you can install and use on-prem. It is distributed under the [Business Source License](https://github.com/sdv-dev/SDV/blob/main/LICENSE), and has been developed on Python [3.9-3.14](https://www.python.org/). As with most Python libraries, we recommend using a virtual environment (such as [virtualenv](https://virtualenv.pypa.io/en/latest/)) to avoid conflicts with other software on your device.

We recommend downloading SDV Community using [pip](https://pypi.org/project/sdv/) (or alternatively [conda](https://anaconda.org/conda-forge/sdv)).

```bash
pip install sdv
```

Then, open up Python and verify that SDV has installed correctly.

```python
import sdv
print(sdv.version.community)
```

SDV Community is great for exploring the benefits of synthetic data. Train a generative AI with your own, simple datasets as a proof-of-concept. Create synthetic data that has the same patterns.

```python
import pandas as pd
from sdv.single_table import GaussianCopulaSynthesizer
from sdv.metadata import Metadata

data_table = pd.read_csv('users.csv')
data = { 'users': data_table }

metadata = Metadata.detect_from_dataframes(data)

synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.fit(data)
synthetic_data = synthesizer.sample(num_rows=1000)
```

**Get started now!** Check out the SDV Community [installation guide](/sdv/explore/sdv-community#installation) and [tutorials](/sdv/get-started).

For more information about the latest version of SDV Community, see the [Release Notes](https://github.com/sdv-dev/SDV/releases).

{% hint style="info" %}
**Having trouble?** Visit our [**troubleshooting section**](/sdv/support/troubleshooting/installation) to diagnose any issues. You can also ask a question on our [forum](https://forum.datacebo.com/).
{% endhint %}

## What's next?

Use SDV Community as an educational resource to learn more about synthetic data and generative relational models. Check out the resources below to learn more:

* [Cookbooks](https://cookbooks.datacebo.com/), notebooks that showcase how to use various SDV features
* [DataCebo Blog](https://datacebo.com/blog/), where we write about important synthetic data developments, our vision, and product updates
* [DataCebo Forum](https://forum.datacebo.com/), for keeping up with the latest releases and answering technical questions

For enterprise use cases, we recommend upgrading to our commercial product, SDV Enterprise. [Compare the Features](/sdv/explore/compare-features) and [Create an Account](https://portal.datacebo.com/login) to get your SDV Enterprise license key.


# SDV Enterprise

Please check out our SDV Enterprise site here: <https://docs.sdv.dev/sdv-enterprise>


# SDV Bundles

SDV Enterprise users have the option to purchase optional add-ons called **SDV Bundles**. Bundles allow you to access additional features for specific needs.

{% hint style="info" icon="user" %}
Bundles are available to licensed users. Sign up now to get your license key. You can then complete your installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="/sdv/explore/sdv-bundles/ai-connectors"><strong>AI Connectors</strong></a></td><td>Directly integrate SDV with your database. Includes metadata, data import, and synthetic data export.</td><td><a href="/sdv/explore/sdv-bundles/ai-connectors">❖ AI Connectors</a></td></tr><tr><td><a href="/sdv/explore/sdv-bundles/cag"><strong>CAG</strong></a></td><td>Use Constraint Augmented Generation (CAG) to apply complex logic between multiple tables.</td><td><a href="/sdv/explore/sdv-bundles/cag">❖ CAG</a></td></tr><tr><td><a href="/sdv/explore/sdv-bundles/differential-privacy"><strong>Differential Privacy</strong></a></td><td>Transform and create synthetic data with differential privacy guarantees.</td><td><a href="/sdv/explore/sdv-bundles/differential-privacy">❖ Differential Privacy</a></td></tr><tr><td><a href="/sdv/explore/sdv-bundles/targeted-sampling"><strong>Targeted Sampling</strong></a></td><td>Create targeted synthetic data by fixing values and conditional sampling across tables.</td><td><a href="/sdv/explore/sdv-bundles/targeted-sampling">❖ Targeted Sampling</a></td></tr><tr><td><a href="/sdv/explore/sdv-bundles/xsynthesizers"><strong>XSynthesizers</strong></a></td><td>Take your gen AI models to the next level with enhanced synthesizers and transformers.</td><td><a href="/sdv/explore/sdv-bundles/xsynthesizers">❖ XSynthesizers</a></td></tr></tbody></table>


# ❖ AI Connectors

Use the **AI Connectors bundle** to directly connect your database to SDV. This bundle offers a seamless integration with SDV synthesizers, with features that are specifically designed for generative AI.

**⭐️ Detect metadata based on the database schema.** Database schemas contain details about the column names and types of connections between tables. AI Connectors use this information to create [Metadata](/sdv/integration/metadata), which is a higher level specification needed for generative AI.

**⭐️ Import a data subsample that is AI-ready.** The data you input into a generative AI needs to be high quality. For example, it should have high coverage and it should not contain broken links. AI Connectors import a data subsample that is ready for AI.

**⭐️ Export synthetic data in the same format as the original.** Many users want to export synthetic data into a new database for downstream use. AI Connectors ensure that the exported database has the same format as the original database, allowing you to easily access the synthetic data in downstream applications.

{% hint style="info" icon="user" %}
Bundles are available to licensed users. Sign up now to get your license key. You can then complete your installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

## Included Features

This bundle includes AI Connectors for a variety of different databases.

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="image">Cover image</th></tr></thead><tbody><tr><td align="center">❖ <a href="/sdv/integration/db/integration/alloydb"><strong>AlloyDB</strong></a></td><td><a href="/sdv/integration/db/integration/alloydb">❖ AlloyDB</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FkJ1bOR4AbOLLD1SkiFzg%2FAlloyDB.png?alt=media&amp;token=cf944e36-a8d7-473f-9938-633d50a770fd">AlloyDB.png</a></td></tr><tr><td align="center">❖<a href="/sdv/integration/db/integration/bigquery"> <strong>BigQuery</strong></a></td><td><a href="/sdv/integration/db/integration/bigquery">❖ BigQuery</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FuYpWlTYYH1CYOr07wzfG%2FBigQuery.png?alt=media&amp;token=cd7b2e45-cd8b-4132-a06f-5518ce84f829">BigQuery.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/mssql"><strong>MSSQL</strong></a></td><td><a href="/sdv/integration/db/integration/mssql">❖ MSSQL</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FPHBms1a4bQgSdGQKCAoJ%2FMSSQL.png?alt=media&amp;token=a6f2021f-1cfa-4b09-9243-0173db44fc4a">MSSQL.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/oracle"><strong>Oracle</strong></a></td><td><a href="/sdv/integration/db/integration/oracle">❖ Oracle</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FZfj0JCHAUEh35s1qaTsk%2FOracle.png?alt=media&amp;token=a8ba5d36-54ef-476c-9557-86a6ecf676f8">Oracle.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/postgresql"><strong>PostgreSQL</strong></a></td><td><a href="/sdv/integration/db/integration/postgresql">❖ PostgreSQL</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FaoQYOyetcdahpAvL3aAS%2FPostgreSQL.png?alt=media&amp;token=4ad8e042-3ffd-4af7-98b4-584ac6191884">PostgreSQL.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/spanner"><strong>Spanner</strong></a></td><td><a href="/sdv/integration/db/integration/spanner">❖ Spanner</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FYlYJaZhTyVM24IriuOns%2FSpanner.png?alt=media&amp;token=6f0a8473-7699-42fa-9783-358ee372bedf">Spanner.png</a></td></tr></tbody></table>

## Installation

Use your SDV Enterprise credentials to install the AI Connectors bundle with the database that you'd like to use (see [the docs](/sdv/integration/overview#connect-to-a-database)).

```bash
% pip install sdv-installer --upgrade
% sdv-installer install --package bundle-ai-connectors --options db-bigquery db-mssql
```

```
Username: <email>
License Key: ********************************

Installing Bundles:
bundle-ai-connectors - Installed!

Success! All packages have been installed. You are ready to use SDV Enterprise.
```

{% hint style="info" %}
For more information about installing SDV Enterprise features and bundles, see our [**Enterprise Installation Guide**](https://docs.sdv.dev/sdv-enterprise/sdv-enterprise/installation).
{% endhint %}


# ❖ CAG

Constraint Augmented Generation (CAG) is a powerful system that allows you to input business logic into your complex schemas. Fast, easy, and flexible, these constraints allow you to create synthetic data that conforms to your logic, 100% of the time.

:star: **Automatically discover business logic & data lineage properties.** CAG can auto-detect commonly occurring business rules in your dataset and recommend constraints to address them.&#x20;

**⭐️ Apply constraints across multiple tables.** Add business logic that determines the connections between tables, or the type of data that linked tables are allowed to have.

**⭐️ Access complex, powerful algorithms with simple APIs.** CAG supports constraints that are more complex, often incorporating different algorithms to get you the data you need. The best part? There's one, simple API to denote your constraint.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2Fp2u3pw3gU0KZSrX5pJWK%2FCAG-bundle_March%2018%202025.png?alt=media&amp;token=e6d0fecb-b3ad-4f09-8eb2-95def3ce9fda" alt=""><figcaption><p>In this example, only Accounts with Type=PREMIUM are allowed to have associated Purchases. Achieve this with the <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/foreigntoprimarykeysubset">ForeignToPrimaryKeySubset</a> constraint.</p></figcaption></figure>

{% hint style="info" %}
Bundles are available to licensed users. Sign up now to get your license key. You can then complete your installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

## Included Features

**Predefined constraints**: This bundle includes an expansive suite of constraints that contain advanced, multi-table logic & algorithms: They include [CarryOverColumns](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/carryovercolumns), [DenormalizedTable](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/denormalizedtable), [PolymorphicRelationship](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/polymorphic-relationship), and more. This bundle also includes access to some advanced single-table constraints.

:star: [**Click to browse all constraints**](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints) :star:

**Constraint auto-detection**: This bundle allows you to [automatically detect predefined constraints](/sdv/modeling/constraint-augmented-generation-cag/auto-detect-constraints), inspect them, and add them to your synthesizer.

**Support for programmable constraints**: Your database may occasionally contain additional logic that cannot be represented by any of the predefined constraints. In this case, our team provides support to meet your business rules.

## Installation

Use your SDV Enterprise credentials to install SDV Enterprise and all bundles that you have access to.

```bash
% pip install sdv-installer --upgrade
% sdv-installer install --upgrade
```

```
Username: <email>
License Key: ********************************

Installing SDV Enterprise:
sdv-enterprise (version 0.30.0) - Installed!

Installing Bundles:
bundle-cag - Installed!
bundle-xsynthesizers - Installed!

Success! All packages have been installed. You are ready to use SDV Enterprise.
```

{% hint style="info" %}
For more information about installing SDV Enterprise features and bundles, see our [**Enterprise Installation Guide**](https://docs.sdv.dev/sdv-enterprise/sdv-enterprise/installation).
{% endhint %}


# ❖ Differential Privacy

The **Differential Privacy** bundle allows you to create synthetic data that is private, according to methods that are backed by [mathematically-rigorous findings](https://en.wikipedia.org/wiki/Differential_privacy). The differential privacy framework enforces a limit on how much one individual record can affect the synthesizer — and ultimately leak into the synthetic data.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FhOJXpELfKcWEqPHAzkrB%2Fdifferential-privacy_June%2004%202025.png?alt=media&amp;token=44c72e38-4353-48bd-b2fa-7acdca3ba3ef" alt=""><figcaption><p>The differential privacy framework enforces a limit on how much 1 individual record (row of data, outlined in orange) can affect what the synthesizer learns (dots, colored in orange).</p></figcaption></figure>

:bar\_chart: **Share your synthetic data broadly.** Our differential privacy synthesizers guarantee that a single row of data will not unduly affect the patterns that the synthesizer learns. We use [ε-differential privacy](https://en.wikipedia.org/wiki/Differential_privacy#%CE%B5-differential_privacy), which allows you to provide a privacy loss budget, ε (epsilon). This budget allows you to control the privacy/quality tradeoffs.

:star: **Upscale your synthetic data.** Once you've fit your synthesizer, use it to create any size of differentially-private synthetic data — even 10x or 100x the original size. Privacy guarantees apply to all data your synthesizer creates.

:floppy\_disk: **Save and share your synthesizer.** Save and load in your synthesizer to sample more synthetic data at any time. No real data or sensitive statistics are saved in the synthesizer, so you can share it without worry.

{% hint style="info" icon="user" %}
Bundles are available to licensed users. Sign up now to get your license key. You can then complete your installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

## Included Features

**Synthesizers for generating differentially-private data.**

* The [**DPGCSynthesizer**](/sdv/modeling/single-table-synthesizers/dpgcsynthesizer) creates differentially private data using the GaussianCopula method
* The experimental [**DPGCFlexSynthesizer**](/sdv/modeling/single-table-synthesizers/dpgcflexsynthesizer) runs a similar method, but offers more flexibility in the data pre-processing that you can use

**Preprocessing methods for generating differentially private columns.**

Under-the-hood, the synthesizers use preprocessing techniques for generating differentially private columns of data. You can apply these transformers in a standalone way.

* Noise the column using differential privacy: [**DPLaplaceNoiser**](https://docs.sdv.dev/rdt/transformers-glossary/numerical/dplaplacenoiser), [**DPTimestampLaplaceNoiser**](https://docs.sdv.dev/rdt/transformers-glossary/datetime/dptimestamplaplacenoiser), [**DPResponseRandomizer**](https://docs.sdv.dev/rdt/transformers-glossary/categorical/dpresponserandomizer), [**DPWeightedResponseRandomizer**](https://docs.sdv.dev/rdt/transformers-glossary/categorical/dpweightedresponserandomizer)
* Normalize the column into numerical data of a specific shape, using differential privacy: [**DPECDFNormalizer**](https://docs.sdv.dev/rdt/transformers-glossary/numerical/dpecdfnormalizer), [**DPDiscreteECDFNormalizer**](https://docs.sdv.dev/rdt/transformers-glossary/categorical/dpdiscreteecdfnormalizer)

**Verify the differential privacy.**

Use the [**differential privacy evaluation tool** ](/sdv/evaluation/privacy/empirical-differential-privacy)to empirically measure the differential privacy of a synthesizer algorithm on a given dataset. Use this with any SDV single-table synthesizer.

## Installation

Use your SDV Enterprise credentials to install SDV Enterprise and all bundles that you have access to.

```bash
% pip install sdv-installer --upgrade
% sdv-installer install --upgrade
```

```
Username: <email>
License Key: ********************************

Installing SDV Enterprise:
sdv-enterprise (version 0.30.0) - Installed!

Installing Bundles:
bundle-cag - Installed!
bundle-xsynthesizers - Installed!

Success! All packages have been installed. You are ready to use SDV Enterprise.
```

{% hint style="info" %}
For more information about installing SDV Enterprise features and bundles, see our [**Enterprise Installation Guide**](https://docs.sdv.dev/sdv-enterprise/sdv-enterprise/installation).
{% endhint %}


# ❖ Targeted Sampling

The **Targeted Sampling** bundle allows you to create synthetic data that is specific and targeted to your usage. You can use targeted sampling to:

:star: **Create minimum viable synthetic test sets** for software testing and QA. Define the exact values that you need across multiple tables and let SDV handle the rest.

:star: **De-bias and rebalance your datasets**. Adjust the proportions of values to generate in the synthetic data to create a more balanced, fairer representation of the data you'd like to use for ML development.

:star: **Extrapolate hypothetical scenarios.** Create edge cases and situations that don't exist in the real world yet, but are theoretically possible to observe. Using this bundle, you can extrapolate outside of the boundaries of your real training data to create a wider variety of synthetic data.

{% hint style="info" icon="user" %}
Bundles are available to licensed users. Sign up now to get your license key. You can then complete your installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

## Included Features

Key features of the Targeted Sampling bundle are:

* [**Specifying ranges for extrapolation**](/sdv/modeling/configuration/range-extrapolation)**.** Before training a synthesizer, you can specify the full range of values you'd like to see in the synthetic data — even if those ranges are outside the boundaries of your training data. This allows you to extrapolate and create new, hypothetical scenarios.
* [**Conditional sampling**](/sdv/sampling/conditional/multi-table)**.** After training your synthesizer, you can provide exactly the conditions you wish to create. The rest of the synthetic data will adapt to those conditions.

## Installation

Use your SDV Enterprise credentials to install SDV Enterprise and all bundles that you have access to.

```bash
% pip install sdv-installer --upgrade
% sdv-installer install --upgrade
```

```
Username: <email>
License Key: ********************************

Installing SDV Enterprise:
sdv-enterprise (version 0.30.0) - Installed!

Installing Bundles:
bundle-cag - Installed!
bundle-xsynthesizers - Installed!

Success! All packages have been installed. You are ready to use SDV Enterprise.
```


# ❖ XSynthesizers

Go the eXtra mile with your synthesis. Use enhanced synthesizers and transformers for **improved synthetic data quality and performance**.

{% hint style="info" icon="user" %}
Bundles are available to licensed users. Sign up now to get your license key. You can then complete your installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

## Included Features

This bundle includes versions of synthesizers and transformers that have been expanded with additional features.

**Enhanced synthesizers for higher quality synthetic data.** Use this to model your real data and create synthetic data.

* [**XGCSynthesizer**](/sdv/modeling/single-table-synthesizers/xgcsynthesizer) is expanded version of the [GaussianCopulaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) that includes modeling support for 100+ additional column shapes. Use this single-table synthesizer in combination with multi-table synthesizers such as [HSASynthesizer](/sdv/modeling/multi-table-synthesizers/hsasynthesizer).
* [**SegmentSynthesizer**](/sdv/modeling/single-table-synthesizers/segmentsynthesizer) is a specialty synthesizer that you can use for a table of highly segmented data.
* [**BootstrapSynthesizer**](/sdv/modeling/single-table-synthesizers/bootstrapsynthesizer), designed specially to learn from data with only a few rows, or "short and wide" data containing more columns than rows.

**Enhanced transformers for more accurate data preprocessing.** Use these as a preprocessing step with any SDV synthesizer.

* [**XGaussianNormalizer**](https://docs.sdv.dev/rdt/transformers-glossary/numerical/xgaussiannormalizer) is an expanded version of the [GaussianNormalizer](https://docs.sdv.dev/rdt/transformers-glossary/numerical/gaussiannormalizer) transformer. This also includes modeling support for 100+ additional column shapes. Use this transformer with any of the SDV synthesizers.
* [**ECDFNormalizer**](https://docs.sdv.dev/rdt/transformers-glossary/numerical/ecdfnormalizer) provides another way to normalize your data by direct, empirical computation. With this transformer, there is no need to choose a predefined column shape. Use this transformer with any of the SDV synthesizers.

## Installation

Use your SDV Enterprise credentials to install SDV Enterprise and all bundles that you have access to.

```bash
% pip install sdv-installer --upgrade
% sdv-installer install --upgrade
```

```
Username: <email>
License Key: ********************************

Installing SDV Enterprise:
sdv-enterprise (version 0.30.0) - Installed!

Installing Bundles:
bundle-cag - Installed!
bundle-xsynthesizers - Installed!

Success! All packages have been installed. You are ready to use SDV Enterprise.
```

{% hint style="info" %}
For more information about installing SDV Enterprise features and bundles, see our [**Enterprise Installation Guide**](https://docs.sdv.dev/sdv-enterprise/sdv-enterprise/installation).
{% endhint %}


# Compare Features

Compare the features available across SDV Community and SDV Enterprise. SDV Enterprise users also have the option of purchasing [**SDV Bundles**](/sdv/explore/sdv-bundles), which are optional add-on packages for targeted needs.

{% hint style="info" icon="user" %}
SDV Enterprise is available to licensed users. Sign up now to get your license key. You can then complete your SDV Enterprise installation on-prem and begin using the features.

<p align="center"><a href="https://portal.datacebo.com/signup" class="button primary">Get Started</a><a href="https://datacebo.com/pricing/" class="button secondary">View Pricing</a></p>
{% endhint %}

### AI-Based Synthesizers

These synthesizers use AI to learn patterns from your data and use them to recreate synthetic data.

<table data-search="false"><thead><tr><th width="400">Feature</th><th width="160" align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td><a href="/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer"><strong>GaussianCopula</strong></a> <em>statistical AI</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/ctgansynthesizer"><strong>CTGAN</strong></a>, <a href="/sdv/modeling/single-table-synthesizers/tvaesynthesizer"><strong>TVAE</strong></a>, <a href="/sdv/modeling/single-table-synthesizers/copulagansynthesizer"><strong>CopulaGAN</strong></a> <em>neural networks</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/xgcsynthesizer"><strong>XGC</strong></a> <em>advanced Copula modeling with flexible shapes, faster runtime and more</em></td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/xsynthesizers"> XSynthesizers bundle</a></td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/segmentsynthesizer"><strong>SegmentSynthesizer</strong></a> <em>for separately modeling highly segmented data</em></td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/xsynthesizers">XSynthesizers bundle</a></td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/bootstrapsynthesizer"><strong>BootstrapSynthesizer</strong></a> <em>for modeling data with a few rows</em></td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/xsynthesizers">XSynthesizers bundle</a></td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/dpgcsynthesizer"><strong>DPGC</strong></a>, and <a href="/sdv/modeling/single-table-synthesizers/dpgcflexsynthesizer"><strong>DPGCFlex</strong></a> Synthesizers <em>for creating synthetic data with differential privacy guarantees</em></td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/differential-privacy">Differential Privacy bundle</a></td></tr><tr><td><a href="/sdv/modeling/sequential-synthesizers/parsynthesizer"><strong>PAR</strong></a> <em>for sequential data</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/multi-table-synthesizers/hmasynthesizer"><strong>HMA</strong></a> <em>multi-table for limited tables (&#x3C;5)</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/multi-table-synthesizers/hsasynthesizer"><strong>HSA</strong></a> <em>multi-table for unlimited tables</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/multi-table-synthesizers/independentsynthesizer"><strong>Independent</strong></a> <em>multi-table for unlimited tables</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/multi-table-synthesizers/performance-estimates"><strong>Performance estimates</strong></a> <em>for multi-table synthesizers with various dataset sizes</em></td><td align="center">❌</td><td align="center">✅</td></tr></tbody></table>

### Test Data Synthesizers

These synthesizers create random test data based on metadata alone. They do not use AI so you do not need to input any training data.

<table><thead><tr><th width="400">Feature</th><th align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td><a href="/sdv/modeling/single-table-synthesizers/dayzsynthesizer"><strong>DayZSynthesizer</strong></a> <em>single table</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/multi-table-synthesizers/dayzsynthesizer"><strong>DayZSynthesizer</strong></a> <em>multi table</em></td><td align="center">❌</td><td align="center">✅</td></tr></tbody></table>

### Synthetic Data Creation

Create synthetic data that matches the patterns in your real data — as well as other specifications that you have for your project.

<table><thead><tr><th width="400">Feature</th><th align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td><a href="/sdv/sampling/sample-realistic-data"><strong>Sample realistic data</strong></a> that matches the patterns in your real data, and scale it up.</td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/sampling/conditional/single-table"><strong>Single-table conditional sampling</strong></a> for fixed values in single-table and sequential datasets</td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/sampling/conditional"><strong>Multi-table conditional sampling</strong></a> for fixed values in multi-table datasets.</td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/targeted-sampling">Targeted Sampling bundle</a></td></tr><tr><td><a href="/sdv/modeling/configuration/range-extrapolation"><strong>Extrapolate synthetic data</strong></a> outside the ranges of your original training data.</td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/targeted-sampling">Targeted Sampling bundle</a></td></tr></tbody></table>

### Data Integrations

These features make it easy to integrate SDV into your application and pipeline.

<table data-search="false"><thead><tr><th width="400">Feature</th><th width="163" align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td><a href="/sdv/integration/local"><strong>Load data from a local file</strong></a> (CSV, Excel, or other tabular formats)</td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/integration/metadata"><strong>Auto-detect metadata</strong></a> using data CSVs or DataFrames</td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/sampling/export#write-locally"><strong>Write synthetic data into a local file</strong></a> (CSV, Excel or other tabular formats)</td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/integration/metadata/metadata-api#detect_from_dataframes"><strong>Auto-detect metadata</strong>.</a> Advanced algorithms that use data to detect primary and foreign keys more accurately.</td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="/sdv/integration/metadata/metadata-api#detect_from_dataframes"><strong>Auto-detect Regex formats</strong></a> for all ID columns, as well as <a href="/sdv/integration/metadata/sdtypes#id"><strong>embedded context</strong></a> within those IDs.</td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="/sdv/integration/metadata/metadata-api#set_primary_key"><strong>Specify composite keys</strong></a>. Specify primary and foreign keys that consist of 2 or more columns combined.</td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="/sdv/integration/db"><strong>Directly connect to a database</strong></a> for importing real data and creating metadata</td><td align="center">❌</td><td align="center">💠  <a href="/sdv/explore/sdv-bundles/ai-connectors">AI Connectors bundle</a></td></tr><tr><td><a href="/sdv/integration/db"><strong>Auto-detect metadata</strong></a> based on your database</td><td align="center">❌</td><td align="center">💠  <a href="/sdv/explore/sdv-bundles/ai-connectors">AI Connectors bundle</a></td></tr><tr><td> <a href="/sdv/sampling/export#export-to-a-database"><strong>Export synthetic data</strong></a> to a database</td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/ai-connectors">  AI Connectors bundle</a></td></tr></tbody></table>

### Pre-Process Statistical Information&#x20;

Transformers are used to pre-process your data, which can improve data quality. SDV synthesizers select transformers by default, but you can always customize these to your dataset.

<table data-search="false"><thead><tr><th width="400">Feature</th><th width="158" align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/floatformatter"><strong>FloatFormatter</strong></a> <em>for missing value imputation, numerical columns</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/clusterbasednormalizer"><strong>ClusterBasedNormalizer</strong></a> and <a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/gaussiannormalizer"><strong>GaussianNormalizer</strong></a> <em>statistical transforms</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/xgaussiannormalizer"><strong>XGaussianNormalizer</strong></a> <em>with support for 100+ statistical distributions</em></td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/xsynthesizers">XSynthesizers bundle</a></td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/ecdfnormalizer"><strong>ECDFNormalizer</strong></a> <em>to normalize any distribution with high fidelity</em></td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/xsynthesizers">XSynthesizers bundle</a></td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/categorical/uniformencoder"><strong>Uniform</strong></a>, <a href="https://docs.sdv.dev/rdt/transformers-glossary/categorical/labelencoder"><strong>Label</strong></a>, and <a href="https://docs.sdv.dev/rdt/transformers-glossary/categorical/onehotencoder"><strong>OneHot</strong></a> Encoding <em>for discrete variables (</em><a data-footnote-ref href="#user-content-fn-1"><em>nominal</em></a> <em>and</em> <a data-footnote-ref href="#user-content-fn-2"><em>ordinal</em></a><em>)</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/datetime"><strong>Datetime</strong></a> Encoding <em>including datetime format parsing, and converting timezones</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/datetime/unixtimestampencoder#examples"><strong>Learning timezones</strong></a> <em>that are attached to your datetime column</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/outlierencoder"><strong>OutlierEncoder</strong></a> <em>for numerical outliers</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/dplaplacenoiser"><strong>DPLaplaceNoiser</strong></a>, <a href="https://docs.sdv.dev/rdt/transformers-glossary/datetime/dptimestamplaplacenoiser"><strong>DPTimestampLaplaceNoiser</strong></a>, <a href="https://docs.sdv.dev/rdt/transformers-glossary/categorical/dpresponserandomizer"><strong>DPResponseRandomizer</strong></a>, <a href="https://docs.sdv.dev/rdt/transformers-glossary/categorical/dpweightedresponserandomizer"><strong>DPWeightedResponseRandomizer</strong></a> <em>for adding noise to a column to guarantee differential privacy</em></td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/differential-privacy"> Differential Privacy bundle</a></td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/numerical/dpecdfnormalizer"><strong>DPECDFNormalizer</strong></a>, <a href="https://docs.sdv.dev/rdt/transformers-glossary/categorical/dpdiscreteecdfnormalizer"><strong>DPDiscreteECDFNormalizer</strong></a> <em>for normalizing a column while guaranteeing differential privacy</em></td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/differential-privacy"> Differential Privacy bundle</a></td></tr></tbody></table>

### Understand & Anonymize Real-World Concepts

Transformers are used to pre-process your data, which can improve data quality. SDV synthesizers select transformers by default, but you can always customize these to your dataset.

These transformers are geared towards columns that correspond to industry or domain-specific concepts. Their structure may be human-created.

<table data-search="false"><thead><tr><th width="400">Feature</th><th width="169" align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/text-id/regexgenerator"><strong>RegexGenerator</strong></a>, <a href="https://docs.sdv.dev/rdt/transformers-glossary/text-id/idgenerator"><strong>IDGenerator</strong></a> <em>for keys and IDs</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/pii/anonymizedfaker"><strong>AnonymizedFaker</strong> </a><em>general-purpose anonymization</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/pii/pseudoanonymizedfaker"><strong>PseudoAnonymizedFaker</strong></a> <em>for general pseudo-anonymization with a mapping</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/deep-data-understanding/email"><strong>Emails</strong></a> <em>understanding domains</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/deep-data-understanding/address"><strong>Addresses</strong></a> <em>understanding locations</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/deep-data-understanding/phone-number"><strong>Phone Numbers</strong></a> <em>understanding country and area codes</em></td><td align="center">❌</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/rdt/transformers-glossary/deep-data-understanding/gps-coordinates"><strong>GPS Coordinates</strong> </a><em>understanding  geographical areas and distances</em> </td><td align="center">❌</td><td align="center">✅</td></tr></tbody></table>

### Constraint-Augmented Generation

Input business rules into your synthesizer using constraints. This ensures high-quality, valid synthetic data, 100% of the time.

<table><thead><tr><th width="400">Feature</th><th width="167" align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td>Predefined logic for individual tables: <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixedincrements"><strong>FixedIncrements</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixedcombinations"><strong>FixedCombinations</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/inequality"><strong>Inequality</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/onehotencoding"><strong>OneHotEncoding</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/range"><strong>Range</strong></a></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/columnformula"><strong>Program your own constraint</strong></a> for single tables</td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/cag"> CAG bundle</a></td></tr><tr><td>Advanced predefined logic for individual tables: <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixednullcombinations"><strong>FixedNullCombinations</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/mixedscales"><strong>MixedScales</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/chainedinequality"><strong>ChainedInequality</strong></a></td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/cag"> CAG bundle</a></td></tr><tr><td>Advanced, predefined logic for multi-table tables: <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/carryovercolumns"><strong>CarryOverColumns</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/denormalizedtable"><strong>DenormalizedTable</strong></a>, <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/polymorphic-relationship"><strong>PolymorphicRelationship</strong></a>, and <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints"><strong>more</strong></a>.</td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/cag">CAG bundle</a></td></tr><tr><td>Automatically <a href="/sdv/modeling/constraint-augmented-generation-cag/auto-detect-constraints"><strong>detect predefined constraint logic</strong></a> and apply it to your synthesizer.</td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/cag">CAG bundle</a></td></tr><tr><td><a href="https://forum.datacebo.com/"><strong>Support for additional constraints</strong></a> that are not yet available as predefined logic</td><td align="center">❌</td><td align="center">💠 <a href="/sdv/explore/sdv-bundles/cag">CAG bundle</a></td></tr></tbody></table>

### Synthetic Data Evaluation

Evaluate your synthetic data by comparing it against the real data.

<table data-search="false"><thead><tr><th width="400">Feature</th><th width="165" align="center">SDV Community</th><th align="center">SDV Enterprise</th></tr></thead><tbody><tr><td>Access to <a href="https://docs.sdv.dev/sdmetrics/"><strong>SDMetrics</strong></a> library <em>vendor-agnostic, open source</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/sdmetrics/reports/diagnostic-report"><strong>Diagnostic Report</strong></a> <em>basic data validity checks , single and multi-table</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="https://docs.sdv.dev/sdmetrics/reports/quality-report"><strong>Quality Report</strong> </a><em>statistical similarity, single and multi-table</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td>Measure the privacy of your data: <a href="https://docs.sdv.dev/sdmetrics/metrics/metrics-glossary/disclosureprotection"><strong>DisclosureProtection</strong></a> and <a href="https://docs.sdv.dev/sdmetrics/metrics/metrics-glossary/disclosureprotection/disclosureprotectionestimate"><strong>DisclosureProtectionEstimate</strong></a></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td><a href="/sdv/evaluation/privacy/empirical-differential-privacy"><strong>Verify the differential privacy</strong></a> <em>of any synthesizer algorithm</em></td><td align="center">❌</td><td align="center">💠<a href="/sdv/explore/sdv-bundles/differential-privacy"> Differential Privacy bundle</a></td></tr><tr><td><a href="https://docs.sdv.dev/sdmetrics/reports/visualization-utilities"><strong>Visualization</strong></a> <em>1D and 2D bars, scatterplots, heatmaps and more</em></td><td align="center">✅</td><td align="center">✅</td></tr><tr><td>Use case-specific metrics: <a href="https://docs.sdv.dev/sdmetrics/metrics/metrics-in-beta/outliercoverage"><strong>OutlierCoverage</strong></a>, <a href="https://docs.sdv.dev/sdmetrics/metrics/metrics-in-beta/smoothnesssimilarity"><strong>SmoothnessSimilarity</strong></a></td><td align="center">❌</td><td align="center">✅</td></tr></tbody></table>

[^1]: Categories without any order, such as ice cream flavor or gender

[^2]: Categories with an order such as school grades ("C+", "B-", "B", etc.)&#x20;


# ❖ SDV Agent

**SDV Agent** allows you to create synthetic data using a chat-based interface powered by LLMs. SDV Agent teams up with the powerful SDV ecosystem. With just a few, quick prompts, the agent understands your synthetic data needs and forwards them to SDV. The agent works with SDV's sampling engine to produce accurate, comprehensive synthetic data for your applications.

💬 **A chat-based approach to synthetic data.** SDV Agent allows you to set up and create synthetic data using natural language. Simply tell the agent what you need in English, and the agent will figure out how to use SDV to achieve your request. There's no need to code or use the Python API.

🧠 **Smart suggestions coming straight from SDV.** SDV Agent is aware of the full SDV ecosystem. It's able to check your prompts, proactively suggesting alternate ideas and corrections if needed. No more time wasted in troubleshooting Python commands.

🌎 **Leverage the knowledge of LLMs.** SDV Agent has the full power of an LLM behind it. Use this to learn more about synthetic data, understand database concepts, or search through similar, publicly available projects as guidance. LLMs are also aware of commonly used industry terms and practices.

## Included Features

{% hint style="warning" %}
**This functionality is in Beta.** At this time, select SDV Enterprise users are able to use this feature and provide feedback.
{% endhint %}

SDV Agent is delivered as an **MCP server**, with hooks that allow an LLM to execute SDV Enterprise functions for detecting metadata, training a synthesizer, creating synthetic data, evaluating it, and more. You can also use this to help you double-check your metadata, add constraints, and identify features for optimizing your synthesizer.

## Installation

Use your SDV Enterprise credentials to install SDV Agent and SDV Enterprise. The agent will work with SDV Enterprise to create high quality synthetic data.

```bash
% pip install sdv-installer --upgrade
% sdv-installer install --upgrade
```

```
Username: <email>
License Key: ********************************

Installing SDV Enterprise:
sdv-enterprise (version 0.30.0) - Installed!

Installing Bundles:
sdv-agent - Installed!

Success! All packages have been installed. You are ready to use SDV Enterprise.
```

{% hint style="info" %}
For more information about installing SDV Enterprise features and bundles, see our [**Enterprise Installation Guide**](https://docs.sdv.dev/sdv-enterprise/sdv-enterprise/installation).
{% endhint %}

## Integrate with a Front-End

You can connect the MCP server to the front-end service of your choice. Popular examples include [Cursor](https://cursor.com/docs/context/mcp), [Claude Desktop](https://support.claude.com/en/articles/10949351-getting-started-with-local-mcp-servers-on-claude-desktop), [Visual Studio](https://learn.microsoft.com/en-us/visualstudio/ide/mcp-servers?view=visualstudio) and more! For more information, see the [Model Context Protocol docs](https://modelcontextprotocol.io/docs/getting-started/intro).

### Cursor

We've tested the MCP Server integration with Cursor. To get started, install [Cursor](https://cursor.com/docs) to use as your IDE. Then, click on the Cursor Settings > Tools & MCP and click to add a new MCP Server. Provide the path to your installed SDV Agent package in the configuration below:

```json
{
  "mcpServers": {
    "sdv_agent": {
      "command": <path>
    }
  }
}
```

{% hint style="info" %}
**What is the path?** This is a local file path that points to the SDV Agent package you've installed. You can run the following command on your terminal to print this out.

```bash
% which sdv-agent
```

This will print out a path such as:

```
/Users/username/Documents/SDV/env/bin/sdv-agent
```

You can then input this as your path:

```json
{
  "mcpServers": {
    "sdv_agent": {
      "command": "/Users/username/Documents/SDV/env/bin/sdv-agent"
    }
  }
}
```

{% endhint %}

If successfully configured, you should see that the MCP tools are enabled and ready to use. The SDV Agent is then able to use the tools to directly forward your requests to SDV. *If you're having trouble with this step, try toggling the server on and off first.*

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FwfxLwF6Aj2PpoJkSgKcd%2Fsdv_agent_screenshot.png?alt=media&amp;token=7e272e5b-9d88-49e2-a88b-7cde4630a3f5" alt=""><figcaption></figcaption></figure>


# Overview

In order to use SDV, you'll need to start with two inputs loaded into Python:

* **Data**, which is the training data for SDV. SDV learns statistical patterns from your training data and aims to replicate them when it creates synthetic data. SDV supports single-table, multi-table, and sequential data. For more details, see the [Data Modalities](/sdv/integration/overview/data-modalities) doc.
* **Metadata**, which is a description of the dataset you want to synthesize. Metadata includes the names of tables, columns, data types in each of the columns, and relationships between tables. Metadata is represented as a Python dictionary.

Once you have your data and metadata, you'll be able to use SDV's synthesizers to create synthetic data.

## Start Your Integration

For the best experience, we recommend connecting your database to import data and metadata. Alternatively if you already have data locally (eg. in CSV files), you can load it into Python.

### ❖ Connect Your Database

To start your data integration, we recommend **Connecting Your Database** to SDV.&#x20;

{% stepper %}
{% step %}

#### Connect Your Database

Use the **❖ AI Connectors** feature to create a connection from SDV to your database.
{% endstep %}

{% step %}

#### Create Metadata

SDV automatically creates metadata based on your database schema. You can view and edit the metadata.
{% endstep %}

{% step %}

#### Import Data

SDV automatically imports training data by selecting a subset of your data from your database. The imported subset is ready-to-use with SDV.
{% endstep %}
{% endstepper %}

For more information see the [Connect Your Database](/sdv/integration/db) docs.

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

### Import Local Data

Alternatively, you can **load in data from local files** such as CSVs.

{% stepper %}
{% step %}

#### Load Your Data

Load your data directly into Python using SDV's local data loading utilities. We recommend starting with CSV or Excel files.
{% endstep %}

{% step %}

#### Auto-Detect Metadata

SDV is able to auto-detect metadata based on your data. We recommend reviewing it for accuracy. You can view and edit the metadata.
{% endstep %}

{% step %}

#### Clean Your Data

In some cases, you may need to further clean your data. For example, you may need to drop any unknown references to ensure that your dataset has referential integrity. Or you may want to subsample it for performance.
{% endstep %}
{% endstepper %}

&#x20;For more information, see the [Import Local Data](/sdv/integration/local) docs.

## FAQ

<details>

<summary>Do you have any example data that I can use for testing?</summary>

Yes, the SDV library contains many different demo datasets that you can use to get started. You'll be able to download the data and metadata for each of the demo datasets.

To learn more, see the [SDV Demo Data](/sdv/integration/utils/demo) docs.

</details>

<details>

<summary>If I don't have any data, can I generate synthetic data from scratch?</summary>

Yes, we have a specialized synthesizer called DayZSynthesizer that is able to generate data from scratch. You only have to provide metadata to describe the structure of the data you want to generate. The DayZSynthesizer is available in [SDV Enterprise](/sdv/explore/sdv-enterprise) for [single-table](/sdv/modeling/single-table-synthesizers/dayzsynthesizer) and [multi-table](/sdv/modeling/multi-table-synthesizers/dayzsynthesizer) datasets.

Note that SDV has been primarily designed to learn from training data and incorporate those patterns in the synthetic data. With the DayZSynthesizer, you'll receive data that is randomly generated without any specific patterns or correlations.

</details>


# Data Modalities

SDV is designed to work on tabular data, where each data table contains rows and columns. SDV supports single-table, multi-table, and sequential data modalities.

## Single Table Data

Single table data contains rows and columns of information. Each row typically represents a new entity such as a user, transaction, or session. Each column represents an attribute of an entity. It should contain the same type of data.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FrXVqtxd5XrixdqYhtyZb%2Fsingle-table-data-data-preparation_May%208%202025.png?alt=media&amp;token=65cf97c2-1da4-476f-8edf-ab0bb8c1b42b" alt=""><figcaption><p>This example of a single table includes a new row for each guest of a hotel.</p></figcaption></figure>

{% hint style="info" %}
SDV supports missing values in single table data.&#x20;

Starting from SDV 2.0, single table data should be loaded into Python as a dictionary. The dictionary should map the table name (string) to a pandas DataFrame object containing the data for that table.

```python
my_data = {
    'guests': <guests DataFrame>
}
```

{% endhint %}

**Key Concepts:**

* Primary key, a column that uniquely identifies every row in the table.

## Multi Table Data

Multi table data is present in multiple tables that each have rows and columns. The tables are connected to each other through foreign and primary key references.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FpPLDP2gOJBQvF5yF6Zmz%2Fmulti-table-data-data-preparation_May%208%202025.png?alt=media&amp;token=f23c6cac-43db-4e08-9eaf-5c850aacecfc" alt=""><figcaption><p>This example of a multi table dataset has a table for hotels and a table for their guests. Each hotel has multiple guests who have visited.</p></figcaption></figure>

**Key Concepts:**

* Primary key, a column that uniquely identifies every row in the table.
* Foreign key, a column that references another primary key column. Typically the reference is to another table's primary key, but it's possible to reference the same table.
* Alternate key, a column that is unique for every row, but is not the primary key.
* Composite key, when multiple columns together act as a primary, foreign, or alternate key to the table.

{% hint style="info" %}
Multi-table data can contain relationships. These are references from one table (foreign key) to another table (primary key). SDV requires *referential integrity*, meaning that  there should be no unknown references or broken links between the tables.

Multi-table data should be loaded into Python as a dictionary. The dictionary should map each table name (string) to a pandas DataFrame object containing the data for that table.

```python
my_data = {
    'hotels': <hotels DataFrame>,
    'guests': <guests DataFrame>
}
```

{% endhint %}

### Sequential Data

Sequential data represents ordered records, such as in a timeseries.  SDV is designed for the case where your table may contain multiple, independent sequences belonging to different entities (users, patients, etc.).

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FzPsbsXbMULfrTaOTilK3%2Fsequential-data-data-preparation_May%208%202025.png?alt=media&amp;token=478612f6-1474-4ae8-99fa-dfec4304b96d" alt=""><figcaption><p>This example shows sequential data related to vital signs. The table contains multiple sequences, each corresponding to a different patient. For each sequences, health measurements change over time.</p></figcaption></figure>

{% hint style="info" %}
The main difference between single-table and sequential data is that in single-table data, the rows do not have an order. In sequential data, you have multiple ordered sequences.

Starting from SDV 2.0, sequential table data should be loaded into Python as a dictionary. The dictionary should map the table name (string) to a pandas DataFrame object containing the data for that table.

```python
my_data = {
    'patients': <patients DataFrame>
}
```

{% endhint %}

**Key Concepts:**

* Primary key, a column that uniquely identifies every row in the table. Sequential data may or may not have a primary key.
* Sequence key, a column that uniquely identifies each multi-row sequence in the table. In our example, the sequence key is `Patient ID`.
* Sequence index, a column that determines the order of the rows. This is typically an index or timestamp. Some types of sequential data may not have a sequence index.


# Connect Your Database

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

We recommend connecting SDV to your database for the smoothest experience. After integration, SDV is automatically able to create metadata based on the database schema and import data to be used for training an SDV synthesizer.

```python
from sdv.io.database import AlloyDBConnector

# 1. Integration: Create a connector and set your config
connector = AlloyDBConnector()
connector.set_import_config('source_db_config.yaml')
connector.set_export_config('destination_db_config.yaml')

# 2. Usage: Use the API create metadata, import training data, and export synthetic data
metadata = connector.create_metadata(table_names=['users', 'transactions', 'sessions'])
data = connector.import_random_subset(metadata=metadata)
...
connector.export(synthetic_data)
```

## ❖ Browse Databases

Use the AI Connectors feature to directly connect to your database and import your data into SDV. Later you can use the same connector to export synthetic data into a new database.

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="image">Cover image</th></tr></thead><tbody><tr><td align="center">❖ <a href="/sdv/integration/db/integration/alloydb"><strong>AlloyDB</strong></a></td><td><a href="/sdv/integration/db/integration/alloydb">❖ AlloyDB</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FkJ1bOR4AbOLLD1SkiFzg%2FAlloyDB.png?alt=media&amp;token=cf944e36-a8d7-473f-9938-633d50a770fd">AlloyDB.png</a></td></tr><tr><td align="center">❖<a href="/sdv/integration/db/integration/bigquery"> <strong>BigQuery</strong></a></td><td><a href="/sdv/integration/db/integration/bigquery">❖ BigQuery</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FuYpWlTYYH1CYOr07wzfG%2FBigQuery.png?alt=media&amp;token=cd7b2e45-cd8b-4132-a06f-5518ce84f829">BigQuery.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/mssql"><strong>MSSQL</strong></a></td><td><a href="/sdv/integration/db/integration/mssql">❖ MSSQL</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FPHBms1a4bQgSdGQKCAoJ%2FMSSQL.png?alt=media&amp;token=a6f2021f-1cfa-4b09-9243-0173db44fc4a">MSSQL.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/oracle"><strong>Oracle</strong></a></td><td><a href="/sdv/integration/db/integration/oracle">❖ Oracle</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FZfj0JCHAUEh35s1qaTsk%2FOracle.png?alt=media&amp;token=a8ba5d36-54ef-476c-9557-86a6ecf676f8">Oracle.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/postgresql"><strong>PostgreSQL</strong></a></td><td><a href="/sdv/integration/db/integration/postgresql">❖ PostgreSQL</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FaoQYOyetcdahpAvL3aAS%2FPostgreSQL.png?alt=media&amp;token=4ad8e042-3ffd-4af7-98b4-584ac6191884">PostgreSQL.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/spanner"><strong>Spanner</strong></a></td><td><a href="/sdv/integration/db/integration/spanner">❖ Spanner</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FYlYJaZhTyVM24IriuOns%2FSpanner.png?alt=media&amp;token=6f0a8473-7699-42fa-9783-358ee372bedf">Spanner.png</a></td></tr></tbody></table>

{% hint style="success" %}
**Request additional databases.** If your database provider is not covered by the list above, please [leave us a request](https://forum.datacebo.com/). If approved our team is able to add a connector for a relational database in a few weeks.&#x20;

*We prioritize requests made by customers who are using AI Connectors.*
{% endhint %}


# ❖ Integration

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

To integrate with your database, create a *connector* object based on your database flavor. Then add config files to your connector, which include information like credentials, database names, and other runtime parameters.

```python
from sdv.io.database import AlloyDBConnector

connector = AlloyDBConnector()
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

After this intitial integration, your connector is ready to use for importing training data, creating metadata, and exporting synthetic data.

## ❖ Supported Databases

Click your database below to get started with the integration. If your database provider is not covered by the list, please [leave us a request](https://forum.datacebo.com/). If approved, our team is able to add a connector for a relational database in a few weeks.&#x20;

*We prioritize requests made by customers who are using AI Connectors.*

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="image">Cover image</th></tr></thead><tbody><tr><td align="center">❖ <a href="/sdv/integration/db/integration/alloydb"><strong>AlloyDB</strong></a></td><td><a href="/sdv/integration/db/integration/alloydb">❖ AlloyDB</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FkJ1bOR4AbOLLD1SkiFzg%2FAlloyDB.png?alt=media&amp;token=cf944e36-a8d7-473f-9938-633d50a770fd">AlloyDB.png</a></td></tr><tr><td align="center">❖<a href="/sdv/integration/db/integration/bigquery"> <strong>BigQuery</strong></a></td><td><a href="/sdv/integration/db/integration/bigquery">❖ BigQuery</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FuYpWlTYYH1CYOr07wzfG%2FBigQuery.png?alt=media&amp;token=cd7b2e45-cd8b-4132-a06f-5518ce84f829">BigQuery.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/mssql"><strong>MSSQL</strong></a></td><td><a href="/sdv/integration/db/integration/mssql">❖ MSSQL</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FPHBms1a4bQgSdGQKCAoJ%2FMSSQL.png?alt=media&amp;token=a6f2021f-1cfa-4b09-9243-0173db44fc4a">MSSQL.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/oracle"><strong>Oracle</strong></a></td><td><a href="/sdv/integration/db/integration/oracle">❖ Oracle</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FZfj0JCHAUEh35s1qaTsk%2FOracle.png?alt=media&amp;token=a8ba5d36-54ef-476c-9557-86a6ecf676f8">Oracle.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/postgresql"><strong>PostgreSQL</strong></a></td><td><a href="/sdv/integration/db/integration/postgresql">❖ PostgreSQL</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FaoQYOyetcdahpAvL3aAS%2FPostgreSQL.png?alt=media&amp;token=4ad8e042-3ffd-4af7-98b4-584ac6191884">PostgreSQL.png</a></td></tr><tr><td align="center">❖ <a href="/sdv/integration/db/integration/spanner"><strong>Spanner</strong></a></td><td><a href="/sdv/integration/db/integration/spanner">❖ Spanner</a></td><td><a href="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FYlYJaZhTyVM24IriuOns%2FSpanner.png?alt=media&amp;token=6f0a8473-7699-42fa-9783-358ee372bedf">Spanner.png</a></td></tr></tbody></table>


# ❖ AlloyDB

This page describes how to integrate with your AlloyDB database.

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

## Installation

To use this feature, please make sure you have installed the bundle with the optional `db-alloydb` dependency. For more information, see the [SDV Enterprise Installation docs](https://docs.sdv.dev/sdv-enterprise/installation/instructions).

```bash
pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-alloydb
```

## Connector

&#x20;Create a connector object that interacts with your AlloyDB database.

```python
from sdv.io.database import AlloyDBConnector

connector = AlloyDBConnector()
```

**Parameters** (None)

**Output** An AlloyDBConnector object that you can use to import data and metadata

## Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

```yaml
connection:
  database_name: <string> 
  schema_name: <string>

auth:
  json_credentials_path: <filepath>

options.AlloyDBConnector:
  region: <region_value>
  cluster: <cluster_value>
  instance_id: <id_value>
  ip_type: <PRIVATE or PUBLIC>
```

**Parameters**:

**`connection`**: This section defines the database tables you want to connect to. It is common to all Google-native databases.

* (required) `database_name`: A top-level logical container for data and database objects within a database system.
* (required) `schema_name`: A named logical grouping of database objects within a database, used for organization and access control.

**`auth`:** This section provides information about your identity, which is required to connect to the database. It is common to all Google-native databases.

* (required) `json_credentials_filepath`: A string with the file that contains the credentials. This can be downloaded for user or service accounts via Google's interface. *Follow the instructions in* [*these docs*](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account) *to select your account, add a key, and download the JSON file. (If you need help with this step, please contact your database admin.)*

**`options.AlloyDBConnector`**: This section includes information that is specific for AlloyDB databases.

* (required) `region`: The Google Cloud region where the AlloyDB cluster is deployed (for example us-central1 or europe-west1).
* (required) `cluster`: The AlloyDB cluster that contains the database instances. A cluster is the top-level AlloyDB resource that manages storage, backups, and instances.
* (required) `instance_id`: The unique name of the specific AlloyDB database instance you want to connect to.
* `ip_type`: Specifies which network IP type the connector should use when connecting to AlloyDB. Common values are `PRIVATE` or `PUBLIC`.&#x20;
  * (default): `PRIVATE`&#x20;

## Setting Your Config File

Use the `set_import_config` and `set_export_config` functions to add your config YAML file to your connector.

```python
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the [Usage API](/sdv/integration/db/api).

## FAQ

<details>

<summary>Which permissions are needed for importing and exporting?</summary>

Importing data requires read access and exporting data requires write access.

Please note that your Python client will also need the base permissions for **roles/alloydb.client** and **roles/alloydb.viewer**. These roles grant the following permissions: `alloydb.clusters.generateClientCertificate`, `alloydb.clusters.get`, `alloydb.instances.connect`, `alloydb.instances.get`, `alloydb.users.login`, `alloydb.instances.executeSql`

If you are using a Google Cloud API, you will also need the Service Usage Consumer  permissions (**roles/serviceusage.serviceUsageConsumer**). To login with a service account, you will need to [enable IAM authentication](https://cloud.google.com/alloydb/docs/manage-iam-authn#enable) with the flag `alloydb.iam_authentication` (this is not enabled by default).

For more information, see [AlloyDB documentation](https://cloud.google.com/alloydb/docs/reference/iam-roles-permissions).

</details>


# ❖ BigQuery

This page describes how to integrate with your BigQuery database.

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

## Installation

To use this feature, please make sure you have installed the bundle with the optional `db-bigquery` dependency. For more information, see the [SDV Enterprise Installation docs](https://docs.sdv.dev/sdv-enterprise/installation/instructions).

```bash
pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-bigquery
```

## Connector

&#x20;Create a connector object that interacts with your BigQuery database.

```python
from sdv.io.database import BigQueryConnector

connector = BigQueryConnector()
```

**Parameters** (None)

**Output** A BigQueryConnector object that you can use to import data and metadata

## Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

```yaml
connection:
  database_name: <string> 
  schema_name: <string>

auth:
  json_credentials_path: <filepath>
```

**Parameters**:

**`connection`**: This section defines the database tables you want to connect to. It is common to all Google-native databases.

* (required) `database_name`: A top-level logical container for data and database objects within a database system.
* (required) `schema_name`: A named logical grouping of database objects within a database, used for organization and access control.

**`auth`:** This section provides information about your identity, which is required to connect to the database. It is common to all Google-native databases.

* (required) `json_credentials_filepath`: A string with the file that contains the credentials. This can be downloaded for user or service accounts via Google's interface. *Follow the instructions in* [*these docs*](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account) *to select your account, add a key, and download the JSON file. (If you need help with this step, please contact your database admin.)*

{% hint style="info" %}
Note that the terminology of BigQuery is slightly different from other databases. In the connection parameters: Set the `database_name` to null, and the `schema_name` to your BigQuery's `dataset_id`.
{% endhint %}

## Setting Your Config File

Use the `set_import_config` and `set_export_config` functions to add your config YAML file to your connector.

```python
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the [Usage API](/sdv/integration/db/api).

## FAQ

<details>

<summary>Which permissions are needed for importing and exporting?</summary>

**Importing data requires read access.** For BigQuery, this includes: `bigquery.jobs.create`, `bigquery.tables.get`,  `bigquery.tables.getData`, and `bigquery.tables.list`, `bigquery.datasets.get`. If you do not have these permissions, please contact your database admin.

**Exporting data requires write access.** For BigQuery, this includes: `bigquery.datasets.create`, `bigquery.datasets.get`, `bigquery.jobs.create`, `bigquery.tables.create`, and `bigquery.tables.export`. If you do not have these permissions, please contact your database admin.

</details>

<details>

<summary>Is there a way to speed up import and export?</summary>

Installing `google-cloud-bigquery-storage` in your environment enables the BigQuery Storage API for faster data import/export. This dependency is not currently required so be sure to install it yourself using your preferred package manager.

```
pip install google-cloud-bigquery-storage
```

To see results, make sure you have the `bigquery.readsessions.create` permission.

</details>


# ❖ MSSQL

This page describes how to integrate with your Microsoft SQL database.

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

## Installation

To use this feature, please make sure you have installed the bundle with the optional `db-mssql` dependency. For more information, see the [SDV Enterprise Installation docs](https://docs.sdv.dev/sdv-enterprise/installation/instructions).

```bash
pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-mssql
```

Microsoft SQL also requires that you have the ODBC Driver installed on your machine. For more information, see instructions from Microsoft for [Windows](https://learn.microsoft.com/en-us/sql/connect/odbc/microsoft-odbc-driver-for-sql-server?view=sql-server-ver16), [MacOS](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/install-microsoft-odbc-driver-sql-server-macos?view=sql-server-ver16), and [Linux](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16\&tabs=alpine18-install%2Calpine17-install%2Cdebian8-install%2Credhat7-13-install%2Crhel7-offline).

## Connector

&#x20;Create a connector object that interacts with your MSSQL database.

```python
from sdv.io.database import MSSQLConnector

connector = MSSQLConnector()
```

**Parameters** (None)

**Output** A connector object that you can use to import data and metadata

## Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

```yaml
connection:
  host: <string>
  port: <string>
  database_name: <string> 
  schema_name: <string>

auth:
  username: <string>
  password: <string>

runtime:
  connection_timeout: <int>

options.MSSQLConnector:
  encrypt: <true | false>
  trust_server_certificate: <true | false>
  max_pool_size: <int>
```

**Parameters**:

**`connection`**: This section defines the database tables you want to connect to. It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* (required) `host`: The network address of the system running the database service. Based on your db setup, this can be a hostname (eg. [db.example.com](http://db.example.com)) or an IP address (eg. 10.0.0.1)
* (required) `port`: The network endpoint on the host where the database service listens for client connections. This is a numeric identifier but please provide it as a string.
* (required) `database_name`: A top-level logical container for data and database objects within a database system.
* (required) `schema_name`: A named logical grouping of database objects within a database, used for organization and access control.

**`auth`:** This section provides information about your identity, which is required to connect to the database.  It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* (required) `username`: The identifier of who is authenticating the connection.
* (required) `password`: A secret credential associated with the username, used to verify its identity during authentication

**`runtime`**: This section contains other parameters that come into play during runtime, either when establishing the connection or running additional queries.  It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* `connection_timeout`:  The maximum amount of time a client will wait while attempting to establish a connection to the database before stopping the attempt. This applies when first establishing the connection, not for any subsequent queries or timeouts.
  * (default) Driver and environment dependent, typically between \~10-30 seconds, but potentially unbounded.

**`options.MSSQLConnector`**: This section includes information that is specific for MSSQL databases

* `encrypt`: Specifies whether the connection between the client (SDV) and the database server is encrypted.&#x20;
  * (default) `true`: Encrypt all data transmitted over the network between the client and server, including authentication credentials and query/result traffic.
  * `false`: Do not encrypt
* `trust_server_certificate`: In the case that encryption is turned on, this parameter controls whether the client (SDV) verifies the identity of the database server when establishing an encrypted connection.
  * (default) `false`: The client (SDV) validates the certificate (recommended, secure)
  * `true`: The client (SDV) skips validation. Encryption still happens, but identity is not verified.&#x20;
* `max_pool_size`: The maximum number of database connections that the application (SDV) is allowed to keep open at the same time.
  * (default) 100

{% hint style="info" %}
**MSSQL terminology** may be different from the above parameters names, which are generic for databases.

* MSSQL typically refers to the `host` as the `server`. This is the same concept.
* If you have not set up a `schema_name` explicitly, use the one that MSSQL assigns by default. This is usually `dbo`. *For more info, see the* [*Microsoft docs*](https://learn.microsoft.com/en-us/sql/relational-databases/security/authentication-access/ownership-and-user-schema-separation?view=sql-server-ver16#the-dbo-schema)*.*
  {% endhint %}

## Setting Your Config File

Use the `set_import_config` and `set_export_config` functions to add your config YAML file to your connector.

```python
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the [Usage API](/sdv/integration/db/api).

## FAQ

<details>

<summary>Which permissions are needed for importing and exporting?</summary>

**Importing data requires read access.** For MSSQL, you will need to have the `db_datareader` role.

**Exporting data requires write access.** For MSSQL, you will need to have the `db_datawriter`. If you'd like to overwrite data, then you'd need additional permissions given by the `db_ddladmin`, `db_owner`, or `sysadmin` roles.

For more information, see the [Microsoft documentation](https://learn.microsoft.com/en-us/sql/relational-databases/security/authentication-access/database-level-roles?view=sql-server-ver16).

</details>


# ❖ Oracle

This page describes how to integrate with your Oracle database.

*Supported database variants: Oracle Database* *23ai, 21c, 19c, 18c, 12c, and 11gR2.*

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

## Installation

To use this feature, please make sure you have installed the bundle with the optional `db-oracle` dependency. For more information, see the [SDV Enterprise Installation docs](https://docs.sdv.dev/sdv-enterprise/installation/instructions).

```bash
pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-oracle
```

## Connector

Use this connector to create a connection to your Oracle database.

```python
from sdv.io.database import OracleConnector

connector = OracleConnector()
```

**Parameters** (None)

**Output** A connector object that you can use to import data and metadata

## Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

```yaml
connection:
  host: <string>
  port: <string>
  database_name: <string> 
  schema_name: <string>

auth:
  username: <string>
  password: <string>

runtime:
  connection_timeout: <int>
```

**Parameters**:

**`connection`**: This section defines the database tables you want to connect to. It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* (required) `host`: The network address of the system running the database service. Based on your db setup, this can be a hostname (eg. [db.example.com](http://db.example.com)) or an IP address (eg. 10.0.0.1)
* (required) `port`: The network endpoint on the host where the database service listens for client connections. This is a numeric identifier but please provide it as a string.
* (required) `database_name`: A top-level logical container for data and database objects within a database system.
* (required) `schema_name`: A named logical grouping of database objects within a database, used for organization and access control.

**`auth`:** This section provides information about your identity, which is required to connect to the database.  It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* (required) `username`: The identifier of who is authenticating the connection.
* (required) `password`: A secret credential associated with the username, used to verify its identity during authentication

**`runtime`**: This section contains other parameters that come into play during runtime, either when establishing the connection or running additional queries.  It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* `connection_timeout`:  The maximum amount of time a client will wait while attempting to establish a connection to the database before stopping the attempt. This applies when first establishing the connection, not for any subsequent queries or timeouts.
  * (default) Driver and environment dependent, typically between \~10-30 seconds, but potentially unbounded.

{% hint style="info" %}
**Oracle terminology** may be different from the above parameters names, which are generic for databases.

* Oracle typically refers to the `database_name` as the `service_name`. This is the same concept.
* Oracle typically refers to the `host` as `server`. This is the same concept.
* If you have not set up a `schema_name` explicitly, use the one that Oracle assigns by default. This is usually your username.
  {% endhint %}

## Setting Your Config File

Use the `set_import_config` and `set_export_config` functions to add your config YAML file to your connector.

```python
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the [Usage API](/sdv/integration/db/api).

## FAQ

<details>

<summary>Which permissions are needed for importing and exporting?</summary>

**Importing data requires read access.** For importing data, you'll need access for running the following commands: `CREATE SESSION`, `SELECT`, and `SELECT ON SYS.DBA_USERS`.

**Exporting data requires write access.** For exporting data, you'll need the same access as importing *plus* access for following commands: `INSERT ANY`, `ALTER ANY`, `DROP ANY`, and `RESOURCE`. &#x20;

For more information, see the [Oracle SQL reference guide](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/GRANT.html).

</details>


# ❖ PostgreSQL

This page describes how to integrate with your Postgres database.

*Supported versions: PostgreSQL 18+*

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

## Installation

To use this feature, please make sure you have installed the bundle with the optional `db-postgresql` dependency. For more information, see the [SDV Enterprise Installation docs](https://docs.sdv.dev/sdv-enterprise/installation/instructions).

```bash
pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-postgresql
```

## Connector

Use this connector to create a connection to your Postgres database.

```python
from sdv.io.database import PostgreSQLConnector

connector = PostgreSQLConnector()
```

**Parameters** (None)

**Output** A connector object that you can use to import data and metadata

## Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

```yaml
connection:
  host: <string>
  port: <string>
  database_name: <string> 
  schema_name: <string>

auth:
  username: <string>
  password: <string>

runtime:
  connection_timeout: <int>
  
options.PostgreSQLConnector:
  application_name: <string>
  sslmode: <disable | allow | prefer | require | verify-ca | verify-full>
```

**Parameters**:

**`connection`**: This section defines the database tables you want to connect to. It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* (required) `host`: The network address of the system running the database service. Based on your db setup, this can be a hostname (eg. [db.example.com](http://db.example.com)) or an IP address (eg. 10.0.0.1)
* (required) `port`: The network endpoint on the host where the database service listens for client connections. This is a numeric identifier but please provide it as a string.
* (required) `database_name`: A top-level logical container for data and database objects within a database system.
* (required) `schema_name`: A named logical grouping of database objects within a database, used for organization and access control.

**`auth`:** This section provides information about your identity, which is required to connect to the database.  It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* (required) `username`: The identifier of who is authenticating the connection.
* (required) `password`: A secret credential associated with the username, used to verify its identity during authentication

**`runtime`**: This section contains other parameters that come into play during runtime, either when establishing the connection or running additional queries.  It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

* `connection_timeout`:  The maximum amount of time a client will wait while attempting to establish a connection to the database before stopping the attempt. This applies when first establishing the connection, not for any subsequent queries or timeouts.
  * (default) Driver and environment dependent, typically between \~10-30 seconds, but potentially unbounded.

**`options.PostgreSQLConnector`**: This section includes information that is specific for Postgres databases

* `application_name`: An optional string with the name of the application that is connecting to the database (for example `"SDV_Enterprise"`). This is useful for reporting and monitoring.
* `sslmode`: Specifies whether the connection between the client (SDV) and the database server is encrypted and if so, whether the client (SDV) verifies the identity of the database server when establishing an encrypted connection.
  * `disable`: Send over the data and queries as plain text. (Do not perform any encryption or validation.)
  * `allow`: Try to send over the data as plain text first. If that fails, then encrypt as a fallback.
  * (default) `prefer`: Try to send over the data using encryption first. If that fails, send plain text as a fallback.
  * `require`: Always send over the data using encryption
  * `verify-ca`: In addition to sending the data using encryption, the client (SDV) also validates the certificate. The hostname is not validated.
  * `verify-full`: In addition to sending the data using encryption, the client (SDV) also validates the certificate and hostname.

## Setting Your Config File

Use the `set_import_config` and `set_export_config` functions to add your config YAML file to your connector.

```python
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the [Usage API](/sdv/integration/db/api).

## FAQ

<details>

<summary>Which permissions are needed for importing and exporting?</summary>

**Importing data requires read access.** For importing data, you'll need the following permissions: `CONNECT` (to the database), `USAGE` (on the schema), `SELECT` (on the tables)

**Exporting data requires write access.** For exporting data, you'll need the same access as importing *plus* access for following commands:

* `CREATE` (on the database, needed to `write` to a new database schema)
* `INSERT` (on individual tables, needed to `append` to an existing database schema)
* `CREATE` (on the schema, needed to `overwrite` an existing database schema)
* Additionally, to `overwrite` an existing database schema, the user must be an owner of the table.

For more information, see the [Postgres reference guide](https://www.postgresql.org/docs/current/ddl-priv.html).

</details>


# ❖ Spanner

This page describes how to integrate with your Spanner database. The Spanner integration supports both GoogleSQL and PostgreSQL.

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

## Installation

To use this feature, please make sure you have installed the bundle with the optional `db-spanner` dependency. For more information, see the [SDV Enterprise Installation guide](https://docs.sdv.dev/sdv-enterprise/installation/instructions).

```bash
pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-spanner
```

## Connector

&#x20;Create a connector object that interacts with your Spanner database.

```python
from sdv.io.database import SpannerConnector

connector = SpannerConnector()
```

**Parameters** (None)

**Output** A SpannerConnector object that you can use to import data and metadata

## Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

```yaml
connection:
  database_name: <string> 
  schema_name: <string>

auth:
  json_credentials_path: <filepath>

options.SpannerConnector:
  instance_id: <id_value>
```

**Parameters**:

**`connection`**: This section defines the database tables you want to connect to. It is common to all Google-native databases.

* (required) `database_name`: A top-level logical container for data and database objects within a database system.
* (required) `schema_name`: A named logical grouping of database objects within a database, used for organization and access control.

**`auth`:** This section provides information about your identity, which is required to connect to the database. It is common to all Google-native databases.

* (required) `json_credentials_filepath`: A string with the file that contains the credentials. This can be downloaded for user or service accounts via Google's interface. *Follow the instructions in* [*these docs*](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account) *to select your account, add a key, and download the JSON file. (If you need help with this step, please contact your database admin.)*

**`options.SpannerConnector`**: This section includes information that is specific for Spanner databases.

* (required) `instance_id`: The unique identifier of the Cloud Spanner instance that contains the databases. In Spanner, an instance is the top-level compute and configuration container.

## Setting Your Config File

Use the `set_import_config` and `set_export_config` functions to add your config YAML file to your connector.

```python
connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')
```

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the [Usage API](/sdv/integration/db/api).

## FAQ

<details>

<summary>Which permissions are needed for importing and exporting?</summary>

**Importing data requires read access.** For Spanner, this includes: `spanner.databases.list`,  `spanner.databases.read`, `spanner.databases.select`, `spanner.instances.get`, `spanner.instances.list`, `spanner.sessions.create`, `spanner.sessions.get`, `spanner.sessions.list`

**Exporting data requires write access.** For Spanner, this includes: `spanner.databaseOperations.get`, `spanner.databases.get`, `spanner.databases.getDdl`, `spanner.databases.list`, `spanner.databases.select`, `spanner.databases.updateDdl`, `spanner.databases.write`, `spanner.instances.list`, `spanner.sessions.create`, `spanner.sessions.get`

For official Spanner documentation about roles and permissions, [click here](https://cloud.google.com/spanner/docs/iam). If you do not have these permissions, please contact your database admin.

</details>


# ❖ Usage API

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

Once you completed [integration](/sdv/integration/db/integration), you should have a database connector object that is ready to use. Use the API endpoints below to create metadata and import training data to use with SDV.

## Importing Data

Import data from your database to use with SDV.

### ❖ set\_import\_config

Use this function to authenticate into the project and database you'd like to import from.

```python
connector.set_import_config(filepath='source_db_config.yaml')
```

**Parameters**

* (required) `filepath`: A string with the filepath that contains your configuration options. This should be a YAML file that ends in `.yaml`. For more information see section for [Configuration](/sdv/integration/db/integration).

**Output** (None)

### ❖ create\_metadata

Use this function to create metadata based on the connection to your database.

```python
metadata = connector.create_metadata(
    table_names=['users', 'transactions', 'sessions'])
```

**Parameters**

* `table_names`: A list of strings representing the table names that you want to create metadata for
  * (default) `None`: Create metadata for all the tables in the database
* `infer_sdtypes`: A boolean describing whether to infer the sdtypes of each column
  * (default) `True`: Infer the sdtypes based on the data
  * `False`: Do not infer the sdtypes. All columns will be marked as `unknown`, ready for you to manually update
* `infer_keys`: A string describing whether to infer the primary and/or foreign keys. Options are:
  * (default) `'primary_and_foreign'`: Infer the primary keys in each table, and the foreign keys in other tables that refer to them
  * `'primary_only'`: Infer only the primary keys of each table
  * `None`: Do not infer any keys

**Output** A [Metadata](/sdv/integration/metadata) object

{% hint style="warning" %}
**The detected metadata is not guaranteed to be accurate or complete.** Be sure to carefully inspect the metadata and update information. For more information, see the [Metadata API](/sdv/integration/metadata/metadata-api).
{% endhint %}

### ❖ import\_random\_subset

Use this function to import a random subset of your data from the database and inferred metadata. The size of the subset is automatically determined.

```python
data = connector.import_random_subset(
    metadata=metadata,
    verbose=True
)
```

**Parameters**

* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object that describes the data you want to import
* `fixed_seed`: A boolean that controls the determinism of the random subset
  * (default) `False`: Different random data will be imported every time you call the function
  * `True`: The same data will be imported every time you import
* `verbose`: A boolean describing whether to print out details about the progress of the import
  * (default) `True`: Print out the table names and number of rows being imported
  * `False`: Do not print any details

**Output** A dictionary that maps each table name of your database (string) to the data, represented as a pandas DataFrame.&#x20;

### ❖ import\_optimized\_subset

Use this function to import a subset of your data, optimized specifically for a given table. You can also control the size.

```python
data = connector.import_optimized_subset(
    metadata=metadata, 
    main_table_name='users',
    num_rows=5000
)
```

**Parameters**

* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object that describes the data you want to import
* (required) `main_table_name`: A string containing the name of the most important table of your database. This table will generally represent the entity that is most critical to your application or business. It must be one of the tables listed in your metadata object.
* `num_rows`: The number of rows to sample from the main table. The size of every other table is automatically determined by its connection to the main table.
* `fixed_seed`: A boolean that controls the determinism of the random subset
  * (default) `False`: Different random data will be imported every time you call the function
  * `True`: The same data will be imported every time you import
* `verbose`: A boolean describing whether to print out details about the progress of the import
  * (default) `True`: Print out the table names and number of rows being imported
  * `False`: Do not print any details

**Output** A dictionary that maps each table name of your database (string) to the data, represented as a pandas DataFrame.

After importing the data and metadata, you are now ready to create an SDV synthesizer.

## Exporting synthetic data

Export synthetic data into a new database.

{% hint style="success" %}
**We recommend using the same connector as your import.** This connector object already knows about the specifics of your database schema. It will ensure that the exported data schema has the same format.
{% endhint %}

### ❖ set\_export\_config

Use this function to authenticate into the project and database you'd like to export to.

```python
connector.set_import_config(filepath='destination_db_config.yaml')
```

**Parameters**

* (required) `filepath`: A string with the filepath that contains your configuration options. For more information see section for [Configuration](/sdv/integration/db/integration).

**Output** (None)

### ❖ export

Use this function to export your synthetic data into a database

```python
connector.export(
    data=synthetic_data,
    mode='write',
    verbose=True)
```

**Parameters**

* (required) `synthetic_data`: A dictionary that maps each table name to the synthetic data, represented as a pandas DataFrame
* `mode`: The mode of writing to use during the export
  * (default) `'write'`: Write a new database from scratch. If the database or data already exists, then the function will error out.
  * `'append'`: Append rows to existing tables in the database
  * `'overwrite'`: Remove any existing tables in the database and replace them with this data
* `verbose`: A boolean describing whether to print out details about export
  * (default) `True`: Print the details
  * `False`: Do not print anything

**Output** (None) Your data will be written to the database and ready for use by your downstream application!

## FAQs

### Authentication & Credentials

Before using your connector, you will need to authenticate into the database. To do this, start by determining whether your account is a **user account** or a **service account**. Service accounts are typically made for non-human users. (For more details, [see the docs](https://cloud.google.com/iam/docs/best-practices-service-accounts).)

<details>

<summary>Authenticating a user account</summary>

If you are connecting a user account, we recommend using the Application Default Credentials (ADC).&#x20;

**Prerequisites**: Run through these commands on your terminal (command line) before using Python.

1. To get started, install the Google CLI using [these instructions](https://cloud.google.com/sdk/docs/install#linux). Then run the following command on your terminal:

```
gcloud init
gcloud auth application-default login
```

This will prompt you to sign in, and then save the credentials onto your local machine in a pre-determined location ([see the docs](https://cloud.google.com/docs/authentication/application-default-credentials#personal)).&#x20;

2. Next, run the following command in your terminal to generate a password to use for authentication. *Save this password.*

```
gcloud auth print-access-token
```

3. Finally run the following command to ensure you have access to the database. Fill in the each of the sections with your information.

```bash
gcloud alloydb users create <email> --cluster=<cluster> --region=<region> --type=IAM_BASED
```

**Authentication in Python**: Now, you can use the Python library. The AI connector will automatically be able to find your file and authenticate you. Provide the following information when setting up your Python connector:

```python
connector.set_import_config(
    schema_name='public',
    auth={
        'database': 'value', # required
        'project_id': 'value', # required
        'region': 'value', # required
        'cluster_id': 'value', # required
        'instance_id': 'value', # required
        'ip_type': 'PUBLIC', # required: either 'PUBLIC' or 'PRIVATE'
        'user': 'youremail@company.com', # required, your email used for authentication
        'password': 'your-access-token', # required, from setp #2
    }
)
```

</details>

<details>

<summary>Authenticating a service account</summary>

If you are connecting a service account, start by downloading a JSON credential file for the service account onto your machine. Follow the instructions in [these docs](https://developers.google.com/workspace/guides/create-credentials#create_credentials_for_a_service_account) to select your service account, add a key, and download the JSON file. (If you need help with this step, please contact your database admin.) Then follow the recommended approach below.

**Recommended approach**: When authenticating your AI connector, pass in the JSON filepath along with other necessary info, as shown below.

```python
connector.set_import_config(
    auth={
        'json_credentials_path': 'my_folder/credentials.json', # passs in JSON filepath
        'database': 'value', # required
        'project_id': 'value', # required
        'region': 'value', # required
        'cluster_id': 'value', # required
        'instance_id': 'value', # required
    },
    schema_name='public'
)
```

**Alternative approach (environment variables)**: Alternatively, you can save the JSON filepath and project ID as global variables by typing the terminal commands below.

```bash
export GOOGLE_APPLICATION_CREDENTIALS=my_folder/credentials.json
export GOOGLE_CLOUD_PROJECT=my_project_id
```

Once you've run the above lines of code, your AI Connector can automatically locate the JSON file that just got saved and pull out the auth credentials from it. You don't have to specify this information anymore.

```python
connector.set_import_config(
    auth={
        'database': 'value', # required
        'project_id': 'value', # required
        'region': 'value', # required
        'cluster_id': 'value', # required
        'instance_id': 'value', # required
    },
    schema_name='public'
)
```

</details>


# Import Local Data

If your data is already available as local files on your machine, load them into Python for SDV to use.

```python
from sdv.io.local import CSVHandler

connector = CSVHandler()
data = connector.read(
    folder_name='project/data/',
    file_names=['users.csv', 'transactions.csv', 'sessions.csv'])
```

## Browse File Types

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="/sdv/integration/local/csv"><strong>CSV Data</strong></a></td><td>Load multiple CSV files into Python.</td><td><a href="/sdv/integration/local/csv">CSV</a></td></tr><tr><td><a href="/sdv/integration/local/excel"><strong>Excel Spreadsheet</strong></a></td><td>Load an entire Excel spreadsheet into Python.</td><td><a href="/sdv/integration/local/excel">Excel</a></td></tr><tr><td><a href="/sdv/integration/local/additional-file-formats"><strong>Additional File Formats</strong></a></td><td>Use the pandas library to load from additional file formats.</td><td><a href="/sdv/integration/local/additional-file-formats">Additional File Formats</a></td></tr></tbody></table>


# CSV

### CSVHandler <a href="#bigqueryconnector" id="bigqueryconnector"></a>

Use this object to create a handler for reading and writing local CSV files.

```python
from sdv.io.local import CSVHandler

connector = CSVHandler()
```

**Parameters** (None)

**Output** A CSVHandler object you can use to read and write CSV files

### read

Use this function to read multiple CSV files form your local machine

```python
data = connector.read(
    folder_name='project/data/',
    file_names=['users.csv', 'transactions.csv', 'sessions.csv'],
    read_csv_parameters={
        'parse_dates': False,
        'encoding':'latin-1'
    }
)
```

**Parameters**

* (required) `folder_name`: A string name of the folder that contains your CSV files
* `file_names`: A list of strings with the exact file names to read
  * (default) `None`: Read all the CSV files that are in the specified folder
  * `<list>`: Only read the list of CSV files that are in the list
* `keep_leading_zeros`: A boolean that describes whether any values with leading zeros should be kept, or whether they can be safely removed and converted to ints/floats.
  * (default) `True`: Keep any leading zeros. This is especially helpful when the data is not truly numerical, for example `"02446"` is a postal code in Boston. It is not valid to read this as a number `2,446`.
  * `False`: Do not keep any leading zeros. Select this for faster read times if you know your data contains numbers.
* `read_csv_parameters`: A dictionary with additional parameters to use when reading the CSVs. The keys are any of the parameter names of the [pands.read\_csv](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html) function and the values are your inputs.
  * (default) `{ 'parse_dates': False, 'low_memory': False, 'on_bad_lines': 'warn'}`: Do not infer any datetime formats, assume low memory, or error if it's not possible to read a line. (Use all the other defaults of the `read_csv` function.)

**Output** A dictionary that contains all the CSV data found in the folder. The key is the name of the file (without the `.csv` suffix) and the value is a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the data.

### write

Use this function to write synthetic data as multiple CSV files

```python
connector.write(
  synthetic_data,
  folder_name='project/synthetic_data',
  to_csv_parameters={
      'encoding': 'latin-1',
      'index': False
  },
  file_name_suffix='_v1', 
  mode='x')
)
```

**Parameters**

* (required) `synthetic_data`: You data, represented as a dictionary. The key is the name of each table and the value is a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the data.&#x20;
* (required) `folder_name`: A string name of the folder where you would like to write the synthetic data
* `to_csv_parameters`: A dictionary with additional parameters to use when writing the CSVs. The keys are any of the parameter names of the [pandas.to\_csv](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html) function and the values are your inputs.
  * (default) `{ 'index': False }`: Do not write the index column to the CSV. (Use all the other defaults of the `to_csv` function.)
* `file_name_suffix`: The suffix to add to each filename. Use this if to add specific version numbers or other info.
  * (default) `None`: Do not add a suffix. The file name will be the same as the table name with a `.csv` extension
  * \<string>: Append the suffix after the table name. Eg. a suffix `'_synth1'` will write a file as `table_synth1.csv`&#x20;
* `mode`: A string signaling which mode of writing to use
  * (default) `'x'`: Write to new files, raising errors if any existing files exist with the same name
  * `'w'`: Write to new files, clearing any existing files that exist
  * `'a'`: Append the new CSV rows to any existing files

**Output** (None) The data will be written as CSV files


# Excel

### Installation <a href="#bigqueryconnector" id="bigqueryconnector"></a>

To work with Excel files in SDV, please install SDV with the optional dependencies using:

```
pip install 'sdv[excel]'
```

### ExcelHandler <a href="#bigqueryconnector" id="bigqueryconnector"></a>

Use this object to create a handler for reading and writing an local Excel spreadsheet

```python
from sdv.io.local import ExcelHandler

connector = ExcelHandler()
```

**Parameters** (None)

**Output** An ExcelHandler object you can use to read and write an Excel spreadsheet

### read

Use this function to read an Excel spreadsheet from your local machine

```python
data = connector.read(
    filepath='project/data.xlsx',
    sheet_names=['guests', 'hotels']
)
```

**Parameters**

* (required) `filepath`: A string name of the Excel spreadsheet, which must end in `.xlsx`
* `sheet_names` : A list of sheet names to read from the overall spreadsheet. Each sheet is a tab in your spreadsheet file, and represents a different table.
  * (default) `None`: Read all the sheets of the overall spreadsheet
  * \<list>: Read only the sheets listed from the overall spreadsheet

**Output** A dictionary that contains all the data found the Excel file. The key is the name of the sheet and the value is a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the data.

### write

Use this function to write synthetic data into an Excel spreadsheet

```python
connector.write(
  synthetic_data,
  filepath='project/synthetic_data.xlsx',
  mode='x'
)
```

**Parameters**

* (required) `synthetic_data`: You data, represented as a dictionary. The key is the name of each table and the value is a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the data.&#x20;
* (required) `filepath`: A string name of the file where you would like to write the synthetic data
* `sheet_name_suffix`: The suffix to add to each sheet name. Use this if to add specific version numbers or other info.
  * (default) `None`: Do not add a suffix. Each sheet will be named with table name
  * \<string>: Append the suffix after each table name. Eg. a suffix `'_synth1'` will write a sheet as `table_synth1`&#x20;
* `mode`: A string signaling which mode of writing to use
  * (default) `'w'`: Write sheets to a new Excel file, clearing any existing file that may exist
  * `'a'`: Write the data as new sheets within an existing Excel file

**Output** (None) The data will be written as an Excel spreadsheet


# Additional File Formats

## Do you have data in other formats?

The SDV uses the [pandas library](https://pandas.pydata.org/) for data manipulation and synthesizing. If your data is in any other format, load it in as a [pandas.DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) object to use in the SDV. For multi table data, make sure you format your data as a dictionary, mapping each table name to a different DataFrame object.

```python
data = {
    'table_name_1': <pandas.DataFrame>,
    'table_name_2': <pandas.DataFrame>,
    ...
}
```

Pandas offers many methods to load in different types of data. For example: [SQL table](https://pandas.pydata.org/docs/reference/api/pandas.read_sql_table.html#pandas.read_sql_table) or [JSON string](https://pandas.pydata.org/docs/reference/api/pandas.read_json.html#pandas.read_json).

```python
import pandas as pd

data_table_1 = pd.read_json('file://localhost/path/to/table_1.json')
data_table_2 = pd.read_json('file://localhost/path/to/table_2.json')
```

For more options, see the [pandas reference](https://pandas.pydata.org/docs/reference/io.html).


# Metadata

**Metadata** is a description of the dataset you want to synthesize. Metadata is required for any SDV synthesizer.

Your data can be present in one or multiple data tables. Metadata includes the names of tables, columns, data types in each of the columns, and relationships. For example, your data may look like this:

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FpPLDP2gOJBQvF5yF6Zmz%2Fmulti-table-data-data-preparation_May%208%202025.png?alt=media&amp;token=f23c6cac-43db-4e08-9eaf-5c850aacecfc" alt=""><figcaption><p>An example of multi table data. The tables are connected to each other through primary/foreign keys.</p></figcaption></figure>

Click below to see the metadata description for this dataset:

<details>

<summary>Click to see the metadata </summary>

```json
{
    "METADATA_SPEC_VERSION": "V2",
    "tables": {
        "hotels": {
            "primary_key": "hotel_id",
            "columns": {
                "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" },
                "city": { "sdtype": "city", "range_is_nullable": false },
                "rating": { "sdtype": "numerical", "range_min": 0, "range_max": 5, "decimal_places": 1 }
            },
            "column_relationships": []
        },
        "guests": {
            "primary_key": "guest_email",
            "columns": {
                "guest_email": { "sdtype": "email", "range_is_nullable": false },
                "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" },
                "checkin_date": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "1970-01-01", "range_max": "2050-12-31" },
                "checkout_date": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "1970-01-01", "range_max": "2050-12-31" },
                "room_type": { "sdtype": "categorical" }
            },
            "column_relationships": []
        }
    },
    "relationships": [{
        "parent_table_name": "hotels",
        "parent_primary_key": "hotel_id",
        "child_table_name": "guests",
        "child_foreign_key": "hotel_id"
    }]
}
```

</details>

## SDV treats metadata as the ground truth

All SDV synthesizers frequently refer to metadata as the ground truth whenever they are creating or evaluating synthetic data. For high quality synthetic data generation, it's vital that your metadata accurately describes the data.

{% hint style="success" %}
**See metadata in action!** Your data may have a column containing values such as `94102`, `94130` and `94702`. What should SDV learn about this data in order to create synthetic data?

If the metadata specifies the data is numerical, SDV learns these values on a sliding number scale. But if the metadata specifies the data actually represents concepts such as postal codes, SDV can better ensure your synthetic data will be valid.
{% endhint %}

## Create and validate your metadata

There are many ways to create metadata. If you're directly [connecting your database](/sdv/integration/db), SDV creates metadata based on your database schema. Otherwise, you can auto-detect the metadata based on any [local data](/sdv/integration/local) that you've loaded. You always view and modify your metadata as needed.

**Please spend some time inspecting and updating your metadata to ensure it accurately describes your data.**&#x20;

```python
from sdv.metadata import Metadata

# Create metadata via auto-detection (if you don't already have it)
metadata = Metadata.detect_from_dataframes(data)

# Carefully inspect and update your metadata
metadata.visualize()
metadata.update_column(
    column_name='room_type',
    sdtype='categorical',
    table_name='guests'
)

metadata.validate()

# When you're done, save it to a file for future use
metadata.save_to_json('my_final_metadata.json')
```

## Resources

Click the links below to learn more about metadata.

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="/sdv/integration/metadata/sdtypes"><strong>Sdtypes</strong></a></td><td>Learn SDV's system for keeping track of data types for each column.     </td><td><a href="/sdv/integration/metadata/sdtypes">Sdtypes</a></td></tr><tr><td><a href="/sdv/integration/metadata/metadata-api"><strong>Metadata API</strong></a></td><td>Use Python to auto-detect, inspect, and update metadata.</td><td><a href="/sdv/integration/metadata/metadata-api">Metadata API</a></td></tr><tr><td><a href="/sdv/integration/metadata/metadata-json"><strong>Metadata JSON</strong></a></td><td>This is the final format that your metadata will be saved as.</td><td><a href="/sdv/integration/metadata/metadata-json">Metadata JSON</a></td></tr></tbody></table>


# Sdtypes

Let SDV know what type of data you have in each column of your table. In the SDV, the data types are specified by **sdtype**, denoting a semantic or statistical meaning. SDV is designed to create synthetic data differently based on each sdtype.

{% hint style="info" %}
An sdtype is a high level concept that tells SDV what your data *means*. An sdtype does not depend on how a computer stores the data. A single sdtype (such as `"categorical"`) can be stored by a computer in several ways (string, integer, etc).
{% endhint %}

## Common Sdtypes

Below are common sdtypes that describe columns in a dataset.

### Categorical

Sdtype `categorical` describes columns that contain discrete, unordered categories.

{% hint style="info" %}
An example of categorical data is tax payer status such as `Single`, `Married filing jointly`, `Head of household`, etc. These discrete categories are unordered.
{% endhint %}

```json
{
    "tax_payer_status": {
        "sdtype": "categorical",
        "range_values": ["single", "married_filing_jointly", "married_filing_separately",
                         "head_of_household", "surviving_spouse"],
        "range_is_nullable": false
    },
    "tax_location_code": {
        "sdtype": "categorical",
        "high_cardinality": true
    }
}
```

**Properties**:

* `range_values`: A list of all possible non-null values that can appear in the categorical column. Because categorical data is unordered, this list can appear in any order. *This list may include category values that are not present in your training data.*
  * (default) If this is not provided, then SDV assumes that the full list of possible values is not known. It may look at your training data to determine the comprehensive list.
* `high_cardinality`: A true/false value describing whether the categorical column has a large number of possible category values that cannot be fully and meaningfully listed in `range_values`. We recommend setting this when there are >500 unique values. If this is set to true, then the `range_values` should not be present.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.

### Ordinal

Sdtype `ordinal` describes columns that contain discrete elements with an inherent order.

{% hint style="info" %}
An example of ordinal data is survey responses such as `strongly_disagree`, `disagree`, etc., as there is an order between these values.

Ordinal data might also be present as integers. For example, the responses could be coded instead as `1`, `2`, `3`, `4`, or `5`.
{% endhint %}

```json
{
    "likert_responses": {
        "sdtype": "ordinal",
        "range_values": ["strongly_disagree", "disagree", "neither_agree_nor_disagree",
                         "agree", "strongly_agree"]
    }
}
```

**Properties:**

* `range_values`: A list of all possible non-null values that can appear in the ordinal column. Because ordinal data is ordered, this list should appear in order from lowest to highest. *This list may include ordinal values that are not present in your training data.*
  * (default) If this is not provided, then SDV assumes that the full list of possible values is not known. It may look at your training data to determine the comprehensive list.
* `high_cardinality`: A true/false value describing whether the ordinal column has a large number of possible values that cannot be fully and meaningfully listed in `range_values`. We recommend setting this when there are >500 unique values. If this is set to true, then the `range_values` should not be present.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.

### Numerical

Sdtype `numerical` describes continuous data represented as numbers. You can apply a variety of mathematical computations to the values (average, sum, etc.) to numerical data. The actual values may follow a specific format, such as being rounded to 2 decimal digits and remaining between min/max bounds.

{% hint style="info" %}
Some data may appear numerical but actually represents distinct categories. For example, HTTP response codes such as `200`, `404`, etc. are categorical data. These numbers don't have a specific order, and they cannot be combined or averaged.
{% endhint %}

**Properties**

* `range_min`: The absolute lowest value that is possible for this column. *This might be lower than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the lowest value is not known. It may look at your training data to determine the lowest value.
* `range_max`: The absolute highest value that is possible for this column. *This might be higher than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the highest value is not known. It may look at your training data to determine the highest value.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
* `decimal_places`: An integer representing the number of decimal places that the numerical values should be rounded to, for example `2` for 2 decimal digits. Note that a designation of `0` could mean either that there are whole numbers (integers) or that all values contain `.0` at the end such as `1.0`, `2.0`, etc.

```json
{
    "age": { 
        "sdtype": "numerical",
        "range_min": 18,
        "range_max": 120,
        "decimal_places": 0
    },
    "transaction_amt": {
        "sdtype": "numerical",
        "range_is_nullable": false,
        "decimal_places": 2
    }
}
```

### Datetime

Sdtype `datetime` describes columns that indicate a point of time. This can be at any granularity: to the nearest day, minute, second or even nanosecond. Typically, the datetime will be represented as a string.

```json
{
    "start_date": { 
        "sdtype": "datetime",
        "datetime_format": "%Y-%m-%d",
        "range_min": "1970-01-01",
        "range_max": "2050-12-31"
    }
}
```

&#x20;**Properties**

* (required) `datetime_format`: A string describing the format as defined by [Python's strftime module](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes).
* `range_min`: The absolute earliest datetime that is possible for this column. *This might be earlier than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the earliest value is not known. It may look at your training data to determine the earliest value.
* `range_max`: The absolute latest value that is possible for this column. *This might be later than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the latest value is not known. It may look at your training data to determine the latest value.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.

### Boolean

A `boolean` columns contains `True` or `False` values and may contain some missing data.

```json
{
    "is_active": {
        "sdtype": "boolean",
        "range_is_nullable": false
    }
}
```

**Properties**:

* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.

### ID

Sdtype `id` describes columns that are used to identify rows (eg. as a primary or foreign key). ID columns may not have a statistical or mathematical meaning behind them. Typically, an ID column follows a particular structure, for example being exactly 8 digits long with a `-` in the middle.&#x20;

**Properties**

* `regex_format`: A string describing the format of the ID as a [regular expression](https://docs.python.org/3/library/re.html)

```json
{
    "product_code": {
        "sdtype": "id",
        "regex_format": "[0-9]{4}-[0-9]{4}"
    }
}
```

{% hint style="success" %}
**What will the synthetic data contain?** Your synthetic data will contain brand new, randomly generated IDs based on the regex. If you have multiple tables, the primary and foreign key IDs will match up.
{% endhint %}

#### ＊ ID Regex Formats in SDV Enterprise

SDV Enterprise is designed to automatically detect Regexes based on your data. You can view or update these Regexes. SDV Enterprise can also detect and accept Regexes that contain **embedded context**.

{% hint style="info" %}
**What is embedded context?** Many ID columns have completely random values that do not contain any statistical meaning. But other times, a portion of your ID column may have a meaning — it is not completely random. We call this portion of the ID column *embedded context.*

For example consider an ID that starts with a 2-letter country code followed by a 5-character random value such as `US-3RO9P` or `CA-99QC4`. In this case, the first 2 letters form the *context* because there is a meaning behind them. SDV should not invent random countries. However, the final 5 characters are random without any statistical meaning.
{% endhint %}

SDV Enterprise detects embedded context using [*named capture groups*](https://docs.python.org/3/howto/regex.html#non-capturing-and-named-groups) within the Regex. Denoted by a parenthesis and the group name, named capture groups let the model know to learn the context exactly as-is and not invent new random values for it. In the example above, the Regex format would be:

```
(?P<country>[A-Z]{2})_[0-9A-Z]{5}
```

The name of group (in this case `country`) is important.

* If it represents and entirely new concept that is not found anywhere else in the data, then it should contain a brand new name that is not the same as any other column.
* If the concept already exists as another column in the data, then it should be exactly the same as the column name.

*SDV Enterprise is designed to auto-detect Regexes, including Regexes with context.*

## Additional Sdtypes: Domain-Specific Concepts & PII

You may find that some of the columns in your dataset represent high-level, concepts in your domain. Such data might also contain sensitive, Personal Identifiable Information (PII).

**Properties**

* `pii`: A boolean denoting whether the data is sensitive
  * (default) `true`: The column is sensitive, meaning the values should be anonymized. If not provided, we assume that the column is PII.
  * `false`: The column is not sensitive, meaning the exact set of values can be reused in the synthetic data
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.

```json
{
    "user_ssn": {
        "sdtype": "ssn",
        "range_is_nullable": true
    },
    "user_city": {
        "sdtype": "city",
        "pii": false,
        "range_is_nullable": false
    }
}
```

{% hint style="success" %}
**Synthetic data goals:** Your synthetic data will contain entirely new values that do not necessarily appear in the original data. If you are using SDV Enterprise, the synthetic data may conform to some generic properties that are non-identifiable (eg. an area code of a phone number).
{% endhint %}

Browse below for some common sdtypes related to different concepts.

{% tabs %}
{% tab title="Personal Info" %}
These sdtypes describe the information about a person.

<table data-header-hidden><thead><tr><th width="205"></th><th></th></tr></thead><tbody><tr><td>＊<code>phone_number</code></td><td>A local or international phone number such as <code>'+1(555)123-4567'</code>. Different countries have different formats.</td></tr><tr><td>＊<code>email</code></td><td>A person's email such as <code>'first_last@gmail.com'</code></td></tr><tr><td><code>ssn</code></td><td>A social security number such as <code>000-00-0000</code></td></tr><tr><td><code>first_name</code></td><td>A person's first name</td></tr><tr><td><code>last_name</code></td><td>A person's last name</td></tr></tbody></table>

*＊ Licensed, enterprise users will see higher quality data. The SDV will extract the deeper meaning and replicate it in the synthetic data. To learn more,* [*visit our website*](https://datacebo.com/pricing/)*.*
{% endtab %}

{% tab title="Location" %}
These sdtypes describe a location around the world.

<table data-header-hidden><thead><tr><th width="280"></th><th></th></tr></thead><tbody><tr><td>＊<code>country_code</code></td><td>A 2-character country code such as <code>'US'</code></td></tr><tr><td>＊<code>administrative_unit</code></td><td>The name of a region inside the country such as <code>'Massachusetts'</code>. Countries call this concept different names such as <em>state</em> or <em>province</em>.</td></tr><tr><td>＊<code>state_abbr</code></td><td>For countries that call their regions <em>states</em>, this refers to the 2-character code such as <code>'MA'</code></td></tr><tr><td>＊<code>city</code></td><td>The full name of the city such as <code>'Boston'</code></td></tr><tr><td>＊<code>postcode</code></td><td>The internationally-recognized, 5-digit postcode such as <code>02116</code></td></tr><tr><td>＊<code>street_address</code></td><td>The street and building number such as <code>'123 Main St'</code>. The exact format of this may vary by country</td></tr><tr><td>＊<code>secondary_address</code></td><td>Additional information about units in the building, such as <code>'Apartment #3'</code>. </td></tr><tr><td><code>latitude</code></td><td>The latitude of a location, expressed as a decimal</td></tr><tr><td><code>longitude</code></td><td>The longitude of a location, expressed as a decimal</td></tr></tbody></table>

*＊ Licensed, enterprise users will see higher quality data. The SDV will extract the deeper meaning and replicate it in the synthetic data. To learn more,* [*visit our website*](https://datacebo.com/pricing/)*.*
{% endtab %}

{% tab title="Networking" %}
These sdtypes describe information about computer networks and the internet.

<table data-header-hidden><thead><tr><th width="233"></th><th></th></tr></thead><tbody><tr><td><code>ipv4_address</code></td><td>An IP address, using the v4 protocol</td></tr><tr><td><code>ipv6_address</code></td><td>An IP address, using the v6 protocol</td></tr><tr><td><code>mac_address</code></td><td>A media access control address</td></tr><tr><td><code>user_agent_string</code></td><td>A user agent string sent by the HTTP protocol</td></tr></tbody></table>

{% endtab %}

{% tab title="Banking" %}
These sdtypes describe information needed for banking functions such as payment transfers.

<table data-header-hidden><thead><tr><th width="242"></th><th></th></tr></thead><tbody><tr><td><code>iban</code></td><td>An international bank account number</td></tr><tr><td><code>swift11</code></td><td>A SWIFT bank code that uses 11 digits</td></tr><tr><td><code>swift8</code></td><td>A SWIFT bank code that uses 8 digits</td></tr><tr><td><code>credit_card_number</code></td><td>A credit card number, expressed using digits</td></tr></tbody></table>

{% endtab %}

{% tab title="Automotive" %}
These sdtypes describe concepts from the automotive industry.

<table data-header-hidden><thead><tr><th width="200"></th><th></th></tr></thead><tbody><tr><td><code>vin</code></td><td>A vehicle identification number</td></tr><tr><td><code>license_plate</code></td><td>A license plate number, expressed using digits, letters or other characters. The format varies by country.</td></tr></tbody></table>
{% endtab %}

{% tab title="Other" %}
Many other sdtypes are possible. The SDV models can use the [Python Faker library](https://faker.readthedocs.io/en/master/providers.html) for new data types. You can input any of the function names as sdtypes. For example, inputting the sdtype `passport_number` will use [this function](https://faker.readthedocs.io/en/master/providers/faker.providers.passport.html#faker.providers.passport.Provider.passport_number) to generate meaningful numbers.&#x20;

For full SDV support, [file a request](https://github.com/sdv-dev/SDV/issues/new/choose) to help us prioritize other data types.
{% endtab %}
{% endtabs %}

### FAQs

<details>

<summary>How does the SDV factor in different countries or languages</summary>

Many concepts vary based on the country and language. For example, phone numbers are represented differently in different countries.

The SDV aims to provide worldwide support. You can specify the locales when you create a synthesizer. This lets the SDV know that any higher-level concepts should conform to the right formatting rules for that country.

For example, assume you provide the following info:

```python
from sdv.single_table import GaussianCopulaSynthesizer

synthesizer = GaussianCopulaSynthesizer(
    metadata, locales=['en_US', 'nl_BE'])
```

Then for every concept described in your metadata, the SDV will generate values only from the US or Belgium in the appropriate language (English or Dutch).

</details>

<details>

<summary>Does the SDV understand the context of PII data?</summary>

The public SDV randomly creates values corresponding to the concept, without taking additional context into account. Sometimes this may not be enough. For example, you may want to extract geographical areas from `phone_number` to ensure that it follows the same patterns.

These features are available to licensed users. To learn more, [contact us](https://datacebo.com/contact/).

</details>


# Metadata API

## Creation API

There are various ways to create your metadata. For example, if you are an SDV Enterprise user, you can directly [connect to a database](/sdv/integration/overview#connect-to-a-database) to load data and create metadata all at once.&#x20;

Otherwise, if you are in possession of your data, you can auto-detect the metadata.

### detect\_from\_dataframes

Use this function to automatically detect metadata from your data that you've loaded as a pandas.DataFrame objects.

**Parameters**:

* (required) `data`: Your data, represented as a dictionary. The keys are your table names and values are the pandas.DataFrame objects containing your data.
* `infer_sdtypes`: A boolean describing whether to infer the sdtypes of each column
  * (default) `True`: Infer the sdtypes of each column based on the data.
  * `False`: Do not infer the sdtypes. All columns will be marked as unknown, ready for you to manually update.
* `infer_keys`: A string describing whether to infer the primary and/or foreign keys.
  * (default) `'primary_and_foreign'`: Infer the primary keys in each table, and the foreign keys in other tables that refer to them
  * `'primary_only'`: Infer the primary keys in each table. You can manually add the foreign key relationships later.
  * `None`: Do not infer any primary or foreign keys. You can manually add these later.
* `foreign_key_inference_algorithm`: The algorithm to use when inferring the foreign key connections to primary keys
  * (default) `'column_name_match'`: Match up foreign and primary key columns that have the same names
  * ＊(default, SDV Enterprise) `'data_match'`: Match up foreign and primary key columns based on the data that they contain
* ＊`infer_regexes`: Infer the regex formats for ID columns based on the data. This feature is only available in SDV Enterprise.
  * (default) `True`: Infer regex format for columns with ID sdtype.
  * `False`: Do not infer regex format.
* `verbose`: A boolean that controls whether anything is printed out when detecting the metadata.
  * (default) `False`: Do not print anything out.
  * `True`: Print out the metadata for all the columns and tables as it's being detected. Use this as a quick check to ensure that the metadata is auto-detected correctly.

**Output** A Metadata object that describes the data

```python
from sdv.metadata import Metadata

metadata = Metadata.detect_from_dataframes(
    data={
        'hotels': hotels_dataframe,
        'guests': guests_dataframe
    })
```

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

{% hint style="danger" %}
**The detected metadata is not guaranteed to be accurate or complete.** Be sure to carefully inspect the metadata and update information
{% endhint %}

## Inspection API

At any point, during the metadata creation or updates, you can inspect the current state of the metadata.

### to\_dict

Use this to get a copy of the Python dictionary that corresponds to the metadata.

**Parameters** (None)

**Output** A Python dictionary that corresponds to the metadata

```python
python_dict = metadata.to_dict()
```

{% hint style="info" %}
Note that the returned object is a representation of the metadata. Changing it will not modify the original metadata object in any way.
{% endhint %}

### visualize

Use this to see a visual representation of the metadata. Use the parameters to control the level of details in the visualization and for saving the image.

**Parameters**&#x20;

* `show_table_details`: Toggle the display of column details

<table data-header-hidden><thead><tr><th width="212"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'full'</code></td><td>Show all the different column names, primary keys and foreign keys</td></tr><tr><td><code>'summarized'</code></td><td>Summarize the columns based on the data type</td></tr><tr><td><code>None</code></td><td><em>Hide the details. Only show the table name.</em></td></tr></tbody></table>

* `show_relationship_labels`: Toggle the display of the table relationships

<table data-header-hidden><thead><tr><th width="170"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>Label each relationship between 2 tables with the column names</td></tr><tr><td><code>False</code></td><td>Do not label the relationships. Only show an arrow between tables.</td></tr></tbody></table>

* `output_filepath`: If provided, save the image at the given location in the given format

{% hint style="warning" %}
The `output_filepath` must end with the filetype that you want to save as. Popular examples are `png`, `jpg` or `pdf`.
{% endhint %}

**Output** A [graphviz.graphs.Digraph](https://graphviz.readthedocs.io/en/stable/manual.html)

```python
metadata.visualize(
    show_table_details='full',
    show_relationship_labels=True,
    output_filepath='my_metadata.png'
)
```

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FP08Tdn7GbwbDH4Jo6dD2%2FMultiTable%20Metadata%20Schema.png?alt=media&amp;token=5a8a308c-3ee6-45c8-b4b4-e4058ecdd45b" alt=""><figcaption></figcaption></figure>

### get\_column\_names

Use this function to look up column names based on the table name or any other metadata properties that they have.

{% hint style="info" %}
This is particularly useful if you want to list all columns assigned to a specific sdtype, such as `unknown` in order to update it.
{% endhint %}

**Parameters**

* (required for multi-table) `table_name`: A string containing the name of the table. *This is required if you have multiple tables.*
* `sdtype`: A string describing the statistical data type.\
  Common types are, `'categorical'`, `'ordinal'`, `'numerical'`,  `'datetime'`, `'boolean'`, and `'id'`. But other types such as `'phone_number'` are also available (see [Sdtypes](/sdv/integration/metadata/sdtypes)).
  * (default) `None`: All columns for the table are returned
* `<other properties>`: Based on the sdtype, provide other parameters. For more information, see the [Metadata Spec](/sdv/integration/metadata).

**Output** A list of strings, with the column names that match the criteria. If no columns match the criteria, then an empty list will be returned.

```python
metadata.get_column_names(table_name='products', sdtype='id')
```

```python
[ 'product_id', 'product_code_name', 'code_type']
```

## Validation API

### validate

Use this to validate that the metadata is written according to the specification. This function will throw descriptive errors if there is anything wrong with the metadata.

**Parameters** (None)

**Output** (None)&#x20;

```python
metadata.validate()
```

```
InvalidMetadataError: The metadata is not valid

Error: Invalid values ("pii") for datetime column "start_date".
Error: Invalid regex format string "[A-{6}" for id column "hotel_id"
```

### validate\_data

Use this method to validate that the metadata accurately describes a particular dataset. This function will throw descriptive errors if there is any mismatch between the metadata and data.

**Parameters:**

* (required) `data`: A dictionary containing your multi-table data. Each key should be the name of a table and the value should be a pandas.DataFrame containing its data. The data should have the same tables and columns as described in the metadata.

**Output** (None)

```python
metadata.validate_data(data={
    'hotels': hotels_dataframe,
    'guests': guests_dataframe
})
```

### validate\_table

Use this method to validate that the metadata accurately describes a single data table. This function will throw descriptive errors if there is any mismatch between the metadata and data.

**Parameters:**

* (required) `data`: A pandas.DataFrame containing data. The data should have the same columns as described in the metadata.
* `table_name`: The name of the table. *This is required if you have multiple tables.*

**Output** (None)

```python
metadata.validate_table(data=my_dataframe)
```

**Output** (None)

## Update API

It is important to verify and update any inaccuracies in the metadata

### update\_column

Use this method to modify the information about a column in your metadata

**Parameters**

* (required) `column_name`:  The name of the column to update
* (required) `sdtype`: A string describing the statistical data type.\
  Common types are `'ordinal'`, `'categorical'`, `'numerical'`,  `'datetime'`, `'boolean'`, and `'id'`. But other types such as `'phone_number'` are also available. For more information, see [SDTypes docs](/sdv/integration/metadata/sdtypes).
* `table_name`: The name of the table. *This is required if you have multiple tables.*
* `<other properties>`: Based on the sdtype, provide other parameters. For more information, see [SDTypes docs](/sdv/integration/metadata/sdtypes).

**Output** (None)

```python
metadata.update_column(
    column_name='start_date',
    sdtype='datetime',
    table_name='guests',
    datetime_format='%Y-%m-%d',
    range_min='1970-01-01')
    
metadata.update_column(
    column_name='user_cell',
    sdtype='phone_number',
    table_name='guests',
    pii=True,
    range_is_nullable=False)
```

### update\_columns

Use this function to make a bulk update to multiple columns at once. This function will allow you to set the same parameters for a group of columns.

**Parameters**

* (required) `column_names`: A list of strings representing the column names to update. All columns must be in the table.
* (required) `sdtype`: A string describing the statistical data type.\
  Common types are `'categorical'`, `'ordinal'`, `'numerical'`, `'datetime'`, `'boolean'` and `'id'`. But other types such as `'phone_number'` are also available (see [Sdtypes](/sdv/integration/metadata/sdtypes)).
* `table_name`: The name of the table. *This is required if you have multiple tables.*
* `<other properties>`: Based on the sdtype, provide other parameters

**Output** (None)

```python
metadata.update_columns(
    column_names=['age', 'transactions', 'session_length'],
    sdtype='numerical',
    table_name='users'
)
```

### update\_columns\_metadata

Use this function to make a bulk update to multiple columns at once. This function will allow you to set the different parameters for each column

**Parameters**

* (required) `column_metadata`: A dictionary mapping each column name you want to update to the metadata information for that column. All columns must be in the table. For the exact format, see the [Metadata Spec](/sdv/integration/metadata).
* `table_name`: The name of the table. *This is required if you have multiple tables.*

**Output** (None)

```python
metadata.update_columns_metadata(
    column_metadata={
        'age': { 'sdtype': 'numerical', 'range_min': 18, 'range_max': 120 },
        'ssn': { 'sdtype': 'ssn', 'pii': True, 'range_is_nullable': True },
        'gender': { 'sdtype': 'categorical', 'range_values': ['M', 'F', 'X'] },
        'dob': { 'sdtype': 'datetime', 'datetime_format': '%Y-%m-%d' },
        ...
    },
    table_name='users',
)
```

### add\_column

Use this function to add a column to your metadata object.

**Parameters**

* (required) `column_name` : Name of the column to be added
* (required) `sdtype`: A string describing the statistical data type. Common types are `'categorical'`, `'ordinal'`, `'numerical'`, `'datetime'`, `'boolean'` and `'id'`. Other types such as `'phone_number'` are also available (see [Sdtypes](https://docs.sdv.dev/sdv/~/changes/T3ZD1DOoRUEqkmrAGBZp/reference/metadata-spec/sdtypes)).
* `table_name`: The name of the table. *This is required if you have multiple tables.*
* `**kwargs`: Any other parameters you need that describe metadata for a column.

**Output** (None)

```python
metadata.add_column(
  column_name='cell_phone_numbers',
  sdtype='phone_number',
  table_name='users',
  pii=True
)
```

### remove\_column

Use this function to remove a column from your metadata object.&#x20;

**Parameters**

* (required) `column_name`: The name of the column to be deleted
* `table_name`: The name of the table. *This is required if you have multiple tables.*

**Output** (None). This removes the column anywhere it appears in the metadata — in a relationship, as a primary key, etc.

```python
metadata.remove_column(
    column_name='cell_phone_numbers',
    table_name='users'
)
```

### add\_column\_relationship

Use this function to specify when a group of columns within the same table represent the same concept.

{% hint style="info" %}
While anyone can add column relationships to their data, SDV Enterprise users will see the highest quality data for the relationships. To learn more about the SDV Enterprise and its extra features, [visit our website](https://datacebo.com/pricing/).
{% endhint %}

**Parameters**

* (required)  `relationship_type`: A string with the type of relationship. This represents a higher level concept. See the tabs below for options.
* (required) `column_names`: A list of column names that are part of that relationship. Make sure that these columns are compatible with the relationship type. See the tabs below for more information.
* `table_name`: The name of the table. *This is required if you have multiple tables.*

{% tabs %}
{% tab title="address" %}
An address is defined by 2 or more columns that have the following sdtypes: `country_code`, `administrative_unit`, `state`, `state_abbr`, `city`, `postcode`, `street_address` and `secondary_address`.

```python
metadata.add_column_relationship(
    relationship_type='address',
    column_names=['addr_line1', 'addr_line2', 'city', 'zipcode', 'state']
)
```

{% endtab %}

{% tab title="gps" %}
A GPS coordinate pair is defined by 2 columns:&#x20;

* sdtype `latitude` &
* sdtype `longitude`

```python
metadata.add_column_relationship(
    relationship_type='gps',
    column_names=['location_lat', 'location_lon']
)
```

{% endtab %}

{% tab title="More coming soon!" %}
Additional column relationships coming soon!

*Do you have a request for a type of column relationship? Please* [*file a feature request*](https://github.com/sdv-dev/SDV/issues/new/choose) *describing your use case.*
{% endtab %}
{% endtabs %}

**Output** (None)

### set\_primary\_key

Use this function to set the primary key of the table. Any existing primary keys will be removed.

{% hint style="info" %}
The primary key uniquely identifies every row in the table. When you set a primary key, the SDV will guarantee that every value in the table is unique.
{% endhint %}

**Parameters**

* (required) `column_name`: The column name of the primary key. The column name must already be defined in the metadata and it must be an ID or another PII sdtype.\
  *＊If the table has a composite key, you may provide a list of column names instead; in this case, at least 1 of the columns in the composite key must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.*
* `table_name`: The name of the table. *This is required if you have multiple tables.*

**Output** (None)

```python
metadata.set_primary_key(
    column_name='hotel_id',
    table_name='hotels',
)
```

### remove\_primary\_key

Use this function to remove any existing primary keys in a table.

**Parameters**

* `table_name`: The name of the table. *This is required if you have multiple tables.*

**Output** (None) The primary key will be removed. Any existing relationships that use the primary key will be removed too.

```python
metadata.remove_primary_key(table_name='guests')
```

### add\_alternate\_keys

Use this function to set alternate keys of the table. This method will add to any existing alternate keys you may have.

{% hint style="info" %}
Similar to primary keys, alternate keys are also unique in your table. However, other tables do not reference alternate keys.
{% endhint %}

**Parameters**

* (required) `column_names`: A list of column names that represent the alternate keys in the table. All column names must already be defined in the metadata and they must be IDs or other PII sdtypes.\
  ＊ *If the table has a composite key, you may provide a list of lists; at least 1 of the columns in the composite key must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.*
* `table_name`: The name of the table. *This is required if you have multiple tables.*

**Output** (None)

```python
metadata.add_alternate_keys(
    column_names=['credit_card_number'],
    table_name='guests',
)
```

### add\_relationship

Use this method to add a relationship between 2 connected tables: A parent and child table. The parent table contains the primary key references while the child table has rows that refer to its parent. Multiple child rows can refer to the same parent row.

**Parameters:**

* (required) `parent_table_name`: The name of the parent table
* (required) `child_table_name`: The name of the child table that refers to the parent
* (required) `parent_primary_key`: The primary key column in the parent table. This column uniquely identifies each row in the parent table .\
  ＊ *If the table has a composite key, you may provide a list of column names instead. Only SDV Enterprise users can create synthesizers with composite keys.*
* (required) `child_foreign_key`: The foreign key column in the child table. The values in this column contain a reference to a row in the parent table\
  ＊ *If the table has a composite key, you may provide a list of column names instead. The length of this list should match the parent primary key. Only SDV Enterprise users can create synthesizers with composite keys.*

**Output** (None)

```python
metadata.add_relationship(
    parent_table_name='hotels',
    child_table_name='guests',
    parent_primary_key='hotel_id',
    child_foreign_key='hotel_id'
)
```

### remove\_relationship

Use this method to remove the connection between a parent and child table. In the case where there are multiple connections, this method will remove all the connections. Use this if the metadata has incorrectly detected relationships.

**Parameters:**

* (required) `parent_table_name`: The name of the parent table
* (required) `child_table_name`: The name of the child table that refers to the parent

**Output** (None)

```python
metadata.remove_relationship(
    parent_table_name='hotels',
    child_table_name='guests'
)
```

### add\_table

Use this method to add a new table to your metadata.

**Parameters:**

* (required) `table_name`: The name of the table to add.

**Output** (None)  After the table is added, the new table will be empty, so be sure to add columns to it using the [add\_column](#add_column) function.

```python
metadata.add_table(
    table_name='travel_details'
)
```

### remove\_table

Use this method to remove an existing table from your metadata.

**Parameters:**

* (required) `table_name`: The name of the table to remove

**Output** (None) The table will be removed, as well as any relationships that included the table.

```
metadata.remove_table(
    table_name='travel_details'
)
```

## Adding multi-sequence information

If your data includes multiple sequences, use these methods to add information about them.

### set\_sequence\_key

Use this function to set the sequence key of your table. Any existing sequence keys will be removed.

{% hint style="info" %}
The **sequence key** is a column that identify which row(s) belong to which sequences. This is usually an ID column but it may also be a PII sdtype (such as `"phone_number"`).  At this time, SDV does not support composite keys.

This is important for tables that contain multiple sequences. In our example, the sequence key is `'Patient ID'` because this column is used to break up the sequences.

If you don't supply a sequence key, the SDV assumes that your table only contains a single sequence. *Note: The SDV sequential models do not fully support single sequence data.*
{% endhint %}

**Parameters**

* (required) `column_name`: The column name of the sequence key. The column name must already be defined in the metadata and it must be an ID or another PII sdtype.
* `table_name`: The name of the table to apply the sequence key to. *This is required if you have multiple tables.*

**Output** (None)

```python
metadata.set_sequence_key(column_name='Patient ID')
```

### set\_sequence\_index

Use this function to set the sequence index of your table. Any existing sequence indices will be removed.

{% hint style="info" %}
The **sequence index** determines the spacing between the rows in a sequence. Use this if you have an explicit index such as a timestamp. If you don't supply a sequence index, the SDV assumes there is equal spacing of an unknown unit.
{% endhint %}

**Parameters**

* (required) `column_name`: The column name of the sequence index. The column name must already be defined in the metadata. It must be either a numerical or datetime column.
* `table_name`: The name of the table to apply the sequence index to. *This is required if you have multiple tables.*

**Output** (None)

```python
metadata.set_sequence_index(column_name='Time')
```

## Saving, Loading & Sharing Metadata

You can save the metadata object as a JSON file and load it again for future use.

### save\_to\_json

Use this to save the metadata object to a new JSON file that will be compatible with SDV 1.0 and beyond. We recommend you write the metadata to a new file every time you update it.

**Parameters**

* (required) `filepath`: The location of the file that will be created with the JSON metadata
* `mode`: A string describing the mode to use when creating the JSON file
  * (default) `'write'`: Write the metadata to the file, raising an error if the file already exists
  * `'overwrite'`: Write the metadata to the file, replacing the contents if the file already exists

**Output** (None)&#x20;

```python
metadata.save_to_json(filepath='metadata.json')
```

### load\_from\_json

Use this method to load your file as a Metadata object.

**Parameters**

* (required) `filepath`: The name of the file containing the JSON metadata

**Output:** A Metadata object.

### load\_from\_dict

Use this class method to load a Python dictionary as a `Metadata` object.

#### Parameters

* (required) `metadata_dict`: A Python dictionary representation of the metadata.

**Output** A Metadata object

```python
from sdv.metadata import Metadata

metadata_obj = Metadata.load_from_dict(metadata_dict)
```

### anonymize

Use this method to anonymize the column names of your metadata. This makes it easier to share your metadata, eg. for debugging purposes.

**Parameters** (None)

**Output** A new Metadata object that represents the anonymized metadata

```python
anonymized_metadata = original_metadata.anonymize()
```

{% hint style="info" %}
**The anonymized metadata contains new column names.** The original names are obfuscated, but the sdtypes and other formatting information remains the same.

```python
>>> anonymized_metadata.to_dict()
{
    'tables': {
        '3oc2d': {
            'primary_key': 'id_0',
            'columns': {
                'id_0': { 'sdtype': 'id', 'regex_format': 'ID_[0-9]{10}' },
                'num_0': { 'sdtype': 'numerical' },
                'num_1': { 'sdtype': 'numerical' },
                'cat_0': { 'sdtype': 'categorical' },
                'dt_0': { 'sdtype': 'datetime', 'datetime_format': '%Y-%m-%d' },
                'pii_0': { 'sdtype': 'ssn' },
                ...
            }
        }
    }
}
```

{% endhint %}

### copy

Use this method to create a copy of your metadata in its current state.

**Parameters** (None)

**Output** A new Metadata object that is a copy of the current metadata. Note that the original metadata and the copy are separate objects, so modifying one will not modify the other.

```python
metadata_copy = metadata.copy()
```


# Metadata JSON

This guide describes the metadata JSON spec.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FpPLDP2gOJBQvF5yF6Zmz%2Fmulti-table-data-data-preparation_May%208%202025.png?alt=media&amp;token=f23c6cac-43db-4e08-9eaf-5c850aacecfc" alt=""><figcaption><p>An example of multi table data. The tables are connected to each other through primary/foreign keys.</p></figcaption></figure>

<details>

<summary>Click to see the metadata JSON file</summary>

This is an example of a JSON file describing a multi table schema.

```json
{
    "METADATA_SPEC_VERSION": "V2",
    "tables": {
        "hotels": {
            "primary_key": "hotel_id",
            "columns": {
                "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" },
                "city": { "sdtype": "city", "range_is_nullable": false },
                "rating": { "sdtype": "numerical", "range_min": 0, "range_max": 5, "decimal_places": 1 }
            },
            "column_relationships": []
        },
        "guests": {
            "primary_key": "guest_email",
            "columns": {
                "guest_email": { "sdtype": "email", "range_is_nullable": false },
                "hotel_id": { "sdtype": "id", "regex_format": "HID_[0-9]{3}" },
                "checkin_date": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "01 Jan 1970", "range_max": "31 Dec 2050" },
                "checkout_date": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "01 Jan 1970", "range_max": "31 Dec 2050" },
                "room_type": { "sdtype": "categorical" }
            },
            "column_relationships": []
        }
    },
    "relationships": [{
        "parent_table_name": "hotels",
        "parent_primary_key": "hotel_id",
        "child_table_name": "guests",
        "child_foreign_key": "hotel_id"
    }]
}
```

</details>

{% hint style="success" %}
**Create your metadata programmatically.** Use the [Python API ](broken://pages/70dzL3ZTDgj8HBQH5gTc)to automatically detect the metadata based on your data.
{% endhint %}

## Overview

The metadata for a single table contains the following elements:

* (required) `"METADATA_SPEC_VERSION"`: The version of the metadata. If you are using SDV 2.0, the metadata version will be `"V2"`
* (required) `"tables"`: A dictionary that maps the table names to the table-specific metadata such as primary keys, column names and data types
* (required) `"relationships"`: A list of dictionaries that specify the connections between the tables

## Tables

The tables dictionary maps each table name to the table-specific metadata. This includes:

* (required) `"columns"`: A dictionary that maps the column names to the data types they represent and any other attributes.
* `"primary_key"`: The column name that is the primary key in the table. This must be an ID or PII sdtype.\
  ＊ *If the table has a composite key, this will be a list of column names instead; in this case, at least 1 of the columns must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.*
* `"alternate_keys"`: A list of column names that can act as alternate keys in the table\
  ＊ *If the table has a composite key, this will be a list of a list of column names instead; in this case, at least 1 of the columns must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.*

### Table Columns

When describing a column, you will provide the column name and the data type, known as the **sdtype**.&#x20;

The common sdtypes are: `"categorical"`, `"ordinal"`, `"numerical"`, `"datetime"`, `"boolean"` and `"id"`. Click on the type below to learn more about the type and how to specify it in the metadata.

{% tabs %}
{% tab title="categorical" %}
Categorical columns represent discrete, unordered data.

```json
{
    "tax_payer_status": {
        "sdtype": "categorical",
        "range_values": ["single", "married_filing_jointly", "married_filing_separately",
                         "head_of_household", "surviving_spouse"],
        "range_is_nullable": false
    },
    "tax_location_code": {
        "sdtype": "categorical",
        "high_cardinality": true
    }
}
```

**Properties:**

* `range_values`: A list of all possible non-null values that can appear in the categorical column. Because categorical data is unordered, this list can appear in any order. *This list may include category values that are not present in your training data.*
  * (default) If this is not provided, then SDV assumes that the full list of possible values is not known. It may look at your training data to determine the comprehensive list.
* `high_cardinality`: A true/false value describing whether the categorical column has a large number of possible category values that cannot be fully and meaningfully listed in `range_values`. We recommend setting this when there are >500 unique values. If this is set to true, then the `range_values` should not be present.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
    {% endtab %}

{% tab title="ordinal" %}
Ordinal columns represent discrete, ordered data.

```json
{
    "likert_responses": {
        "sdtype": "ordinal",
        "range_values": ["strongly_disagree", "disagree", "neither_agree_nor_disagree",
                         "agree", "strongly_agree"]
    }
}
```

**Properties:**

* `range_values`: A list of all possible non-null values that can appear in the ordinal column. Because ordinal data is ordered, this list should appear in order from lowest to highest. *This list may include ordinal values that are not present in your training data.*
  * (default) If this is not provided, then SDV assumes that the full list of possible values is not known. It may look at your training data to determine the comprehensive list.
* `high_cardinality`: A true/false value describing whether the ordinal column has a large number of possible values that cannot be fully and meaningfully listed in `range_values`. We recommend setting this when there are >500 unique values. If this is set to true, then the `range_values` should not be present.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
    {% endtab %}

{% tab title="numerical" %}
Numerical columns represents continuous numerical ranges of values.&#x20;

```json
{
    "age": { 
        "sdtype": "numerical",
        "range_min": 18,
        "range_max": 120,
        "decimal_places": 0
    },
    "transaction_amt": {
        "sdtype": "numerical",
        "range_is_nullable": false,
        "decimal_places": 2
    }
}
```

**Properties**

* `range_min`: The absolute lowest value that is possible for this column. *This might be lower than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the lowest value is not known. It may look at your training data to determine the lowest value.
* `range_max`: The absolute highest value that is possible for this column. *This might be higher than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the highest value is not known. It may look at your training data to determine the lowest value.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
* `decimal_places`: An integer representing the number of decimal places that the numerical values should be rounded to, for example `2` for 2 decimal digits. Note that a designation of `0` could mean either that there are whole numbers (integers) or that all values contain `.0` at the end such as `1.0`, `2.0`, etc.
  {% endtab %}

{% tab title="datetime" %}
Date columns represent a point in time

```json
{
    "start_date": { 
        "sdtype": "datetime",
        "datetime_format": "%Y-%m-%d",
        "range_min": "1970-01-01",
        "range_max": "2050-12-31"
    }
}
```

**Properties**

* (required) `datetime_format`: A string describing the format as defined by [Python's strftime module](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes).
* `range_min`: The absolute earliest datetime that is possible for this column. *This might be earlier than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the earliest value is not known. It may look at your training data to determine the earliest value.
* `range_max`: The absolute latest value that is possible for this column. *This might be later than what your training data contains.*
  * (default) If this is not provided, then SDV assumes that the latest value is not known. It may look at your training data to determine the latest value.
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
    {% endtab %}

{% tab title="id" %}
ID columns represent identifiers that do not have any special mathematical or semantic meaning

```json
"hotel_id": { 
    "sdtype": "id",
    "regex_format": "HID_[0-9]{3}"
}
```

**Properties**

* `regex_format`: A string describing the format of the ID as a [regular expression](https://docs.python.org/3/library/re.html)
  {% endtab %}

{% tab title="boolean" %}
Boolean columns represent True or False values.

```json
"is_active" : {
    "sdtype": "boolean"
}
```

**Properties**:

* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
    {% endtab %}

{% tab title="other" %}
You can input any other data type such as `'phone_number'`, `'ssn'` or `'email'`. See the [Sdtypes Reference](/sdv/integration/metadata/sdtypes#conceptual-sdtypes) for a full list.

```json
"guest_email": {
    "sdtype": "email",
    "pii": true
}
```

**Properties**

* `pii`: A boolean denoting whether the data is sensitive
  * (default) `true`: The column is sensitive, meaning the values should be anonymized&#x20;
  * `false`: The column is not sensitive, meaning the exact set of values can be reused in the synthetic data
* `range_is_nullable`: Whether the missing values are allowed in the column.
  * (default) If this is not provided, then SDV looks at your training data as the main example that  determines whether nulls should be allowed.
    {% endtab %}
    {% endtabs %}

### Column Relationships

Annotate groups of columns that represents higher level concepts. Denote the concept using the `"relationship_type"` keyword, followed by `"column_names"` with the list of columns involved. The column names can be present in any order.

Each relationship type supports different types of columns. Browse the table below to explore different options.

{% tabs %}
{% tab title="address" %}
An address is defined by 2 or more columns that have the following sdtypes: `country_code`, `administrative_unit`, `state`, `state_abbr`, `city`, `postcode`, `street_address` and `secondary_address`.

```python
{
    "type": "address",
    "column_names": ["addr_line1", "addr_line2", "city", "state", "zipcode"]
}
```

{% endtab %}

{% tab title="gps" %}
A GPS coordinate pair is defined by 2 columns:&#x20;

* sdtype `latitude` &
* sdtype `longitude`

```python
{
    "type": "gps",
    "column_names": ["location_lat", "location_lon"]
}
```

{% endtab %}

{% tab title="More coming soon!" %}
Additional column relationships coming soon!

*Do you have a request for a type of column relationship? Please* [*file a feature request*](https://github.com/sdv-dev/SDV/issues/new/choose) *describing your use case.*
{% endtab %}
{% endtabs %}

{% hint style="info" %}
While anyone can add column relationships to their data, SDV Enterprise users will see the highest quality data for the relationships. To learn more about the SDV Enterprise and its extra features, [visit our website](https://datacebo.com/pricing/).
{% endhint %}

## Relationships

A list of dictionary objects that describe the relationship between 2 connected tables, parent and child. The parent table contains the primary key references while the child table has rows that refer to its parent. Multiple child rows can refer to the same parent row.

* `"parent_table_name"`: The name of the parent table
* `"parent_primary_key"`: The primary key column in the parent table. This column uniquely identifies each row in the parent table. The column must be an ID or another PII sdtype.\
  ＊ *If the table has a composite key, this will be a list a list of column names instead; in this case, at least 1 of the columns must be an ID or another PII sdtype. Only SDV Enterprise users can create synthesizers with composite keys.*
* `"child_table_name"`: The name of the child table that refers to the parent
* `"child_foreign_key"`: The foreign key column in the child table. The values in this column contain a reference to a row in the parent table\
  ＊ *If the table has a composite key, this will be a list of column names instead. The length of this list should match the parent primary key. Only SDV Enterprise users can create synthesizers with composite keys.*

Use multiple dictionaries to represent multiple relationships.

```json
"relationships": [{
    "parent_table_name": "users",
    "parent_primary_key": "user_id",
    "child_table_name": "sessions",
    "child_foreign_key": "user_id"
}, {
    "parent_table_name": "sessions",
    "parent_primary_key": "session_id",
    "child_table_name": "transaction",
    "child_foreign_key": "transacted_session_id"
}]
```

## Multi Sequence Data

In some cases, you may have a table that describes multiple, ordered sequences of data such as the one shown below.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FzPsbsXbMULfrTaOTilK3%2Fsequential-data-data-preparation_May%208%202025.png?alt=media&amp;token=478612f6-1474-4ae8-99fa-dfec4304b96d" alt=""><figcaption><p>An example of sequential data. There are multiple sequences (one for each Patient ID). Within each sequence is an ordered set of rows.</p></figcaption></figure>

You can annotate the sequences within a table by applying the `sequence_key` and `sequence_index` keywords.

<details>

<summary>Click to see the metadata JSON</summary>

```json
{
    "METADATA_SPEC_VERSION": "V2",
    "tables": {
        "patients": {
            "sequence_key": "Patient ID",
            "sequence_index": "Time",
            "columns": {
                "Patient ID": { "sdtype": "id", "regex_format": "ID_[0-9]{3}" },
                "Address": { "sdtype": "physical_address", "pii": true, "range_is_nullable": false },
                "Smoker": { "sdtype": "boolean" },
                "Time": { "sdtype": "datetime", "datetime_format": "%d %b %Y", "range_min": "Jan 01 1970", "range_max": "Dec 31 2050" },
                "Heart Rate": { "sdtype": "categorical", "range_values": ["LOW", "RESTING", "ELEVATED"] },
                "Systolic BP": { "sdtype": "numerical", "range_min": 50, "range_max": 200 }
            }
        }
    }
}
```

</details>

**`"sequence_key"`**: A column name of the sequence key, if you have multi-sequence data

{% hint style="info" %}
The **sequence key** is a column that identify which row(s) belong to which sequences. This is usually an ID column but it may also be a PII sdtype (such as `"phone_number"`).&#x20;

This is important for tables that contain multiple sequences. In our example, the sequence key is `'Patient ID'` because this column is used to break up the sequences.

If you don't supply a sequence key, the SDV assumes that your table only contains a single sequence. *Note: The SDV sequential models do not fully support single sequence data.*
{% endhint %}

**`"sequence_index"`**: A column name of the sequence index, if you have sequential data

{% hint style="info" %}
The **sequence index** determines the spacing between the rows in a sequence. Use this if you have an explicit index such as a timestamp. If you don't supply a sequence index, the SDV assumes there is equal spacing of an unknown unit.
{% endhint %}


# Data Lineage

SDV's metadata object includes basic structural elements of a database that are typically found in a schema definition. For example:

* The type of data in each column
* The primary key of each table (or composite keys if you have them)
* Foreign keys that reference primary keys of other tables

However, you may have other rules in your dataset that are the result of data moving and evolving over time (part of data lineage flow). For example, only certain types of primary keys are allowed to be referenced from another table, as illustrated below.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FNDP7hSnEP6enF0xY0cNc%2Fforeign-to-primary-key-subset_May%208%202025.png?alt=media&amp;token=5ef39dc2-e6af-45f0-be1d-854402197bb5" alt=""><figcaption><p>Data lineage rules dictate that only accounts that are type "Premium" are allowed to have perks. That means that the foreign key from Perks can only reference a subset of Accounts.</p></figcaption></figure>

## Add data lineage rules as constraints

Data lineage rules cannot be added to the metadata, but they can be incorporated later into SDV's modeling step. When modeling, you can **automatically detect data lineage rules** as constraints and **add them to your synthesizer**. This ensures that the synthetic data you generate always adheres to the data lineage rules that are part of your dataset.

```python
from sdv.multi_table import HSASynthesizer

synthesizer = HSASynthesizer(metadata)

# detect data lineage logic as constraints
detected_constraints = synthesizer.detect_constraints(data)
synthesizer.add_constraints(detected_constraints)

# now the synthetic data includes data lineage rules
synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about constraints, please see the [Constraint-Augmented Generation](/sdv/modeling/constraint-augmented-generation-cag) docs.


# Utilities


# Cleaning Your Data

If you're importing local data, you may need to clean your data before using it.

## Data Cleaning API

### drop\_unknown\_references

Multi-table SDV synthesizers work best when your dataset has *referential integrity*, meaning that all the references in a foreign key refer to an existing value in the primary key. Use this function to drop rows that contain unknown references for your synthesizer.

**Parameters**

* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object
* (required) `data`: A dictionary that maps each table name to a pandas DataFrame containing data. This data should match your metadata.
* `drop_missing_values`: A boolean that describes whether to drop missing values in the foreign key
  * (default) `True`: If a foreign key has a missing value, treat it as an unknown reference and drop it. *We recommend this setting for maximum efficiency with SDV.*
  * `False`: If a foreign key has a missing value, treat it as a valid reference and keep it
* `verbose`: A boolean that controls whether to print out a summary of the results
  * (default) `True`: Print a summary of the number of rows that are dropped from each table

**Output** A dictionary that maps each table name to a pandas DataFrame containing data. The data will contain referential integrity, meaning that there will be no unknown foreign key references.

```python
from sdv.utils import drop_unknown_references

cleaned_data = drop_unknown_references(data, metadata)
```

```
Success! All foreign keys have referential integrity. 

Table Name    # Rows (Original)    # Invalid Rows   # Rows (New)
sessions      1200                 50               1150     
transactions  5000                 0                5000
```

### simplify\_schema

Use this function to simplify a complex multi-table dataset into only a few small tables. This makes it easier to iterate when performing a proof-of-concept using SDV Community. Keep in mind that SDV Community synthesizers like HMA are not designed to handle large, complex datasets.

{% hint style="success" %}
After completing your proof-of-concept, purchase [SDV Enterprise](https://datacebo.com/pricing/) for access to scalable synthesizers. The synthesizers in SDV Enterprise support unlimited tables with complex schemas, so you will no longer have to simplify the schema.
{% endhint %}

**Parameters**

* (required) `data`: A dictionary that maps each table name to a pandas DataFrame containing data. This data should match your metadata.
* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object that describes your data.

**Output**:&#x20;

* `simplified_data`: A dictionary that maps a table name to a pandas DataFrame containing the data. The simplified data schema may have fewer tables and columns than the original.
* `simplified_metadata`: A Metadata object that describes the simplified data

```python
from sdv.utils import poc

simplified_data, simplified_metadata = poc.simplify_schema(data, metadata)
```

### get\_random\_subset

Use this function to randomly subsample data from a single table of data. This function will keep the same overall schema (and columns) but it will reduce the number of rows in each table.

```python
from sdv.utils import poc

subsampled_data = poc.get_random_subset(
    data, 
    metadata,
    main_table_name="users", 
    num_rows=100
)
```

**Parameters**

* (required) `data`: Your full dataset. A dictionary that maps a table name to a pandas DataFrame containing the data
* (required) `metadata`: A Metadata object that describes the data
* (required) `main_table_name`: A string with the name of the most important table in your dataset. We'll make sure the subsample is optimized for this main table.
* (required) `num_rows`: The number of rows to subsample from the main table. All other table's sizes will be algorithmically determined based on this.
* `verbose`: Whether to print out the results
  * (default) `True`: Print out how many rows were included from each table
  * `False`: Do not print anything out

**Output** A dataset with fewer rows than before. The dataset will continue to have *referential integrity* meaning that there will be no invalid or missing references between the tables.

### get\_random\_sequence\_subset

Use this function to subsample data for a sequential dataset. Given multi-sequence data, this function will randomly select sequences and clip them to the desired length.

```python
from sdv.utils import get_random_sequence_subset

subsampled_data = get_random_sequence_subset(
    data, 
    metadata,
    num_sequences=10,
    max_sequence_length=100,
    long_sequence_subsampling_method='last_rows'
)
```

**Parameters**

* (required) `data`: A dictionary that maps each table name to a pandas DataFrame containing data. This data should match your metadata.
* (required) `metadata`: A Metadata object that describes the data. The metadata must describe multi-sequence data, meaning that it must have a sequence key specified.
* (required) `num_sequences`: An int describing the number of sequences to subsample from the data
* `table_name`: The name of the table from which to get random sequences. *This is required if you have multiple tables.*
* `max_sequence_length`: The maximum length each sequence is allowed to be
  * (default) `None`: Do not enforce any max length, meaning that entire sequences will appear in the subsampled data
  * `<integer>`: An integer describing the max sequence length. Any sequence that is longer than this value will be shortened based on the method below
* `long_sequence_subsampling_method`: The method for shortening sequences that are too long
  * (default) `'first_rows'`: Keep the first *n* rows of each sequence as they appear, where *n* is the max sequence length
  * `'last_rows'`: Keep the last *n* rows of each sequence as they appear, where *n* is the max sequence length
  * `'random'`: Randomly choose *n* rows of each sequence, where *n* is the max sequence length. Note the randomly chosen rows will still appear in the same order as the original data.

**Output** A dataset with fewer rows than before. The dataset will continue to represent multiple sequences of potentially varying lengths.


# Demo Data

The SDV library contains many different demo datasets that you can use to get started. Use the `demo` module to access the datasets and their metadata.

```python
from sdv.datasets.demo import get_available_demos
from sdv.datasets.demo import download_demo

get_available_demos(modality='multi_table')

data, metadata = download_demo(
    modality='multi_table',
    dataset_name='fake_hotels'
)
```

{% hint style="warning" %}
The `demo` module accesses the SDV's public dataset repository. These methods require an internet connection.
{% endhint %}

### get\_available\_demos

Use this method to get information about all the available demos in the SDV's public dataset repository.

**Parameters**

* (required) `modality`: A string containing the type of demo data. Options are:
  * `'single_table'`: For data contained in a single table, where each row is independent.
  * `'multi_table'`: For data contained in a relational, multi-table dataset. Tables can reference each other.
  * `'sequential'`: For data representing ordered sequences. The sequences should be present in a single table.
* *\[private]* `s3_bucket_name`: Additional, private demo datasets may be available in other buckets. If you have been given access to one of these buckets, you can input the name as a string. *This is currently only available to internal, DataCebo users.*
* *\[private]* `credentials`: Provide a dictionary with the SDV credentials in order to access the bucket. The expected keys are `'username'` and `'license_key'`. *This is currently only available to internal, DataCebo users.*

**Returns** A pandas DataFrame object containing the name of the dataset, its size (in MB) and the number of tables it contains.

```python
from sdv.datasets.demo import get_available_demos

get_available_demos(modality='multi_table')
```

```
dataset_name            size_MB        num_tables
Accidents_v1            172.3          3
airbnb-simplified       371.5          2
Atherosclerosis_v1      2.9            4                     
...                     ...            ...
```

### download\_demo

Use this method to download a demo dataset from the SDV's public dataset repository.

**Parameters**

* (required) `modality`: A string containing the type of demo data. Options are:
  * `'single_table'`: For data contained in a single table, where each row is independent.
  * `'multi_table'`: For data contained in a relational, multi-table dataset. Tables can reference each other.
  * `'sequential'`: For data representing ordered sequences. The sequences should be present in a single table.
* (required) `dataset_name`: A string with the name of the demo dataset. You can use any of the dataset names from the `get_available_demos` method.
* `output_folder_name`: A string with the name of a folder. If provided, this method will download the data and metadata into the folder, in addition to returning the data.
  * (default) `None`: Do not save the data into a folder. The data will still be returned so that you can use it in your Python script.
* *\[private]* `s3_bucket_name`: Additional, private demo datasets may be available in other buckets. If you have been given access to one of these buckets, you can input the name as a string. *This is currently only available to internal, DataCebo users.*
* *\[private]* `credentials`: Provide a dictionary with the SDV credentials in order to access the bucket. The expected keys are `'username'` and `'license_key'`. *This is currently only available to internal, DataCebo users.*

**Output** A tuple `(data, metadata)`.

The `data` is a dictionary that maps each table name to a pandas DataFrame containing the demo data for that table. The `metadata` is a [Metadata](/sdv/integration/metadata) object that describes the data.

```python
from sdv.datasets.demo import download_demo

data, metadata = download_demo(
    modality='multi_table',
    dataset_name='fake_hotels'
)

guests_table = data['guests']
hotels_table = data['hotels']
```

### get\_source

Some datasets have a source file that describes where the dataset comes from. This can include information like a URL, citations for the original publication, and other information that tracks the dataset's provenance. Use this function to get all this source information.

**Parameters**

* (required) `modality`: A string containing the type of demo data. Options are:
  * `'single_table'`: For data contained in a single table, where each row is independent.
  * `'multi_table'`: For data contained in a relational, multi-table dataset. Tables can reference each other.
  * `'sequential'`: For data representing ordered sequences. The sequences should be present in a single table.
* (required) `dataset_name`: A string with the name of the demo dataset. You can use any of the dataset names from the `get_available_demos` method.
* `output_filepath`: A string with the name of a file path. If provided, this method will create the file and write the source information to the file, in addition to returning it.
  * (default) `None`: Do not save the source information into a file. The contents will still be returned so that you can print it out and read it.
* *\[private]* `s3_bucket_name`: Additional, private demo datasets may be available in other buckets. If you have been given access to one of these buckets, you can input the name as a string. *This is currently only available to internal, DataCebo users.*
* *\[private]* `credentials`: Provide a dictionary with the SDV credentials in order to access the bucket. The expected keys are `'username'` and `'license_key'`. *This is currently only available to internal, DataCebo users.*

**Output** A string containing the contents of the source information. You can print it out to read it. *(If no source information is available for a dataset, the function returns `None` and no file will be written.)*

```python
from sdv.datasets.demo import get_source

source_text = get_source(
    modality='multi_table',
    dataset_name='Bupa',
    output_filepath='Bupa_source.txt'
)
print(source_text)
```

```
Source URL: https://archive.ics.uci.edu/dataset/60/liver+disorders
License name: Creative Commons Attribution 4.0 International

Citations:
[1] Liver Disorders [Dataset]. (2016). UCI Machine Learning Repository. https://doi.org/10.24432/C54G67.
```

### get\_readme

Some datasets have a README file that describes more information about what the dataset means. This could include explanations for naming conventions used in the dataset, mappings for ID codes, or business logic. Use this function to get the README (if it exists).

{% hint style="info" %}
**README information is coming soon!** At this time, SDV demo datasets do not contain any README information. If you're looking for more information about the dataset, we recommend [getting the source](#get_source). From there, you'll be able to navigate to any URLs or contact the original authors as needed. &#x20;
{% endhint %}

**Parameters**

* (required) `modality`: A string containing the type of demo data. Options are:
  * `'single_table'`: For data contained in a single table, where each row is independent.
  * `'multi_table'`: For data contained in a relational, multi-table dataset. Tables can reference each other.
  * `'sequential'`: For data representing ordered sequences. The sequences should be present in a single table.
* (required) `dataset_name`: A string with the name of the demo dataset. You can use any of the dataset names from the `get_available_demos` method.
* `output_filepath`: A string with the name of a file path. If provided, this method will create the file and write the README information to the file, in addition to returning it.
  * (default) `None`: Do not save the README information into a file. The contents will still be returned so that you can print it out and read it.
* *\[private]* `s3_bucket_name`: Additional, private demo datasets may be available in other buckets. If you have been given access to one of these buckets, you can input the name as a string. *This is currently only available to internal, DataCebo users.*
* *\[private]* `credentials`: Provide a dictionary with the SDV credentials in order to access the bucket. The expected keys are `'username'` and `'license_key'`. *This is currently only available to internal, DataCebo users.*

**Output** A string containing the contents of the README information. You can print it out to read it. *(If no README information is available for a dataset, the function returns `None` and no file will be written.)*

### save\_resource

This is a generic function to download and save any resource related to a dataset such as the data, metadata, readme, or source.&#x20;

**Parameters**

* (required) `modality`: A string containing the type of demo data. Options are:
  * `'single_table'`: For data contained in a single table, where each row is independent.
  * `'multi_table'`: For data contained in a relational, multi-table dataset. Tables can reference each other.
  * `'sequential'`: For data representing ordered sequences. The sequences should be present in a single table.
* (required) `dataset_name`: A string with the name of the demo dataset. You can use any of the dataset names from the `get_available_demos` method.
* (required) `resource_filepath`: A string with the name of the resources to download and save. The provided path should be relative based on the dataset.
  * Current options include: `"data.zip"`, `"metadata.json"`, `"README.txt"`, or `"SOURCE.txt"`. Not all resources are available for every dataset. In the future, additional resources may be available too.
* (required) `output_filepath`: A string with the name of a file path. If provided, this method download and save the resource locally at the filepath provided.
* *\[private]* `s3_bucket_name`: Additional, private demo datasets may be available in other buckets. If you have been given access to one of these buckets, you can input the name as a string. *This is currently only available to internal, DataCebo users.*
* *\[private]* `credentials`: Provide a dictionary with the SDV credentials in order to access the bucket. The expected keys are `'username'` and `'license_key'`. *This is currently only available to internal, DataCebo users.*
* *(deprecated) `resource_filename`: Please use `resource_filepath` instead.*

**Output**: None. The requested resource is saved to the file path provided. If the resource doesn't exist for the dataset, you will see a warning and nothing will be saved.

```python
from sdv.datasets.demo import save_resource

save_resource(
    modality='multi_table',
    dataset_name='Bupa',
    resource_filename='metadata.json',
    output_filepath='Bupa_metadata.json'
)
```


# Single Table Synthesizers

SDV offers a variety of synthesizers, which use different algorithms to generate synthetic data. This section describes single table synthesizers, which act on [single table data](/sdv/integration/overview/data-modalities).

## Basic Single Table Synthesizers

These synthesizers are available in SDV Community. They build a generative AI model using your training data, and use it to create synthetic data.

{% hint style="success" %}
**We recommend starting with** [**GaussianCopulaSynthesizer**](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) **for fast performance**, good quality, and customization.

For higher fidelity, try a neural network-based synthesizer such as [CTGANSynthesizer](/sdv/modeling/single-table-synthesizers/ctgansynthesizer) or [TVAESynthesizer](/sdv/modeling/single-table-synthesizers/tvaesynthesizer). Modeling and sampling performance may be slower for these synthesizers, especially if you have categorical columns with many different values (high cardinality).
{% endhint %}

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer"><strong>GaussianCopulaSynthesizer</strong></a></td><td>Use a classical ML algorithm to learn from real data. This is fast, transparent, and customizable.</td><td><a href="/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer">GaussianCopulaSynthesizer</a></td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/ctgansynthesizer"><strong>CTGANSynthesizer</strong></a></td><td>Use GAN-based ML algorithm to learn from real data. This may take longer to learn and be harder to debug.</td><td><a href="/sdv/modeling/single-table-synthesizers/ctgansynthesizer">CTGANSynthesizer</a></td></tr><tr><td><a href="/sdv/modeling/single-table-synthesizers/tvaesynthesizer"><strong>TVAE Synthesizer</strong></a></td><td>Use a variational autoencoder ML model to learn from real data. This may take longer to learn and be harder to debug.</td><td><a href="/sdv/modeling/single-table-synthesizers/tvaesynthesizer">TVAESynthesizer</a></td></tr></tbody></table>

*Experimental synthesizer: The* [***CopulaGANSynthesizer***](/sdv/modeling/single-table-synthesizers/copulagansynthesizer) *combines classical statistics with GAN-based modeling.*

## Specialty Synthesizers

Specialty synthesizers are available for special situations — such as improving speed, enhancing quality, or providing privacy guarantees.

{% hint style="info" %}
*Specialty synthesizers available for licensed, SDV Enterprise users (denoted by **＊**) or through purchasing additional bundles (denoted by ❖). For more information, see the* [*feature comparison*](/sdv/explore/compare-features)*.*
{% endhint %}

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td>❖ <a href="/sdv/modeling/single-table-synthesizers/xgcsynthesizer"><strong>XGCSynthesizer</strong></a> </td><td>Use extra features on top of Gaussian Copula for higher quality synthetic data and improved performance.</td><td><a href="/sdv/modeling/single-table-synthesizers/xgcsynthesizer">❖ XGCSynthesizer</a></td></tr><tr><td>❖ <a href="/sdv/modeling/single-table-synthesizers/bootstrapsynthesizer"><strong>BootstrapSynthesizer</strong></a></td><td>A synthesizer that is optimized to learn from a smaller number of training rows.</td><td></td></tr><tr><td>❖ <a href="/sdv/modeling/single-table-synthesizers/segmentsynthesizer"><strong>SegmentSynthesizer</strong></a></td><td>Use this synthesizer when your real data is highly segmented, with different patterns for each.</td><td><a href="/sdv/modeling/single-table-synthesizers/segmentsynthesizer">❖ SegmentSynthesizer</a></td></tr><tr><td><strong>＊</strong><a href="/sdv/modeling/single-table-synthesizers/dayzsynthesizer"><strong>DayZSynthesizer</strong></a></td><td>Generate synthetic data from scratch. Use this when you don't have a lot of real data.</td><td></td></tr><tr><td>❖ <a href="/sdv/modeling/single-table-synthesizers/dpgcsynthesizer"><strong>DPGCSynthesizer</strong></a></td><td>Use Gaussian Copula while guaranteeing differential privacy.</td><td><a href="/sdv/modeling/single-table-synthesizers/dpgcsynthesizer">❖ DPGCSynthesizer</a></td></tr><tr><td>❖ <a href="/sdv/modeling/single-table-synthesizers/dpgcflexsynthesizer"><strong>DPGCFlexSynthesizer</strong></a></td><td><em>[Experimental]</em> Use and customize Gaussian Copula while guaranteeing differential privacy.</td><td><a href="/sdv/modeling/single-table-synthesizers/dpgcflexsynthesizer">❖ DPGCFlexSynthesizer</a></td></tr></tbody></table>


# GaussianCopulaSynthesizer

The Gaussian Copula Synthesizer uses classic, statistical methods to train a model and generate synthetic data.

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

```python
from sdv.single_table import GaussianCopulaSynthesizer

synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = GaussianCopulaSynthesizer(
    metadata, # required
    enforce_min_max_values=True,
    enforce_rounding=False,
    numerical_distributions={
        'amenities_fee': 'beta',
        'checkin_date': 'uniform'
    },
    default_distribution='norm'
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data.</td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`numerical_distributions`**: Set the distribution shape of any numerical columns that appear in your table. Input this as a dictionary, where the key is the name of the numerical column and the values is a numerical distribution.

```python
numerical_distributions = {
    <column name>: 'norm',
    <column name>: 'uniform', 
    ...
}
```

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>Use the default distribution for the column name.</td></tr><tr><td><code>&#x3C;dictionary></code></td><td>Apply the given distribution to each column name. The distribution name should be one of: <code>'norm'</code> <code>'beta'</code>, <code>'truncnorm'</code>, <code>'uniform'</code>, <code>'gamma'</code> or <code>'gaussian_kde'</code> </td></tr></tbody></table>

**`default_distribution`**: Set the distribution shape to use by default for all columns. Input this as a single string.

<table data-header-hidden><thead><tr><th width="204.0625"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'beta'</code></td><td>Model the column as a beta distribution</td></tr><tr><td><code>&#x3C;distribution_name></code></td><td>Model the column as the given distribution. The distribution name should be one of: <code>'norm'</code> <code>'beta'</code>, <code>'truncnorm'</code>, <code>'uniform'</code>, <code>'gamma'</code> or <code>'gaussian_kde'</code> </td></tr></tbody></table>

{% hint style="warning" %}
Setting the distribution to `'gaussian_kde'` increases the time it takes to train your synthesizer.
{% endhint %}

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'enforce_min_max_values': True,
    'enforce_rounding': False
    'default_distribution': 'beta',
    'numerical_distributions': {
        'amenities_fee': 'beta',
        'checkin_date': 'uniform'
    },
    ...
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer uses Gaussian Copulas to learn the overall distribution of the real data. This happens in two stages:

1. Learning the distribution of each individual column, also known as the marginal distribution. For example a beta distribution with α=2 and β=5. The synthesizer uses the learned distribution to normalize the values, creating normal curves with µ=0 and σ=1.&#x20;
2. Learning the covariance of each pair of normalized columns. This is stored as an *n* x *n* matrix, where *n* is the number of columns in the table.

The [Synthetic Data Vault paper](https://dai.lids.mit.edu/wp-content/uploads/2018/03/SDV.pdf) has more information about the Gaussian normalization process and the Copula estimations.
{% endhint %}

### get\_learned\_distributions

After fitting this synthesizer, you can access the marginal distributions that were learned to estimate the shape of each column.

**Parameters** None

**Output** A dictionary that maps the name of each learned column to the distribution that estimates its shape

```python
synthesizer.get_learned_distributions()
```

```
{
    'amenities_fee': {
        'distribution': 'beta',
        'learned_parameters': { 'a': 2.22, 'b': 3.17, 'loc': 0.07, 'scale': 48.5 }
    },
    'checkin_date': { 
        ...
    },
    ...
}
```

For more information about the distributions and their parameters, visit the[ Copulas library](https://sdv.dev/Copulas/).

{% hint style="info" %}
Learned parameters are only available for parametric distributions. For eg. you will not be able to access learned distributions for the `'gaussian_kde'` technique.

In some cases, the synthesizer may not be able to fit the exact distribution shape you requested, so you may see another distribution shape as a fallback (eg. `'truncnorm'` instead of `'beta'`).
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the Gaussian Copula algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Custom Preprocessing](/sdv/modeling/configuration/preprocessing).

</details>

<details>

<summary>Why is <code>'beta'</code> the default distribution &#x26; when should I change it?</summary>

To create high quality synthetic data, the synthesizer should be able to match the shape of data for some optimal set of parameters. (The synthesizer learns and optimizes the parameters.)

We chose `'beta'` as the default distribution because it's capable of matching a variety of different shapes. It's also time efficient compared to other distributions like `'gaussian_kde'`.

**This default is not guaranteed to work on every dataset.** Consider changing the default distribution if all your columns have specific characteristics that you want to capture. If you have only a few columns that are highly important to match, then you can set those shapes specifically using the `numerical_distributions` parameter.

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you've previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>


# CTGANSynthesizer

The CTGAN Synthesizer uses GAN-based, deep learning methods to train a model and generate synthetic data.

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

```python
from sdv.single_table import CTGANSynthesizer

synthesizer = CTGANSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
from sdv.single_table import CTGANSynthesizer

synthesizer = CTGANSynthesizer(
    metadata, # required
    enforce_rounding=False,
    epochs=500,
    verbose=True
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data.</td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`epochs`**: Number of times to train the GAN. Each new epoch can improve the model.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>300</code></td><td>Run all the data through the Generator and Discriminator 300 times during training</td></tr><tr><td><code>&#x3C;number></code></td><td>Train for a different number of epochs. Note that larger numbers will increase the modeling time.</td></tr></tbody></table>

**`verbose`**: Control whether to print out the results of each epoch. You can use this to track the training time as well as the improvements per epoch.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>False</code></td><td>Do not print out any results</td></tr><tr><td><code>True</code></td><td>Print out the Generator and Discriminator loss values per epoch. The loss values indicate how well the GAN is currently performing, lower values indicating higher quality.</td></tr></tbody></table>

**`enable_gpu`**: Whether to enable GPU usage when training the synthesizer. This may speed up the modeling time.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>If available, use the GPU to speed up modeling time. Currently, this will look for either <a href="https://developer.nvidia.com/how-to-cuda-python">CUDA</a> (on Linux/Windows machines) or <a href="https://developer.apple.com/metal/pytorch/">MPS</a> (on Mac).  If this is not available, then the GPU will not be used.  </td></tr><tr><td><code>False</code></td><td>Do not use the GPU to speed up modeling time.</td></tr></tbody></table>

{% hint style="info" %}
**Is my synthesizer using the GPU?** After calling `fit`, you may notice that the GPU is not immediately used. This is because GPU is not used for the initial, data preprocessing step. After data preprocessing is complete, you should then see the GPU being used for the neural network training.
{% endhint %}

*(deprecated) `cuda`: Please use the `enable_gpu` option to use CUDA, if it's available on your platform.*

{% hint style="info" %}
**Looking for more customizations?** Other settings are available to fine-tune the architecture of the underlying GAN used to model the data. Click the section below to expand.
{% endhint %}

<details>

<summary>Click to expand additional GAN customization options</summary>

These settings are specific to the GAN. Use these settings if you want to optimize the technical architecture and modeling.

**`batch_size`**: Number of data samples to process in each step. This value must be even, and it must be divisible by the `pac` parameter (see below). Defaults to `500`.&#x20;

**`discriminator_dim`**: Size of the output samples for each one of the Discriminator Layers. A Linear Layer will be created for each one of the values provided. Defaults to `(256, 256)`.

**`discriminator_decay`**: Discriminator weight decay for the Adam Optimizer. Defaults to `1e-6`.

**`discriminator_lr`**: Learning rate for the discriminator. Defaults to `2e-4`.

**`discriminator_steps`**: Number of discriminator updates to do for each generator update. Default `1` to match the original CTGAN implementation

**`embedding_dim`**: Size of the random sample passed to the Generator. (Default `128`)

**`generator_decay`**: Generator weight decay for the Adam Optimizer. Defaults to `1e-6`

**`generator_dim`**: Size of the output samples for each one of the Residuals. A Residual Layer will be created for each one of the values provided. Defaults to `(256, 256)`.

**`generator_lr`**: Learning rate for the generator. Defaults to `2e-4`.

**`log_frequency`**: Whether to use log frequency of categorical levels in conditional sampling. Defaults to `True`.

**`pac`**: Number of samples to group together when applying the discriminator. Defaults to 10.

</details>

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'enforce_rounding': False,
    'epochs': 500,
    'verbose': True,
    ...
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer uses the CTGAN to learn a model from real data and create synthetic data. The CTGAN uses generative adversarial networks (GANs) to model data, as described in the [Modeling Tabular data using Conditional GAN](https://arxiv.org/pdf/1907.00503.pdf) paper which was presented at the NeurIPS conference in 2019.
{% endhint %}

### get\_loss\_values

After fitting, you can access the loss values computed during each epoch for both the generator and discriminator.

**Parameters** (None)

**Output** A pandas.DataFrame object containing epoch number, generator loss value and discriminator loss value.

```python
synthesizer.get_loss_values()
```

```python
Epoch  Generator Loss  Discriminator Loss
1      1.7863          -0.3639
2      1.5484          0.2260
3      1.3633          -0.0441
...
```

### get\_loss\_values\_plot

After fitting, you can plot the loss values at each epoch, for both the generator and discriminator.

**Parameters** (None)

**Output** A [plotly Figure](https://plotly.com/python-api-reference/generated/plotly.graph_objects.Figure.html) object that plots the loss values per epoch

{% hint style="info" %}
Use `fig.show()` to see the plot in an iPython notebook. The plot is interactive, allowing you to zoom, scroll and take screenshots.
{% endhint %}

```python
fig = synthesizer.get_loss_values_plot()
fig.show()
```

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F8ftnGvFYK1fcSUdDXh69%2Fctgan_loss_plot.png?alt=media&amp;token=67a0c396-7603-4042-b4f5-2dff450ee72e" alt=""><figcaption></figcaption></figure>

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the CTGAN algorithm is designed for complete data with non-missing values, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/JtxkSK060ZygzbDLf4xl).

</details>

<details>

<summary>How do I tune the hyperparameters? (Such as <code>epochs</code> or other values)</summary>

Unfortunately, there is no one-size-fits-all solution for this question! The optimal hyperparameters depend on both the complexity of your dataset and the metrics you are using to quantify success.

* **Epochs** is a well-studied parameter. Our experiments suggest that increasing the number of epochs helps up until a certain inflection point. After this, there is no significant improvement. Keep in mind that increasing the epochs also increases the training time. More information is available in [this discussion](https://github.com/sdv-dev/SDV/discussions/980) and [blog post](https://datacebo.com/blog/interpreting-ctgan-progress/).
* For **other hyperparameters**, you will have to do some experimentation yourself. You may have luck using external hyperparameter tuning libraries. Usually, these libraries test a combination of hyperparameters to determine the best set for your desired goal or metric.

</details>

<details>

<summary>The <code>verbose</code> setting is reporting negative loss. Is that ok?</summary>

Yes, both the generator and discriminator are trained to minimize the loss value, which often becomes negative. Our experiments suggest that the generator loss frequently tends to become more negative as the epochs progress, while the discriminator loss tends to fluctuate at around 0.

Other trends are possible for different datasets. But look out for cases where the generator and discriminator loss values are not converging -- this may indicate that you need to modify the GAN's architecture or that your data is not suitable for CTGAN.

More information is available in [this discussion](https://github.com/sdv-dev/SDV/discussions/980).

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you've previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>

<details>

<summary>What is the difference between CTGANSynthesizer and the CTGAN Library?</summary>

The `CTGANSynthesizer` is part of the SDV library. It provides an end-to-end workflow for creating synthetic data, which includes:

1. Pre-processing the data and handing constraints
2. Running the core ML algorithm (CTGAN) on the preprocessed data
3. Post-processing synthetic data to the correct format and specifications

Step #2 uses the [CTGAN Library](https://github.com/sdv-dev/CTGAN), which contains just the core machine learning algorithm (GAN implementation).

</details>

<details>

<summary>How do I cite CTGAN?</summary>

*Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni*. **Modeling Tabular data using Conditional GAN**. NeurIPS, 2019.

```
@inproceedings{ctgan,
   title={Modeling Tabular data using Conditional GAN},
   author={Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
   booktitle={Advances in Neural Information Processing Systems},
   year={2019}
}
```

</details>


# TVAESynthesizer

The TVAE Synthesizer uses a variational autoencoder (VAE)-based, neural network techniques to train a model and generate synthetic data.

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

```python
from sdv.single_table import TVAESynthesizer

synthesizer = TVAESynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = TVAESynthesizer(
    metadata, # required
    enforce_min_max_values=True,
    enforce_rounding=False,
    epochs=500
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data.</td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`epochs`**: Number of times to train the VAE. Each new epoch can improve the model.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>300</code></td><td>Run all the data through the VAE 300 times during training</td></tr><tr><td><code>&#x3C;number></code></td><td>Train for a different number of epochs. Note that larger numbers will increase the modeling time.</td></tr></tbody></table>

**`verbose`**: Control whether to print out the results of each epoch. You can use this to track the training time as well as the improvements per epoch.

<table data-header-hidden><thead><tr><th width="195"></th><th></th></tr></thead><tbody><tr><td>(default) <code>False</code></td><td>Do not print out any results</td></tr><tr><td><code>True</code></td><td>Print out the loss values per epoch. The loss values indicate how well the VAE is currently performing, lower values indicate higher quality.</td></tr></tbody></table>

**`enable_gpu`**: Whether to enable GPU usage when training the synthesizer. This may speed up the modeling time.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>If available, use the GPU to speed up modeling time. Currently, this will look for either <a href="https://developer.nvidia.com/how-to-cuda-python">CUDA</a> (on Linux/Windows machines) or <a href="https://developer.apple.com/metal/pytorch/">MPS</a> (on Mac).  If this is not available, then the GPU will not be used.  </td></tr><tr><td><code>False</code></td><td>Do not use the GPU to speed up modeling time.</td></tr></tbody></table>

{% hint style="info" %}
**Is my synthesizer using the GPU?** After calling `fit`, you may notice that the GPU is not immediately used. This is because GPU is not used for the initial, data preprocessing step. After data preprocessing is complete, you should then see the GPU being used for the neural network training.
{% endhint %}

*(deprecated) `cuda`: Please use the `enable_gpu` option to use CUDA, if it's available on your platform.*

{% hint style="info" %}
**Looking for more customizations?** Other settings are available to fine-tune the architecture of the neural network used to model the data. Click the section below to expand.
{% endhint %}

<details>

<summary>Click to expand additional VAE customization options</summary>

These settings are specific to the neural network. Use these settings if you want to optimize the technical architecture and modeling.

**`batch_size`**: Number of data samples to process in each step.

**`compress_dims`**: Size of each hidden layer in the encoder. Defaults to `(128, 128)`.

**`decompress_dims`**: Size of each hidden layer in the decoder. Defaults to `(128, 128)`.

**`embedding_dim`**: Size of the embedding dimension used by the encoder and decoder. Defaults to `128`.

**`l2scale`**: Regularization term. Defaults to `1e-5`.

**`loss_factor`**: Multiplier for the reconstruction error. Defaults to `2`.

</details>

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'enforce_rounding': False,
    'epochs': 500,
    ...
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

{% hint style="info" %}
**Technical Details:** This synthesizer uses the TVAE to learn a model from real data and create synthetic data. The TVAE uses variational autoencoders (VAEs) to model data, as described in the [Modeling Tabular data using Conditional GAN](https://arxiv.org/pdf/1907.00503.pdf) paper which was presented at the NeurIPS conference in 2019.
{% endhint %}

### get\_loss\_values

After fitting, you can access the loss values computed during each epoch and batch.

**Parameters** (None)

**Output** A pandas.DataFrame object containing epoch number,  batch number and loss value.

```python
synthesizer.get_loss_values()
```

```python
Epoch     Batch    Loss 
1         1        1.7863
1         2        1.5484
1         3        1.3633
...
```

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the TVAE algorithm is designed for complete data with non-missing values, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/JtxkSK060ZygzbDLf4xl).

</details>

<details>

<summary>How many epochs should I train for?</summary>

Unfortunately, there is no one-size-fits-all solution for this question! The optimal number of epochs depends on both the complexity of your dataset and the metrics you are using to quantify success.

Our experiments suggest that increasing the number of epochs helps up until a certain inflection point. After this, there is no significant improvement. Keep in mind that increasing the epochs also increases the training time. More information is available in [this discussion](https://github.com/sdv-dev/SDV/discussions/980).

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you're previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>

<details>

<summary>How do I cite TVAE?</summary>

The TVAE model was introduced in the same paper as CTGAN.

*Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni*. **Modeling Tabular data using Conditional GAN**. NeurIPS, 2019.

```
@inproceedings{tvae,
   title={Modeling Tabular data using Conditional GAN},
   author={Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
   booktitle={Advances in Neural Information Processing Systems},
   year={2019}
}
```

</details>


# ❖ XGCSynthesizer

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **XSynthesizers Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [XSynthesizers Bundle](/sdv/explore/sdv-bundles/xsynthesizers) page.
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The XGCSynthesizer stands for *eXtraGaussianCopula.* It uses classic, statistical methods to train a model and generate synthetic data similar to the [GaussianCopulaSynthesizer](#gaussiancopulasynthesizer.load). However, it contains some additional features for higher quality modeling.

```python
from sdv.single_table import XGCSynthesizer

synthesizer = XGCSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = XGCSynthesizer(
    metadata, # required
    enforce_min_max_values=True,
    enforce_rounding=False,
    numerical_distributions={
        'amenities_fee': 'beta',
        'checkin_date': 'scipy.stats.dweibull'
    },
    default_distribution='norm'
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data. </td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`numerical_distributions`**: Set the distribution shape of any numerical columns that appear in your table. Input this as a dictionary, where the key is the name of the numerical column and the values is a numerical distribution.

```python
numerical_distributions = {
    <column name>: 'norm',
    <column name>: 'uniform', 
    ...
}
```

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>Use the default distribution for the column name.</td></tr><tr><td><code>&#x3C;dictionary></code></td><td>Apply the given distribution to each column name. The distribution name should be one of: <code>'norm'</code> <code>'beta'</code>, <code>'truncnorm'</code>, <code>'uniform'</code>, <code>'gamma'</code> or <code>'gaussian_kde'</code> </td></tr><tr><td><code>'scipy.stats'.&#x3C;distribution_name></code></td><td>Use a continuous distribution from the <a href="https://docs.scipy.org/doc/scipy/reference/stats.html#continuous-distributions">scipy library</a>. Make sure to provide the full path, including the prefix <code>scipy.stats.</code> — for example <code>'scipy.stats.dweibull'</code>  to refer to <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.dweibull.html#scipy.stats.dweibull">scipy's dweibull distribution</a>. </td></tr></tbody></table>

**`default_distribution`**: Set the distribution shape to use by default for all columns. Input this as a single string.

<table data-header-hidden><thead><tr><th width="204.0625"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'beta'</code></td><td>Model the column as a beta distribution</td></tr><tr><td><code>&#x3C;distribution_name></code></td><td>Model the column as the given distribution. The distribution name should be one of: <code>'norm'</code> <code>'beta'</code>, <code>'truncnorm'</code>, <code>'uniform'</code>, <code>'gamma'</code> or <code>'gaussian_kde'</code> </td></tr><tr><td><code>'scipy.stats'.&#x3C;distribution_name></code></td><td>Use a continuous distribution from the <a href="https://docs.scipy.org/doc/scipy/reference/stats.html#continuous-distributions">scipy library</a>. Make sure to provide the full path, including the prefix <code>scipy.stats.</code> — for example <code>'scipy.stats.dweibull'</code>  to refer to <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.dweibull.html#scipy.stats.dweibull">scipy's dweibull distribution</a>. </td></tr></tbody></table>

{% hint style="warning" %}
Setting the distribution to `'gaussian_kde'` increases the time it takes to train your synthesizer.
{% endhint %}

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'enforce_min_max_values': True,
    'enforce_rounding': False
    'default_distribution': 'beta',
    'numerical_distributions': {
        'amenities_fee': 'beta',
        'checkin_date': 'scipy.stats.dweibull'
    },
    ...
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer uses Gaussian Copulas to learn the overall distribution of the real data. This happens in two stages:

1. Learning the distribution of each individual column, also known as the marginal distribution. For example a beta distribution with α=2 and β=5. The synthesizer uses the learned distribution to normalize the values, creating normal curves with µ=0 and σ=1.&#x20;
2. Learning the covariance of each pair of normalized columns. This is stored as an *n* x *n* matrix, where *n* is the number of columns in the table.

The [Synthetic Data Vault paper](https://dai.lids.mit.edu/wp-content/uploads/2018/03/SDV.pdf) has more information about the Gaussian normalization process and the Copula estimations.
{% endhint %}

### get\_learned\_distributions

After fitting this synthesizer, you can access the marginal distributions that were learned to estimate the shape of each column.

**Parameters** None

**Output** A dictionary that maps the name of each learned column to the distribution that estimates its shape

```python
synthesizer.get_learned_distributions()
```

```
{
    'amenities_fee': {
        'distribution': 'beta',
        'learned_parameters': { 'a': 2.22, 'b': 3.17, 'loc': 0.07, 'scale': 48.5 }
    },
    'checkin_date': { 
        'distribution': 'scipy.stats.dweibull',
        'learned_parameters': {...}
    },
    ...
}
```

For more information about the distributions and their parameters, visit the[ Copulas library](https://sdv.dev/Copulas/).

{% hint style="info" %}
Learned parameters are only available for parametric distributions. For eg. you will not be able to access learned distributions for the `'gaussian_kde'` technique.

In some cases, the synthesizer may not be able to fit the exact distribution shape you requested, so you may see another distribution shape as a fallback (eg. `'truncnorm'` instead of `'beta'`).
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the Gaussian Copula algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/JtxkSK060ZygzbDLf4xl).

</details>

<details>

<summary>Why is <code>'beta'</code> the default distribution &#x26; when should I change it?</summary>

To create high quality synthetic data, the synthesizer should be able to match the shape of data for some optimal set of parameters. (The synthesizer learns and optimizes the parameters.)

We chose `'beta'` as the default distribution because it's capable of matching a variety of different shapes. It's also time efficient compared to other distributions like `'gaussian_kde'`.

**This default is not guaranteed to work on every dataset.** Consider changing the default distribution if all your columns have specific characteristics that you want to capture. If you have only a few columns that are highly important to match, then you can set those shapes specifically using the `numerical_distributions` parameter.

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you're previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>


# ❖ BootstrapSynthesizer

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **XSynthesizers Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [XSynthesizers Bundle](/sdv/explore/sdv-bundles/xsynthesizers) page.
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The BootstrapSynthesizer is a synthesizer specifically designed to work when you only have a few rows of data — or if your data is "short and wide", containing more columns than rows. This synthesizer internally bootstraps your real data, and then uses the bootstrapped data to build a model. The modeling part is compatible with any other [single-table synthesizer](/sdv/modeling/single-table-synthesizers).

```python
from sdv.single_table import BootstrapSynthesizer

synthesizer = BootstrapSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = BootstrapSynthesizer(
    metadata, # required
    num_rows_bootstrap=1000,
    bootstrap_noise_amt=1.5,
    data_synthesizer='GaussianCopulaSynthesizer',
    enforce_min_max_values=True,
    synthesize_missing_values=False
)
```

### Parameter Reference

**`num_rows_bootstrap`** : Specify the number of additional rows to bootstrap before modeling the data.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>1000</code></td><td>Bootstrap the original data by creating 1000 rows of additional data</td></tr><tr><td><code>&#x3C;integer></code></td><td>Create the desired number of bootstrapped rows before building the model</td></tr></tbody></table>

**`bootstrap_noise_amount`** : The amount of noise to add when bootstrapping the data. Some noise is necessary to provide a greater diversity of data points for modeling.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>1.5</code></td><td>When bootstrapping the data, add noise that is equal to 1.5x the standard deviation of each row.</td></tr><tr><td><code>&#x3C;float></code></td><td>Add the desired amount of noise to the bootstrapped data. This is the multiplier to the standard deviation, so 1.5 means 1.5x the standard deviation, 2 means 2x the standard deviation, etc.</td></tr></tbody></table>

**`data_synthesizer`** : The single-table synthesizer to use when modeling the bootstrapped data.

<table data-header-hidden><thead><tr><th width="267.59375"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'GaussianCopulaSynthesizer'</code></td><td>Use the <a href="/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer">GaussianCopulaSynthesizer</a> to build a model of the bootstrapped data</td></tr><tr><td><code>&#x3C;synthesizer_name></code></td><td>Supply a synthesizer name from the list of <a href="/sdv/modeling/single-table-synthesizers">single table synthesizers</a>. For example <code>'XGCSynthesizer'</code> or <code>'CTGANSynthesizer'</code>.</td></tr></tbody></table>

**`data_synthesizer_params`** : A dictionary of parameters to use for the synthesizer

<table data-header-hidden><thead><tr><th width="221.0078125"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>Use the default parameters for the synthesizer</td></tr><tr><td><code>&#x3C;dictionary></code></td><td>Update the default parameters for the synthesizer you've chosen by providing a dictionary of key/values pairs for each parameter. Refer to the docs for your synthesizer for possible parameters. For example, for <a href="/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer">GaussianCopulaSynthesizer</a> you can supply: <code>{'default_distribution': 'norm'}</code>.</td></tr></tbody></table>

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data.</td></tr></tbody></table>

**`synthesize_missing_values`**: Control whether the synthetic data should include missing values.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain missing values in roughly the same proportion as the original data</td></tr><tr><td><code>False</code></td><td>The synthetic data may should not contain any missing values for numerical and datetime columns.</td></tr></tbody></table>

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'num_rows_bootstrap': 1000,
    'bootstrap_noise_amt': 1.5,
    'data_synthesizer': 'GaussianCopulaSynthesizer',
    'enforce_min_max_bounds': True,
    'synthesize_missing_values': False
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer internally bootstraps your real data by adding noise, and then uses the bootstrapped data to build a model. The modeling part is compatible with any other [single-table synthesizer](/sdv/modeling/single-table-synthesizers).
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQ

<details>

<summary>What does it mean to bootstrap data?</summary>

Bootstrapping data means creating more examples of training data using your original rows. Bootstrapping involves duplicating the original rows, and then adding some noise to the values in order to create a larger, more varied dataset.

This is necessary because the AI-based models expect a larger number of data points for accurate learning.

</details>


# ❖ SegmentSynthesizer

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **XSynthesizers Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [XSynthesizers Bundle](/sdv/explore/sdv-bundles/xsynthesizers) page.
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The SegmentSynthesizer calculates different segments of real data, and computes a different model for each one. You can supply any single-table synthesizer for computing the per-segment model. Use this when your real data is highly segmented, containing different patterns for each.

```python
from sdv.single_table import SegmentSynthesizer

synthesizer = SegmentSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = SegmentSynthesizer(
    metadata, # required
    segmentation_params={
        'method': 'exact_values',
        'column_name': 'made_purchase'
    },
    per_segment_synthesizer='GaussianCopulaSynthesizer'
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data.</td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`segmentation_params`**: A dictionary of parameters that govern how to perform the segmentation. This allows for one of two possible methods.

<table data-header-hidden><thead><tr><th width="226"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'algorithmic'</code> segmentation</td><td><p>Allow the synthesizer to algorithmically compute segments based on the data. You can optionally provide:</p><ul><li><code>n_segments</code>: The number of segments (defaults to 3)</li><li><code>column_names</code>: A list of column names to use for the algorithmic segmentation (defaults to all columns)</li></ul><pre class="language-python"><code class="lang-python">segmentation_params={
    'method': 'algorithmic', # required
    'n_segments': 5, # defaults to 3
    'column_names': ['age', 'income'] # defaults to all
}
</code></pre></td></tr><tr><td><code>'exact_values'</code> segmentation</td><td><p>Supply a categorical column that already contains the segments. The exact values from that column are used to identify the segments. </p><pre class="language-python"><code class="lang-python">segmentation_params={
    'method': 'exact_values', # required
    'column_name': 'made_purchase' # required
}
</code></pre></td></tr></tbody></table>

**`per_segment_synthesizer`**: A string with the type of synthesizer to use for modeling each individual segment. *You can update individual segment synthesizers later using the `set_synthesizer_for_segment` method, detailed below.*

<table data-header-hidden><thead><tr><th width="325"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'GaussianCouplaSynthesizer'</code></td><td>Use the GaussianCopulaSynthesizer to model each segment.</td></tr><tr><td><code>&#x3C;synthesizer_name></code></td><td>Supply a synthesizer name from the list of <a href="/sdv/modeling/single-table-synthesizers">single table synthesizers</a>. For example <code>'XGCSynthesizer'</code> or <code>'CTGANSynthesizer'</code>.</td></tr></tbody></table>

**`per_segment_synthesizer_params`**: A dictionary of parameters to use for each of the per segment synthesizers.

<table data-header-hidden><thead><tr><th width="267"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>Use the default parameters for the synthesizer</td></tr><tr><td><code>&#x3C;dictionary></code></td><td>Update the default parameters for the synthesizer you've chosen by providing a dictionary of key/values pairs for each parameter. This is different for each synthesizer. Refer to the <a href="/sdv/modeling/single-table-synthesizers">synthesizer's API</a>.<br><br>For example, for <a href="#gaussiancopulasynthesizer.load">GaussianCopulaSynthesizer</a> you can supply: <code>{'default_distribution': 'norm'}</code>.</td></tr></tbody></table>

### set\_synthesizer\_for\_segment

Use this function to set the algorithm to use for a specific segment of the data. This is most useful if you are using the `'exact_values'` segmentation, as you already know the segments that the synthesizer will use.

**Parameters**

* (required) `segment_name`: The exact categorical value that corresponds to the segment. This should be a value that appears in the column used for segmentation.
* (required) `synthesizer_name`: A string with the type of synthesizer to use for modeling each individual segment. For example `'GaussianCopulaSynthesizer'` or `'CTGANSynthesizer'`.
* `synthesizer_params`: A dictionary of parameters to use for the synthesizer. This is different for each synthesizer. Refer to the [synthesizer's API](/sdv/modeling/single-table-synthesizers).

**Output**: None. The synthesizer corresponding to the segment is set.

```python
synthesizer.set_synthesizer_for_segment(
    segment_name=True, # everything labeled as True is one segment
    synthesizer_name='CTGANSynthesizer' # the name of any SDV single-table synthesizer
    synthesizer_params={
        'epochs': 100
    }
)
```

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'n_segements': 5,
    'per_segment_synthesizer': 'GaussianCopulaSynthesizer',
    ...
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer uses an algorithm to segment your real data into different groups. Each group may have different patterns. This synthesizer models each segment separately by calling upon other single-table synthesizers.

Since each segment is ultimately modeled separately, the overall fit time is expected to increase linearly with the number of segments.
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Most algorithms that you can use for the per-segment modeling are designed for numerical data. This synthesizer ensures that all segments are appropriately converted to numerical data before modeling using Reversible Data Transformers (RDTs).&#x20;

*Currently, it is not posisble to access and modify these transformations. Though this feature is coming soon!*

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you're previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>


# ＊ DayZSynthesizer

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

The Day Z Synthesizer produces synthetic data from scratch using  the metadata. This allows you start generating synthetic data from **day zero**: no machine learning required!

```python
from sdv.single_table import DayZSynthesizer

synthesizer = DayZSynthesizer(metadata)
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="success" %}
**For the most realistic data**, we recommend making sure that your [metadata](/sdv/integration/metadata) is accurate. The DayZ synthesizer will create values and ranges based on the specifications in your metadata. This includes information such as the min/max range of numerical columns and the possible category values in categorical columns.
{% endhint %}

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. We also recommend setting the parameters at this time.

```python
synthesizer = DayZSynthesizer(
    metadata,
    locales=['en_US', 'en_CA', 'fr_CA']
)
```

### Parameter Reference

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

### Programmatic Parameters API

For even more realism, use the API below to set some additional parameters.

<details>

<summary><strong><code>set_missing_values</code></strong></summary>

Use this method to set the proportion of missing values to generate in a column

**Parameters**

* (required) `column_name`: A string representing the name of the column. *This column cannot be a primary or foreign key.*
* (required) `missing_values_proportion`: A float representing the proportion of missing values
  * Any float between 0.0 and 1.0: Randomly create this proportion of missing values in the column
* `table_name`: A string representing the name of the table

```python
synthesizer.set_missing_values(
    column_name='room_type',
    missing_values_proportion=0.1
)
```

</details>

## Saving your synthesizer

Save your synthesizer for future use

### save

Use this function to save your synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}


# ❖ DPGCSynthesizer

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **Differential Privacy Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [Differential Privacy Bundle](/sdv/explore/sdv-bundles/differential-privacy) page.
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The DPGCSynthesizer creates synthetic data that is differential private. It is based on the classical statistical methods from the Gaussian Copula synthesizer with added privacy guarantees. DPGC stands for ***Differential Privacy for Gaussian Copula***. For more information about the algorithm, refer to the [research paper](https://openproceedings.org/2014/conf/edbt/LiXJ14.pdf).

```python
from sdv.single_table import DPGCSynthesizer

synthesizer = DPGCSynthesizer(metadata, epsilon=2.5)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in:

* A [Metadata](/sdv/integration/metadata) object as the first argument
* An **`epsilon`** value as the second argument. This is a float (>0) that represents the *privacy loss budget* you're willing to accommodate. (See the parameter reference below for more information.)

All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = DPGCSynthesizer(
  metadata,
  epsilon=2.5, # we recommend values in the 0-10 range; 0-1 is the most conservative
  known_min_max_values={
    'age': {'min': 0, 'max': 120 },
    'salary': { 'min': 0 }
  },
  enforce_rounding=True,
  locales=['en_US'],
)
```

### Parameter Reference

(required) **`epsilon`**: A float >0 that represents the privacy loss budget you are willing to accommodate.&#x20;

{% hint style="info" %}
**How should I chose my privacy loss budget (epsilon)?** The value of epsilon is a measure of how much risk you're willing to take on when it comes to privacy.

* Values in the 0-1 range indicate that you are not willing to take on too much risk. As a result, the synthetic data will have strong privacy guarantees — potentially at the expense of data quality.
* Values in the 2-10 range indicate that you're willing to accept some privacy risk in order to preserve more data quality.

Note: The smaller your epsilon value, the more data the synthesizer will require to fully enforce differential privacy. The exact size of data required also depends on the # of columns in your dataset. For reference, a dataset with 14 columns will require at least 15K rows for an epsilon of 2.5.
{% endhint %}

**`known_min_max_values`**: A dictionary that provides the already-known min/max values for any of the numerical or datetime columns. Providing these values will help to conserve the privacy budget and ultimately yield higher quality synthetic data (for the same epsilon value).

{% hint style="danger" %}
**The min/max values should represent prior knowledge of the data.** In order to enforce differential privacy, it is critical that these min/max values are prior knowledge that is *not* based on any computations of the real data.
{% endhint %}

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>There are no known min/max values. The synthesizer will use up some of your privacy loss budget to compute differentially-private min/max values.</td></tr><tr><td><code>&#x3C;dictionary></code></td><td><p>A dictionary with the known min/max values. The keys are the column names, and the value is another dictionary containing <code>'min'</code> and <code>'max'</code> keys. (You can provide one or both.)</p><p></p><p>For numerical columns, represent the min/max values as floats; for datetimes, represent them as pd.Timestamp objects.</p><pre class="language-python"><code class="lang-python">{
  'age': { 'min': 0, 'max': 120 },
  'salary': { 'min': 0 }
}
</code></pre></td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>["en_US", "fr_CA"]</code></p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'epsilon': 2.5,
    'known_min_max_values': {
        'age': { 'min': 0, 'max': 120 },
        'salary': { 'min': 0 }
    }
    'enforce_rounding': False
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer uses the Gaussian Copula methodology, but with modifications to ensure differential privacy.  For more information about the algorithm, please refer to [this research paper](https://openproceedings.org/2014/conf/edbt/LiXJ14.pdf).
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

{% hint style="success" %}
**Get the SDVerified stamp of approval.** Run the [differential privacy verification](/sdv/evaluation/privacy/empirical-differential-privacy) on your synthesizer. Verify the results before you decide to sample any synthetic data or share your synthesizer.
{% endhint %}

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the Gaussian Copula algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs).&#x20;

</details>


# ❖ DPGCFlexSynthesizer

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **Differential Privacy Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [Differential Privacy Bundle](/sdv/explore/sdv-bundles/differential-privacy) page.
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The DPGCFlexSynthesizer creates synthetic data that is differential private. It is similar to the [DPGCSynthesizer](/sdv/modeling/single-table-synthesizers/dpgcsynthesizer) but with modifications that allow you to be more flexible with the preprocessing. You can [update the transformers](/sdv/modeling/configuration/preprocessing) for most columns without violating differential privacy. For more information about the methodology, refer to the [research paper](https://openproceedings.org/2014/conf/edbt/LiXJ14.pdf).

{% hint style="warning" %}
**This is an experimental synthesizer!** Let us know if you're finding the modeling process and synthetic data creation useful.
{% endhint %}

```python
from sdv.single_table import DPGCFlexSynthesizer

synthesizer = DPGCFlexSynthesizer(metadata, epsilon=2.5)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in:

* A [Metadata](/sdv/integration/metadata) object as the first argument
* An **`epsilon`** value as the second argument. This is a float (>0) that represents the *privacy loss budget* you're willing to accommodate. (See the parameter reference below for more information.)

All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = DPGCFlexSynthesizer(
  metadata,
  epsilon=2.5, # we recommend values in the 0-10 range; 0-1 is the most conservative
  known_min_max_values={
    'age': {'min': 0, 'max': 120 },
    'salary': { 'min': 0 }
  },
  enforce_rounding=True,
  locales=['en_US'],
)
```

### Parameter Reference

(required) **`epsilon`**: A float >0 that represents the privacy loss budget you are willing to accommodate.&#x20;

{% hint style="info" %}
**How should I chose my privacy loss budget (epsilon)?** The value of epsilon is a measure of how much risk you're willing to take on when it comes to privacy.

* Values in the 0-1 range indicate that you are not willing to take on too much risk. As a result, the synthetic data will have strong privacy guarantees — potentially at the expense of data quality.
* Values in the 2-10 range indicate that you're willing to accept some privacy risk in order to preserve more data quality.

Note: The smaller your epsilon value, the more data the synthesizer will require to fully enforce differential privacy. The exact size of data required also depends on the # of columns in your dataset. For reference, a dataset with 14 columns will require at least 15K rows for an epsilon of 2.5.
{% endhint %}

**`known_min_max_values`**: A dictionary that provides the already-known min/max values for any of the numerical or datetime columns. Providing these values will help to conserve the privacy budget and ultimately yield higher quality synthetic data (for the same epsilon value).

{% hint style="danger" %}
**The min/max values should represent prior knowledge of the data.** In order to enforce differential privacy, it is critical that these min/max values are prior knowledge that is *not* based on any computations of the real data.
{% endhint %}

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>There are no known min/max values. The synthesizer will use up some of your privacy loss budget to compute differentially-private min/max values.</td></tr><tr><td><code>&#x3C;dictionary></code></td><td><p>A dictionary with the known min/max values. The keys are the column names, and the value is another dictionary containing <code>'min'</code> and <code>'max'</code> keys. (You can provide one or both.)</p><p></p><p>For numerical columns, represent the min/max values as floats; for datetimes, represent them as pd.Timestamp objects.</p><pre class="language-python"><code class="lang-python">{
  'age': { 'min': 0, 'max': 120 },
  'salary': { 'min': 0 }
}
</code></pre></td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>["en_US", "fr_CA"]</code></p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

### get\_parameters

Use this function to access all the parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'epsilon': 2.5,
    'known_min_max_values': {
        'age': { 'min': 0, 'max': 120 },
        'salary': { 'min': 0 }
    }
    'enforce_rounding': False
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details:** This synthesizer uses the Gaussian Copula methodology, but with modifications to ensure differential privacy.  For more information about the algorithm, please refer to [this research paper](https://openproceedings.org/2014/conf/edbt/LiXJ14.pdf).
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

{% hint style="success" %}
**Get the SDVerified stamp of approval.** Run the [differential privacy verification](/sdv/evaluation/privacy/empirical-differential-privacy) on your synthesizer. Verify the results before you decide to sample any synthetic data or share your synthesizer.
{% endhint %}

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the Gaussian Copula algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs).&#x20;

</details>

<details>

<summary>What is the difference between DPGCFlex and the regular DPGC Synthesizer?</summary>

Both the DPGCFlex and the regular DPGC Synthesizer create synthetic data with differential privacy. The difference is that the DPGCFlex Synthesizer is more *flexible* in terms of the data pre-processing you can apply. The Flex Synthesizer will allow you to update any of the transformers for `numerical`, `categorical` and `datetime` data while still maintaining differential privacy. The Flex Synthesizer achieves this by adding differentially private noise immediately to your real data — that way you can apply preprocessing without violating differential privacy guarantees.

However, you may notice that the added flexibility comes at the price of data quality. The DPGCFlex Synthesizer is experimental. Please try it out and let us know if it's useful!

</details>


# CopulaGANSynthesizer

The Copula GAN Synthesizer uses a mix classic, statistical methods and GAN-based deep learning methods to train a model and generate synthetic data.

{% hint style="warning" %}
**This is an experimental synthesizer!** [Let us know](https://forum.datacebo.com/) if you're finding the modeling process and synthetic data creation useful.
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

```python
from sdv.single_table import CopulaGANSynthesizer

synthesizer = CopulaGANSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_rows=10)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = CopulaGANSynthesizer(
    metadata, # required
    enforce_min_max_values=True,
    enforce_rounding=False,
    numerical_distributions={
        'amenities_fee': 'beta',
        'checkin_date': 'uniform'
    },
    epochs=500,
    verbose=True
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data. </td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`numerical_distributions`**: Set the distribution shape of any numerical columns that appear in your table. Input this as a dictionary, where the key is the name of the numerical column and the values is a numerical distribution.

```python
numerical_distributions = {
    <column name>: 'norm',
    <column name>: 'uniform', 
    ...
}
```

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>None</code></td><td>Use the default distribution for the column name.</td></tr><tr><td><code>&#x3C;dictionary></code></td><td>Apply the given distribution to each column name. The distribution name should be one of: <code>'norm'</code> <code>'beta'</code>, <code>'truncnorm'</code>, <code>'uniform'</code>, <code>'gamma'</code> or <code>'gaussian_kde'</code> </td></tr></tbody></table>

**`default_distribution`**: Set the distribution shape to use by default for all columns. Input this as a single string.

<table data-header-hidden><thead><tr><th width="204.0625"></th><th></th></tr></thead><tbody><tr><td>(default) <code>'beta'</code></td><td>Model the column as a beta distribution</td></tr><tr><td><code>&#x3C;distribution_name></code></td><td>Model the column as the given distribution. The distribution name should be one of: <code>'norm'</code> <code>'beta'</code>, <code>'truncnorm'</code>, <code>'uniform'</code>, <code>'gamma'</code> or <code>'gaussian_kde'</code> </td></tr></tbody></table>

{% hint style="warning" %}
Setting the distribution to `'gaussian_kde'` increases the time it takes to train your synthesizer.
{% endhint %}

**`epochs`**: Number of times to train the GAN. Each new epoch can improve the model.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>300</code></td><td>Run all the data through the Generator and Discriminator 300 times during training</td></tr><tr><td><code>&#x3C;number></code></td><td>Train for a different number of epochs. Note that larger numbers will increase the modeling time.</td></tr></tbody></table>

**`verbose`**: Control whether to print out the results of each epoch. You can use this to track the training time as well as the improvements per epoch.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>False</code></td><td>Do not print out any results</td></tr><tr><td><code>True</code></td><td>Print out the Generator and Discriminator loss values per epoch. The loss values indicate how well the GAN is currently performing, lower values indicating higher quality.</td></tr></tbody></table>

**`enable_gpu`**: Whether to enable GPU usage when training the synthesizer. This may speed up the modeling time.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>If available, use the GPU to speed up modeling time. Currently, this will look for either <a href="https://developer.nvidia.com/how-to-cuda-python">CUDA</a> (on Linux/Windows machines) or <a href="https://developer.apple.com/metal/pytorch/">MPS</a> (on Mac).  If this is not available, then the GPU will not be used.  </td></tr><tr><td><code>False</code></td><td>Do not use the GPU to speed up modeling time.</td></tr></tbody></table>

{% hint style="info" %}
**Is my synthesizer using the GPU?** After calling `fit`, you may notice that the GPU is not immediately used. This is because GPU is not used for the initial, data preprocessing step. After data preprocessing is complete, you should then see the GPU being used for the neural network training. For more information about data preprocessing, see the [this guide](broken://pages/yKD7rwpmUt1JJLCbDLqE).
{% endhint %}

*(deprecated) `cuda`: Please use the `enable_gpu` option to use CUDA, if it's available on your platform.*

{% hint style="info" %}
**Looking for more customizations?** Other settings are available to fine-tune the architecture of the underlying GAN used to model the data. Click the section below to expand.
{% endhint %}

<details>

<summary>Click to expand additional GAN customization options</summary>

These settings are specific to the GAN. Use these settings if you want to optimize the technical architecture and modeling.

**`batch_size`**: Number of data samples to process in each step. This value must be even, and it must be divisible by the `pac` parameter (see below). Defaults to `500`.&#x20;

**`discriminator_dim`**: Size of the output samples for each one of the Discriminator Layers. A Linear Layer will be created for each one of the values provided. Defaults to `(256, 256)`.

**`discriminator_decay`**: Discriminator weight decay for the Adam Optimizer. Defaults to `1e-6`.

**`discriminator_lr`**: Learning rate for the discriminator. Defaults to `2e-4`.

**`discriminator_steps`**: Number of discriminator updates to do for each generator update. Default `1` to match the original CTGAN implementation

**`embedding_dim`**: Size of the random sample passed to the Generator. (Default `128`)

**`generator_decay`**: Generator weight decay for the Adam Optimizer. Defaults to `1e-6`

**`generator_dim`**: Size of the output samples for each one of the Residuals. A Residual Layer will be created for each one of the values provided. Defaults to `(256, 256)`.

**`generator_lr`**: Learning rate for the generator. Defaults to `2e-4`.

**`log_frequency`**: Whether to use log frequency of categorical levels in conditional sampling. Defaults to `True`.

**`pac`**: Number of samples to group together when applying the discriminator. Defaults to 10.

</details>

### get\_parameters

Use this function to access the all parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'enforce_min_max_values': True
    'enforce_rounding': False,
    'epochs': 500,
    'verbose': True,
    'numerical_distributions': {
        'amenities_fee': 'beta',
        'checkin_date': 'uniform'
    },
    ...
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

{% hint style="info" %}
**Technical Details:** This synthesizer learns the marginal distributions of the real data columns and normalizes them. Then, it uses CTGAN to learn the normalized data. This takes place in two stages, as shown below.

1. **Statistical Learning**: The synthesizer learns the distribution (shape) of each individual column, also known as the 1D or marginal distribution. For example a beta distribution with α=2 and β=5. The synthesizer uses the learned distribution to normalize the values, creating normal curves with µ=0 and σ=1. The [Synthetic Data Vault paper](https://dai.lids.mit.edu/wp-content/uploads/2018/03/SDV.pdf) has more information about the Gaussian normalization process.
2. **GAN-based Learning**: This synthesizer uses CTGAN to train the normalized data. The CTGAN uses generative adversarial networks (GANs) to model data, as described in the [Modeling Tabular data using Conditional GAN](https://arxiv.org/pdf/1907.00503.pdf) paper which was presented at the NeurIPS conference in 2019.
   {% endhint %}

### get\_learned\_distributions

After fitting this synthesizer, you can access the marginal distributions that were learned to estimate the shape of each column.

**Parameters** None

**Output** A dictionary that maps the name of each learned column to the distribution that estimates its shape

```python
synthesizer.get_learned_distributions()
```

```
{
    'amenities_fee': {
        'distribution': 'beta',
        'learned_parameters': { 'a': 2.22, 'b': 3.17, 'loc': 0.07, 'scale': 48.5 }
    },
    'checkin_date': { 
        ...
    },
    ...
}
```

For more information about the distributions and their parameters, visit the[ Copulas library](https://sdv.dev/Copulas/).

{% hint style="info" %}
Learned parameters are only available for parametric distributions. For eg. you will not be able to access learned distributions for the `'gaussian_kde'` technique.

In some cases, the synthesizer may not be able to fit the exact distribution shape you requested, so you may see another distribution shape (eg. `'truncnorm'` instead of `'beta'`).
{% endhint %}

### get\_loss\_values

After fitting, you can access the loss values computed during each epoch for both the numerator and denominator.

**Parameters** (None)

**Output** A pandas.DataFrame object containing epoch number, generator loss value and discriminator loss value.

```python
synthesizer.get_loss_values()
```

```python
Epoch  Generator Loss  Discriminator Loss
1      1.7863          -0.3639
2      1.5484          0.2260
3      1.3633          -0.0441
...
```

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_rows=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and update the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the Gaussian Copula algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/JtxkSK060ZygzbDLf4xl).

</details>

<details>

<summary>Why is <code>'beta'</code> the default distribution &#x26; when should I change it?</summary>

To create high quality synthetic data, the distribution should be able to match the shape of data for some optimal set of parameters. (The synthesizer learns and optimizes the parameters.)

We chose `'beta'` as the default distribution because it can take different characteristics based on the parameters, which means it's capable of matching a variety of different shapes. It's also time efficient compared to other distributions like `'gaussian_kde'`.

**This default is not guaranteed to work on every dataset.** Consider changing the default distribution if all your columns have specific characteristics that you want to capture. If you have only a few columns that are highly important to match, then you can set those shapes specifically using the `numerical_distributions` parameter.

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you're previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>

<details>

<summary>How do I cite CopulaGAN?</summary>

The CopulaGAN is a hybrid of the Gaussian Copula and the CTGAN algorithms. The [Gaussian Copula](https://en.wikipedia.org/wiki/Copula_\(probability_theory\)#:~:text=The%20Gaussian%20copula%20is%20a,using%20the%20probability%20integral%20transform.) is a well-known statistical approach. The CTGAN is a research project that you can cite using the following text:

*Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni*. **Modeling Tabular data using Conditional GAN**. NeurIPS, 2019.

```
@inproceedings{ctgan,
   title={Modeling Tabular data using Conditional GAN},
   author={Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
   booktitle={Advances in Neural Information Processing Systems},
   year={2019}
}
```

</details>


# Multi Table Synthesizers

The SDV offers a variety of synthesizers, which use different algorithms to generate synthetic data. This section describes multi-table synthesizers, which are able to model interconnected, [multi-table datasets](/sdv/integration/overview/data-modalities#multi-table-data).

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>＊</strong><a href="/sdv/modeling/multi-table-synthesizers/dayzsynthesizer"><strong>Day Z Synthesizer</strong></a></td><td>Generate synthetic data from scratch using only the metadata. This synthesizer produces multi table data with the correct formatting and referential integrity. </td><td><a href="/sdv/modeling/multi-table-synthesizers/dayzsynthesizer">＊ DayZSynthesizer</a></td></tr><tr><td>＊ <a href="/sdv/modeling/multi-table-synthesizers/independentsynthesizer"><strong>Independent Synthesizer</strong></a></td><td>Use various ML algorithms to model each table independently. This synthesizer offers the fastest performance for unlimited tables.</td><td><a href="/sdv/modeling/multi-table-synthesizers/independentsynthesizer">＊ IndependentSynthesizer</a></td></tr><tr><td><a href="/sdv/modeling/multi-table-synthesizers/hmasynthesizer"><strong>HMA Synthesizer</strong></a></td><td>Use a hierarchical ML algorithm to learn from real data and generate synthetic data. Produce data for limited columns in up to 5 tables with a depth of 1.</td><td><a href="/sdv/modeling/multi-table-synthesizers/hmasynthesizer">HMASynthesizer</a></td></tr><tr><td><strong>＊</strong><a href="/sdv/modeling/multi-table-synthesizers/hsasynthesizer"><strong>HSA Synthesizer</strong></a></td><td>Use a segment-based ML algorithm to learn from real data and generate synthetic data. This synthesizer offers fast performance for unlimited tables.</td><td><a href="/sdv/modeling/multi-table-synthesizers/hsasynthesizer">＊ HSASynthesizer</a></td></tr></tbody></table>

## Which synthesizer should I use?

Use the table below to determine the right synthesizer for your needs.

<table data-full-width="true"><thead><tr><th width="283">Feature</th><th width="121" align="center">＊ Day Z</th><th width="164" align="center">＊ Independent</th><th align="center">HMA</th><th align="center">＊ HSA</th></tr></thead><tbody><tr><td>Synthesize with only metadata</td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"></td><td align="center"></td><td align="center"></td></tr><tr><td>Add constraints</td><td align="center"></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td>Anonymize columns</td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td>Maintain <a data-footnote-ref href="#user-content-fn-1">referential integrity</a></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td>Learn <a data-footnote-ref href="#user-content-fn-2">intra-table patterns</a></td><td align="center"></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td>Learn <a data-footnote-ref href="#user-content-fn-3">inter-table patterns</a></td><td align="center"></td><td align="center"></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td>Learn <a data-footnote-ref href="#user-content-fn-4">cardinality</a></td><td align="center"></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td><a data-footnote-ref href="#user-content-fn-5">Scalability</a></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td><td align="center"></td><td align="center"><span data-gb-custom-inline data-tag="emoji" data-code="2714">✔️</span></td></tr><tr><td>Availability</td><td align="center">Enterprise</td><td align="center">Enterprise</td><td align="center">Public</td><td align="center">Enterprise</td></tr></tbody></table>

*＊SDV Enterprise Features.* *These features are only available for licensed, enterprise users. To learn more about the SDV Enterprise features and purchasing a license,* [*visit our website*](https://datacebo.com/pricing/)*.*

[^1]: With referential integrity, all references between tables are present. That is, there are no dangling (lost) references.

[^2]: Patterns inside of a single table, which include min/max boundaries, column shapes (marginals) and trends (correlations)

[^3]: Trends between multiple tables include correlations.

[^4]: Cardinality refers the number of children each parent has. You may have a strict cardinality requirement, such as an exact 1-1 mapping.

[^5]: Easily scale to more tables without significantly sacrificing performance or adding computational requirements


# ＊ DayZSynthesizer

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

The Day Z Synthesizer produces synthetic data from scratch using the metadata. This allows you start generating synthetic data from **day zero**: no machine learning required!

```python
from sdv.multi_table import DayZSynthesizer

synthesizer = DayZSynthesizer(metadata)
synthetic_data = synthesizer.sample_scale(scale=1.0)
```

{% hint style="success" %}
**For the most realistic data**, we recommend making sure that your [metadata](/sdv/integration/metadata) is accurate. The DayZ synthesizer will create values and ranges based on the specifications in your metadata. This includes information such as the min/max range of numerical columns and the possible category values in categorical columns.
{% endhint %}

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. We also recommend setting the parameters at this time.

```python
synthesizer = DayZSynthesizer(
    metadata,
    locales=['en_US', 'en_CA', 'fr_CA']
)
```

### Parameter Reference

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

### Programmatic Parameters API <a href="#programmatic-parameters-api" id="programmatic-parameters-api"></a>

For even more realism, use the API below to set some additional parameters.

<details>

<summary><strong><code>set_missing_values</code></strong></summary>

Use this method to set the proportion of missing values to generate in a column

**Parameters**

* (required) `table_name`: A string representing the name of the table
* (required) `column_name`: A string representing the name of the column.  *This column cannot be a primary or foreign key.*
* (required) `missing_values_proportion`: A float representing the proportion of missing values
  * Any float between 0.0 and 1.0: Randomly create this proportion of missing values in the column

```python
synthesizer.set_missing_values(
    table_name='guests',
    column_name='room_type',
    missing_values_proportion=0.1
)
```

**Output** (None) Sets the proportion of the missing values

</details>

<details>

<summary><strong><code>set_cardinality</code></strong></summary>

Use this function to set the cardinality of a parent/child relationship. The *cardinality* refers to the number of children that each parent row is allowed to have. This can be anywhere from 0 to infinity.

This function can help you create realistic data for many relationship types such as 1-1, 1-to-many, etc.

```python
# each hotel must have 1 or more guests
synthesizer.set_cardinality(
    parent_table_name='hotels',
    child_table_name='guests',
    parent_primary_key='hotel_id',
    child_foreign_key='hotel_id',
    min_cardinality=1,
    max_cardinality=None
)
```

**Parameters**

* (required) `parent_table_name`: The name of the parent table
* (required) `child_table_name`: The name of the child table
* (required) `parent_primary_key`: The name of the primary key in the parent
* (required) `child_foreign_key`: The name of the foreign key in the child that refers to the primary key of the parent
* `min_cardinality`: The minimum # of children each parent must have, must be an integer >=0
  * (default) `0`: A parent row must have 0 or more children
  * `<integer>`: An integer representing the minimum # of children
* `max_cardinality`: The maximum # of children each parent must have, must be an integer >0
  * (default) `None`: Do not enforce a maximum (i.e. the maximum # of children can be infinite)
  * `<integer>`: An integer > `min_cardinality` representing the maximum # of children
  * *Note that If min cardinality = max cardinality, then that means there is a fixed # of children for each parent.*

**Output** (None) Sets the min and max cardinality of the parent/child relationship, or updates it if the cardinality was already set.&#x20;

</details>

<details>

<summary><strong><code>set_table_sizes</code></strong></summary>

Use this function to set the (relative) table sizes of each table in the dataset. When sampling, you can scale the entire dataset up or down.

```python
synthesizer = DayZSynthesizer(metadata)
synthesizer.set_table_sizes(
    num_rows_per_table={
        'hotels': 1000,
        'guests': 2500
    }
)
```

**Parameters:**&#x20;

* (required) `num_rows_per_table`: A dictionary containing the number of rows to set per table. The keys are the names of the tables, and the values are integers representing the number of rows.

</details>

### get\_parameters

Use this function to access the all parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters**

* `output_filepath`: A string representing the name of the file to write the parameters to. We recommend storing this as a JSON file. Defaults to `None`, meaning that no output filepath is written.

**Output** A dictionary with the table names and parameters for each table.

{% hint style="info" %}
These parameters are only for the multi-table synthesizer. To get individual table-level parameters, use the `get_table_parameters` function.

The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

```python
synthesizer.get_parameters()
```

```python
{
    'locales': ['en_US', 'fr_CA'],
    ...
}
```

### get\_table\_parameters

Use this function to access the all parameters a table synthesizer uses -- those you have provided as well as the default ones.

**Parameters**

* (required) `table_name`: A string describing the name of the table

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_table_parameters(table_name='users')
```

```python
{
    'synthesizer_name': 'DayZSynthesizer',
    'synthesizer_parameters': {
        ...
    }
}
```

## Saving your synthesizer

Save your synthesizer for future use

### save

Use this function to save your synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`&#x20;

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next? <a href="#whats-next" id="whats-next"></a>

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details. *(This synthesizer does not yet offer support for conditional sampling.)*

```python
synthetic_data = synthesizers.sample_scale(scale=1.0)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and customize the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}


# ＊ IndependentSynthesizer

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The Independent Synthesizer learns each table's patterns independently. This synthesizer offers fast performance for unlimited tables.

```python
from sdv.multi_table import IndependentSynthesizer

synthesizer = IndependentSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample_scale(scale=1.0)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument.

```python
synthesizer = IndependentSynthesizer(metadata)
```

All other parameters are optional. You can include them to customize the synthesizer.

### Parameter Reference

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

```python
synthesizer = IndependentSynthesizer(
    metadata,
    locales=['en_US', 'en_CA', 'fr_CA']
)
```

### set\_table\_parameters

The Independent Synthesizer models each individual table. You can get and set the parameters for each table.

**Parameters**

* (required) `table_name`: A string describing the name of the table
* `table_synthesizer`: The single table synthesizer to use for modeling the table
  * (default) `'GaussianCopulaSynthesizer'`: Use the [GaussianCopulaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) to model the single table
  * Other available options: `'GaussianCopulaSynthesizer'`, `'CTGANSynthesizer'`, `'TVAESynthesizer'`, `'CopulaGANSynthesizer'`. For more information, see [Single Table Synthesizers](/sdv/modeling/single-table-synthesizers).
* `table_parameters`: A dictionary mapping the name of the parameter (string) to the value of the parameter (various). These parameters are different for each synthesizer. For more information, see [Single Table Synthesizers](/sdv/modeling/single-table-synthesizers).

**Output** (None)

```python
synthesizer.set_table_parameters(
    table_name='guests',
    table_synthesizer='GaussianCopulaSynthesizer',
    table_parameters={
        'enforce_min_max_values': True,
        'default_distribution': 'truncnorm',
        'numerical_distributions': { 
            'checkin_date': 'uniform',
            'amenities_fee': 'beta' 
        }
    }
)
```

### get\_parameters

Use this function to access the all parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** (None)

**Output** A dictionary with the table names and parameters for each table.

{% hint style="info" %}
These parameters are only for the multi-table synthesizer. To get individual table-level parameters, use the `get_table_parameters` function.

The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

```python
synthesizer.get_parameters()
```

```python
{
    'locales': ['en_US', 'fr_CA'],
    ...
}
```

### get\_table\_parameters

Use this function to access the all parameters a table synthesizer uses -- those you have provided as well as the default ones.

**Parameters**

* (required) `table_name`: A string describing the name of the table

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_table_parameters(table_name='users')
```

```python
{
    'synthesizer_name': 'GaussianCopulaSynthesizer',
    'synthesizer_parameters': {
        'default_distribution': 'beta',
        ...
    }
}
```

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping each table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

{% hint style="info" %}
**Technical Details:** The Independent Synthesizer models each table independently, as well as the cardinality between the tables (i.e. the number of children that each parent row has). It can use any [single table synthesizer](/sdv/modeling/single-table-synthesizers) to model the individual tables.
{% endhint %}

### get\_learned\_distributions

After fitting this synthesizer, you can access the marginal distributions that were learned to estimate the shape of each column.

**Parameters**

* (required) `table_name`: A string with the name of the table

**Output** A dictionary that maps the name of each learned column to the distribution that estimates its shape

```python
synthesizer.get_learned_distributions(table_name='guests')
```

```python
{
    'amenities_fee': {
        'distribution': 'beta',
        'learned_parameters': { 'a': 2.22, 'b': 3.17, 'loc': 0.07, 'scale': 48.5 }
    },
    'checkin_date': { 
        ...
    },
    ...
}
```

For more information about the distributions and their parameters, visit the[ Copulas library](https://sdv.dev/Copulas/).

{% hint style="info" %}
Learned parameters are only available for parametric models and distributions. For eg. you will not be able to access learned distributions for GAN-based synthesizers (such as CTGAN) or the `'gaussian_kde'` technique.

In some cases, the synthesizer may not be able to fit the exact distribution shape you requested, so you may see another distribution shape (eg. `'truncnorm'` instead of `'beta'`).
{% endhint %}

### get\_loss\_values

After fitting, you can access the loss values computed during each epoch for both the numerator and denominator.

**Parameters**

* (required) `table_name`: A string with the name of the table

**Output** A pandas.DataFrame object containing epoch number, generator loss value and discriminator loss value.

```python
synthesizer.get_loss_values(table_name='users')
```

```python
Epoch  Generator Loss  Discriminator Loss
1      1.7863          -0.3639
2      1.5484          0.2260
3      1.3633          -0.0441
...
```

{% hint style="info" %}
Loss values are only available for tables that use neural network-based models. such as CTGAN, TVAE or CopulaGAN.
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details. *(This synthesizer does not yet offer support for conditional sampling.)*

```python
synthetic_data = synthesizers.sample_scale(scale=1.0)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and customize the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if the columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Some of the underlying machine learning algorithms are designed for only numerical data. This synthesizer processes the data using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/TuZCZml0pBWlKM6dOapf).

</details>


# HMASynthesizer

The HMA Synthesizer uses hierarchical ML algorithm to learn from real data and generate synthetic data.  The algorithm uses classical statistics.

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

```python
from sdv.multi_table import HMASynthesizer

synthesizer = HMASynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample_scale(scale=1.0)
```

{% hint style="warning" %}
**Is the HMASynthesizer suited for your dataset?** The HMASynthesizer is designed to capture correlations between different tables with high quality. The algorithm is optimized for smaller datasets with around 5 tables and 1 level of depth (eg. a parent and its child table).

**Want to model more complex graphs?** Upgrade to [SDV Enterprise](/sdv/explore/sdv-enterprise) for access to [HSASynthesizer](/sdv/modeling/multi-table-synthesizers/hsasynthesizer), which contains an improved algorithm. HSASynthesizer scales to more tables and improves performance by orders of magnitude. To get started, [click here](https://datacebo.com/pricing/).&#x20;
{% endhint %}

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument.

```python
synthesizer = HMASynthesizer(metadata)
```

All other parameters are optional. You can include them to customize the synthesizer.

### Parameter Reference

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

```python
synthesizer = HMASynthesizer(
    metadata,
    locales=['en_US', 'en_CA', 'fr_CA']
)
```

`verbose`: A boolean describing whether or not to show the progress when fitting the synthesizer.

<table data-header-hidden><thead><tr><th width="187"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>Show the progress when fitting the synthesizer. You'll see printed progress bars during every stage of the fitting process: Preprocessing, learning relationships and modeling tables.</td></tr><tr><td><code>False</code></td><td>Do not show progress. The synthesizer will fit the data silently.</td></tr></tbody></table>

### set\_table\_parameters

The HMA Synthesizer is a multi-table algorithm that models each individual table as well as the connections between them. You can get and set the parameters for each individual table.

**Parameters**

* (required) `table_name`: A string describing the name of the table
* `table_parameters`: A dictionary mapping the name of the parameter (string) to the value of the parameter (various). See [GaussianCouplaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer#parameter-reference) for more details.

**Output** (None)

```python
synthesizer.set_table_parameters(
    table_name='guests',
    table_parameters={
        'enforce_min_max_values': True,
        'default_distribution': 'truncnorm',
        'numerical_distributions': { 
            'checkin_date': 'uniform',
            'amenities_fee': 'beta' }
    }
)
```

{% hint style="warning" %}
**Which distributions can I use with the HMA?** Please note that the HMA algorithm is only compatible with parametric distributions that have a predefined number of parameters. You will not be able to use the `'gaussian_kde'` distribution with HMA.
{% endhint %}

### get\_parameters

Use this function to access the all parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** (None)

**Output** A dictionary with the table names and parameters for each table.

{% hint style="info" %}
These parameters are only for the multi-table synthesizer. To get individual table-level parameters, use the `get_table_parameters` function.

The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

```python
synthesizer.get_parameters()
```

```python
{
    'locales': ['en_US', 'fr_CA'],
    ...
}
```

### get\_table\_parameters

Use this function to access the all parameters a table synthesizer uses -- those you have provided as well as the default ones.

**Parameters**

* (required) `table_name`: A string describing the name of the table

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_table_parameters(table_name='users')
```

```python
{
    'synthesizer_name': 'GaussianCopulaSynthesizer',
    'synthesizer_parameters': {
        'default_distribution': 'beta',
        ...
    }
}
```

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping each table name to a pandas.DataFrame containing the real data that the machine learning model will learn from

**Output** (None)

{% hint style="info" %}
**Technical Details:** HMA, which stands for *Hierarchical Modeling Algorithm*, uses a recursive technique to model the parent-child relationships of a multi-table datasets. At a base level, it uses Gaussian Copulas to model individual tables.&#x20;

See:

* [GaussianCopulaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) for more information on the GaussianCopula framework
* The [Synthetic Data vault paper](https://dai.lids.mit.edu/wp-content/uploads/2018/03/SDV.pdf), published in the International Conference on Data Science and Advance Analytics, October 2016
  {% endhint %}

### get\_learned\_distributions

After fitting this synthesizer, you can access the marginal distributions that were learned to estimate the shape of each column.

**Parameters**

* (required) `table_name`: A string with the name of the table

**Output** A dictionary that maps the name of each learned column to the distribution that estimates its shape

```python
synthesizer.get_learned_distributions(table_name='guests')
```

```
{
    'amenities_fee': {
        'distribution': 'beta',
        'learned_parameters': { 'a': 2.22, 'b': 3.17, 'loc': 0.07, 'scale': 48.5 }
    },
    'checkin_date': { 
        ...
    },
    ...
}
```

For more information about the distributions and their parameters, visit the[ Copulas library](https://sdv.dev/Copulas/).

{% hint style="info" %}
Learned parameters are only available for parametric distributions. For eg. you will not be able to access learned distributions for the `'gaussian_kde'` technique.

In some cases, the synthesizer may not be able to fit the exact distribution shape you requested, so you may see another distribution shape (eg. `'truncnorm'` instead of `'beta'`).
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details. *(This synthesizer does not yet offer support for conditional sampling.)*

```python
synthetic_data = synthesizers.sample_scale(scale=1.0)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and customize the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>How do I cite the HMA?</summary>

*Neha Patki, Roy Wedge, Kalyan Veeramachaneni.* **The Synthetic data vault.** DSAA, 2016.

```
@inproceedings{
    HMA,
    title={The Synthetic data vault},
    author={Patki, Neha and Wedge, Roy and Veeramachaneni, Kalyan},
    booktitle={IEEE International Conference on Data Science and Advanced Analytics (DSAA)},
    year={2016},
    pages={399-410},
    doi={10.1109/DSAA.2016.49},
    month={Oct}
}
```

</details>

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the HMA algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/TuZCZml0pBWlKM6dOapf).

</details>


# ＊ HSASynthesizer

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

The HSA Synthesizer uses a segment-based algorithm to learn from your real data and generate synthetic data. This synthesizer offers fast performance for unlimited tables.

```python
from sdv.multi_table import HSASynthesizer

synthesizer = HSASynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample_scale(scale=1.0)
```

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument.

```python
synthesizer = HSASynthesizer(metadata)
```

All other parameters are optional. You can include them to customize the synthesizer.

### Parameter Reference

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

```python
synthesizer = HSASynthesizer(
    metadata,
    locales=['en_US', 'en_CA', 'fr_CA']
)
```

**`default_num_clusters`**: The number of clusters to segment each table into

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) 3</td><td>Split each table into 3 clusters for the purposes of capturing correlations between parent and child tables</td></tr><tr><td><code>&#x3C;integer></code></td><td>Split each table into the desired number of clusters. A smaller number of clusters makes the fit and sampling more efficient, and generally noises the data. A large number of clusters allows the model to learn more specific correlations across parent/child tables.</td></tr></tbody></table>

### set\_table\_parameters

The HSA Synthesizer is a multi-table algorithm that models each individual table as well as the connections between them. You can get and set the parameters for each individual table.

**Parameters**

* (required) `table_name`: A string describing the name of the table
* `table_synthesizer`: The single table synthesizer to use for modeling the table
  * (default) `'GaussianCopulaSynthesizer'`: Use the [GaussianCopulaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) to model the single table
  * Other available options: `'GaussianCopulaSynthesizer'`, `'CTGANSynthesizer'`, `'TVAESynthesizer'`, `'CopulaGANSynthesizer'`. For more information, see [Single Table Synthesizers](/sdv/modeling/single-table-synthesizers).
* `table_parameters`: A dictionary mapping the name of the parameter (string) to the value of the parameter (various). These parameters are different for each synthesizer. For more information, see [Single Table Synthesizers](/sdv/modeling/single-table-synthesizers).

**Output** (None)

```python
synthesizer.set_table_parameters(
    table_name='guests',
    table_synthesizer='GaussianCopulaSynthesizer',
    table_parameters={
        'enforce_min_max_values': True,
        'default_distribution': 'truncnorm',
        'numerical_distributions': { 
            'checkin_date': 'uniform',
            'amenities_fee': 'beta' 
        }
    }
)
```

### get\_parameters

Use this function to access the all parameters your synthesizer uses -- those you have provided as well as the default ones.

**Parameters** (None)

**Output** A dictionary with the table names and parameters for each table.

{% hint style="info" %}
These parameters are only for the multi-table synthesizer. To get individual table-level parameters, use the `get_table_parameters` function.

The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

```python
synthesizer.get_parameters()
```

```python
{
    'locales': ['en_US', 'fr_CA'],
    ...
}
```

### get\_table\_parameters

Use this function to access the all parameters a table synthesizer uses -- those you have provided as well as the default ones.

**Parameters**

* (required) `table_name`: A string describing the name of the table

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_table_parameters(table_name='users')
```

```python
{
    'synthesizer_name': 'GaussianCopulaSynthesizer',
    'synthesizer_parameters': {
        'default_distribution': 'beta',
        ...
    }
}
```

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping each table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

{% hint style="info" %}
**Technical Details:** HSA, which stands for *Hierarchical Segmentation Algorithm*, uses a segment-based approach to model the parent-child relationships of a multi-table datasets. At a base level, it can model individual tables using any [single table synthesizer](/sdv/modeling/single-table-synthesizers).&#x20;
{% endhint %}

### get\_learned\_distributions

After fitting this synthesizer, you can access the marginal distributions that were learned to estimate the shape of each column.

**Parameters**

* (required) `table_name`: A string with the name of the table

**Output** A dictionary that maps the name of each learned column to the distribution that estimates its shape

```python
synthesizer.get_learned_distributions(table_name='guests')
```

```python
{
    'amenities_fee': {
        'distribution': 'beta',
        'learned_parameters': { 'a': 2.22, 'b': 3.17, 'loc': 0.07, 'scale': 48.5 }
    },
    'checkin_date': { 
        ...
    },
    ...
}
```

For more information about the distributions and their parameters, visit the[ Copulas library](https://sdv.dev/Copulas/).

{% hint style="info" %}
Learned parameters are only available for parametric models and distributions. For eg. you will not be able to access learned distributions for GAN-based synthesizers (such as CTGAN) or the `'gaussian_kde'` technique.

In some cases, the synthesizer may not be able to fit the exact distribution shape you requested, so you may see another distribution shape (eg. `'truncnorm'` instead of `'beta'`).
{% endhint %}

### get\_loss\_values

After fitting, you can access the loss values computed during each epoch for both the numerator and denominator.

**Parameters**

* (required) `table_name`: A string with the name of the table

**Output** A pandas.DataFrame object containing epoch number, generator loss value and discriminator loss value.

```python
synthesizer.get_loss_values(table_name='users')
```

```python
Epoch  Generator Loss  Discriminator Loss
1      1.7863          -0.3639
2      1.5484          0.2260
3      1.3633          -0.0441
...
```

{% hint style="info" %}
Loss values are only available for tables that use neural network-based models. such as CTGAN, TVAE or CopulaGAN.
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details. *(This synthesizer does not yet offer support for conditional sampling.)*

```python
synthetic_data = synthesizers.sample_scale(scale=1.0)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and customize the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>What happens if the columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the HSA algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/TuZCZml0pBWlKM6dOapf).

</details>


# ＊ Performance Estimates

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

How well will SDV synthesizers be able to model your full data schema? Use this feature to get some estimates with only your metadata.

### ＊ create\_and\_test\_multi\_table

Simulate the performance of different multi-table synthesizers using your metadata.

This function uses the multi-table [DayZSynthesizer](/sdv/modeling/multi-table-synthesizers/dayzsynthesizer) to create random data. Then it runs the random data through the different multi-table synthesizers to estimate their performance, as well as the different evaluation reports.

```python
from sdv.utils.multi_table import create_and_test_multi_table

create_and_test_multi_table(
  metadata=my_metadata,
  synthesizers=['HMASynthesizer', 'HSASynthesizer'],
  output_folder='my_performance_results/',
  default_num_rows=1_000_000,
  timeout=3600 # 1 hour per synthesizer
)
```

**Parameters:**

* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object
* (required) `synthesizers`: A list of strings representing the multi-table synthesizers that you want to test. Options are: `'HMASynthesizer'`, `'HSASynthesizer'` or `'IndependentSynthesizer'`
* (required) `output_folder`: A destination folder where the random data, results, and other artifacts will be saved
* `default_num_rows`: An integer with the number of rows to create by default for all tables
  * (default) 1000: Create 1000 rows for every table
* `num_rows_per_table`: A dictionary that maps each table name to the number of rows to create for only that table. Values here will override the default num rows set in the previous parameter
  * (default) `None`: Do not override the default number of rows for any individual table
* `timeout`: The maximum number of seconds to give to each synthesizer to train and sample the dataset
  * (default) `None`: Do not set a maximum. Allow the synthesizer to take as long as it needs.
  * `<integer>`: Allow a synthesizer to run on the integer number of seconds for each dataset. If the synthesizer is exceeding the time, the output will include a TimeoutError.

**Output** A pandas DataFrame with detailed performance results from each synthesizer

#### Interpreting the results

Your results include detailed timings for training, sampling, and evaluations.

```
synthesizer        init_time    preprocess_time    fit_processed_time    sample_time    diagnostic_time    diagnostic_score    quality_time
DayZSynthesizer    0.0009       None               None                  1.23           None               None                None
HMASynthesizer     0.00098      12.34              456.789               234.567        1.23               1.0                 234.12
HSASynthesizer     0.0008       12.45              34.566                23.456         1.25               1.0                 239.45
```

<details>

<summary>Expand to see the description for each column</summary>

* `synthesizer`: The name of the synthesizer. The first row contains the `DayZSynthesizer`, which is used for creating the random data. Any subsequent rows include the results for the different multi-table synthesizers you are testing.
* `init_time`: The time it takes to initialize the synthesizer
* `preprocess_time`: The time it takes to preprocess the data, getting into a ready state for modeling, in seconds
* `fit_processed_time`: The time it takes to train a model using the processed data, in seconds.
* `sample_time`: The time it takes to generate synthetic data from the trained model, in seconds. This step generates synthetic data that is the same size as the input data.
* `diagnostic_time`: The time it takes to run the [DiagnosticReport](/sdv/evaluation/diagnostic) for the synthetic data, in seconds.
* `diagnostic_score`: The final score of the [DiagnosticReport](/sdv/evaluation/diagnostic), on a scale from 0 (worst) to 1 (best). We expect that all SDV synthesizers produce synthetic data with a score of 1.0
* `quality_time`: The time it takes to run the [QualityReport](/sdv/evaluation/data-quality), in seconds.

</details>

#### Output folder

Your output folder contains the final results in `results.csv`, the random DayZ data, as well as each diagnostic reports for each synthesizer.

```
my_performance_results/
|--- results.csv
|--- DayZ-Data/
       |--- users.csv
       |--- transactions.csv
|--- Diagnostic-Reports/
       |--- hsa_diagnostic.pkl
       |--- independent_diagnostic.pkl
...
```

## FAQ

<details>

<summary>What kinds of results are expected for the DayZSynthesizer?</summary>

The DayZSynthesizer is a special synthesizer used for boostrapping. It does not use machine learning and is only capable of creating data from scratch. Therefore, you should only see results for `init_time` (for initializing the synthesizer) and `sample_time` (for sampling random data).

It does not make sense to run diagnostics or measure the quality of random data, so these columns will also remain blank.

</details>

<details>

<summary>Why do the results contain the diagnostic score but not the quality score?</summary>

All SDV synthesizers are meant to gaurantee a diagnostic score of 1.0 regardless of the data that they are trained on. So even for random data, it is good to check that this score is 1.0 -- and if it's not, please [file a bug](https://github.com/sdv-dev/SDV/issues/new/choose) so our team can take a look.

Meanwhile, the quality score measures the statistical similarity between the input data and synthetic data. It's important to know how long this will take, but the actual score is not meaningful since the function is inputting random data.

</details>


# Sequential Synthesizers

The SDV offers a variety of synthesizers, which use different algorithms to generate synthetic data. This section describes sequential synthesizers, which are able to model [sequential data](/sdv/integration/overview/data-modalities#sequential-data).


# PARSynthesizer

The `PARSynthesizer` uses a deep learning method to train a model and generate synthetic data.

{% hint style="info" %}
**Prerequisites for modeling.** Make sure that you have loaded in your `data` and created a `metadata` object that describes the data. For more information, see the [Data Integration](/sdv/integration/overview) docs.
{% endhint %}

```python
from sdv.sequential import PARSynthesizer

synthesizer = PARSynthesizer(metadata)
synthesizer.fit(data)

synthetic_data = synthesizer.sample(num_sequences=100)
```

{% hint style="warning" %}
**Is the PARSynthesizer suited for your dataset?** The PARSynthesizer is designed to work on **multi-sequence data**, which means that there are multiple sequences (usually belonging to different entities) present within the same dataset. This means that your metadata should include a `sequence_key`. Using this information, the PARSynthesizer creates brand new entities and brand new sequences for each one.

If your dataset contains only a single sequence of data, then the PARSynthesizer is not suited for your dataset.
{% endhint %}

## Creating a synthesizer

When creating your synthesizer, you are required to pass in a [Metadata](/sdv/integration/metadata) object as the first argument. All other parameters are optional. You can include them to customize the synthesizer.

```python
synthesizer = PARSynthesizer(
    metadata, # required
    enforce_min_max_values=True,
    enforce_rounding=False,
    context_columns=['Address', 'Smoker']
)
```

### Parameter Reference

**`enforce_min_max_values`**: Control whether the synthetic data should adhere to the same min/max boundaries set by the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will contain numerical values that are within the ranges of the real data.</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain numerical values that are less than or greater than the real data.</td></tr></tbody></table>

**`enforce_rounding`**: Control whether the synthetic data should have the same number of decimal digits as the real data

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>The synthetic data will be rounded to the same number of decimal digits that were observed in the real data</td></tr><tr><td><code>False</code></td><td>The synthetic data may contain more decimal digits than were observed in the real data</td></tr></tbody></table>

**`locales`**: A list of locale strings. Any PII columns will correspond to the locales that you provide.

<table data-header-hidden><thead><tr><th width="218"></th><th></th></tr></thead><tbody><tr><td>(default) <code>['en_US']</code></td><td>Generate PII values in English corresponding to US-based concepts (eg. addresses, phone numbers, etc.)</td></tr><tr><td><code>&#x3C;list></code></td><td><p>Create data from the list of locales. Each locale string consists of a 2-character code for the language and 2-character code for the country, separated by an underscore.</p><p></p><p>For example <code>[</code><a href="https://faker.readthedocs.io/en/master/locales/en_US.html"><code>"en_US"</code></a><code>,</code> <a href="https://faker.readthedocs.io/en/master/locales/fr_CA.html"><code>"fr_CA"</code></a><code>]</code>. </p><p>For all options, see the <a href="https://faker.readthedocs.io/en/master/locales.html">Faker docs</a>.</p></td></tr></tbody></table>

**`context_columns`**: Provide a list of strings that represent the names of the context columns. Context columns do not vary inside of a sequence. For example, a user's `'Address'` may not vary within a sequence while other columns such as `'Heart Rate'` would. Defaults to an empty list.

**`epochs`**: Number of times to train the GAN. Each new epoch can improve the model.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>128</code></td><td>Run all the data through the neural network 128 times during training</td></tr><tr><td><code>&#x3C;number></code></td><td>Train for a different number of epochs. Note that larger numbers will increase the modeling time.</td></tr></tbody></table>

**`verbose`**: Control whether to print out the results of each epoch. You can use this to track the training time as well as the improvements per epoch.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>False</code></td><td>Do not print out any results</td></tr><tr><td><code>True</code></td><td>Print out the loss value per epoch. The loss values indicate how well the neural network is currently performing, lower values indicating higher quality.</td></tr></tbody></table>

**`cuda`**: Whether to enable GPU usage when training the synthesizer. This may speed up the modeling time.

<table data-header-hidden><thead><tr><th width="179"></th><th></th></tr></thead><tbody><tr><td>(default) <code>True</code></td><td>If available, use the GPU to speed up modeling time. Currently, this will look for <a href="https://developer.nvidia.com/how-to-cuda-python">CUDA</a>, which is available on Linux/Windows machines. If this is not available, then the GPU will not be used.  </td></tr><tr><td><code>False</code></td><td>Do not use the GPU to speed up modeling time.</td></tr></tbody></table>

{% hint style="info" %}
**Is my synthesizer using the GPU?** After calling `fit`, you may notice that the GPU is not immediately used. This is because GPU is not used for the initial, data preprocessing step. After data preprocessing is complete, you should then see the GPU being used for the neural network training.
{% endhint %}

*(deprecated) `cuda`: Please use the `enable_gpu` option to use CUDA, if it's available on your platform.*

{% hint style="info" %}
**Looking for more customizations?** Other settings are available to fine-tune the architecture of the underlying neural network used to model the data. Click the section below to expand.
{% endhint %}

<details>

<summary>Click to expand additional neural network customization options</summary>

These settings are specific to the neural network. Use these settings if you want to optimize the technical architecture and modeling.

**`sample_size`**: The number of times to sample (before choosing and returning the sample which maximizes the likelihood). Defaults to `1`.

**`segment_size`**: Cut each training sequence into several segments by using this parameter. For example, if the `segment_size=10` then each segment contains 10 data points. Defaults to `None`, which means the sequences are not cut into any segments.

</details>

### get\_parameters

Use this function to access the custom parameters you have included for the synthesizer

**Parameters** None

**Output** A dictionary with the parameter names and the values

```python
synthesizer.get_parameters()
```

```python
{
    'enforce_min_max_values': True,
    'enforce_rounding': False
    'context_columns': ['Address', 'Smoker']
}
```

{% hint style="info" %}
The returned parameters are a copy. Changing them will not affect the synthesizer.
{% endhint %}

### get\_metadata

Use this function to access the metadata object that you have included for the synthesizer

**Parameters** None

**Output** A [Metadata](/sdv/integration/metadata) object

```python
metadata = synthesizer.get_metadata()
```

{% hint style="info" %}
The returned metadata is a copy. Changing it will not affect the synthesizer.
{% endhint %}

## Learning from your data

To learn a machine learning model based on your real data, use the `fit` method.

### fit

**Parameters**

* (required) `data`: A dictionary mapping the table name to a [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) containing the real data that the machine learning model will learn from

**Output** (None)

```python
synthesizer.fit(data)
```

{% hint style="info" %}
**Technical Details**: PAR is a Probabilistic Auto-Regressive model that is based in neural networks. It learns how to create brand new sequences of multi-dimensional data, by conditioning on the unchanging, context values.&#x20;

For more details, see the [Sequential Models in the Synthetic Data Vault](https://arxiv.org/pdf/2207.14406.pdf), a preprint from June 2022 that describes the PAR model.
{% endhint %}

## Saving your synthesizer

Save your trained synthesizer for future use.

### save

Use this function to save your trained synthesizer as a Python pickle file.

**Parameters**

* (required) `filepath`: A string describing the filepath where you want to save your synthesizer. Make sure this ends in `.pkl`

**Output** (None) The file will be saved at the desired location

```python
synthesizer.save(
    filepath='my_synthesizer.pkl'
)
```

### load (utility function)

Use this utility function to load a trained synthesizer from a Python pickle file. After loading your synthesizer, you'll be able to sample synthetic data from it.

**Parameters**

* (required) `filepath`: A string describing the filepath of your saved synthesizer

**Output** Your synthesizer object

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(
    filepath='my_synthesizer.pkl'
)
```

*This utility function works for any SDV synthesizer.*

## What's next?

After training your synthesizer, you can now sample synthetic data. See the [Sampling](/sdv/sampling/sample-realistic-data) section for more details.

```python
synthetic_data = synthesizer.sample(num_sequences=10)
```

{% hint style="info" %}
**Want to improve your synthesizer?** Input logical rules in the form of [constraints](/sdv/modeling/constraint-augmented-generation-cag) and customize the transformations used for [preprocessing](/sdv/modeling/configuration/preprocessing) the data.
{% endhint %}

## FAQs

<details>

<summary>How do I cite PAR?</summary>

*Kevin Zhang, Kalyan Veeramachaneni, Neha Patki.* **Sequential Models in the Synthetic Data Vault.** Preprint, June 2022.

```
@unpublished{par,
   title={Sequential Models in the Synthetic Data Vault},
   author={Zhang, Kevin and Veeramachaneni, Kalyan and Patki, Neha},
   year={2022}
}
```

</details>

<details>

<summary>What happens if columns don't contain numerical data?</summary>

This synthesizer models non-numerical columns, including columns with missing values.

Although the Gaussian Copula algorithm is designed for only numerical data, this synthesizer converts other data types using Reversible Data Transforms (RDTs). To access and modify the transformations, see [Advanced Features](broken://pages/lkQwNv8HIr77DavQ7W3W).

</details>

<details>

<summary>Can I call <code>fit</code> again even if I've previously fit some data?</summary>

Yes, even if you're previously fit data, you should be able to call the `fit` method again.

If you do this, the synthesizer will **start over from scratch** and fit the new data that you provide it. This is the equivalent of creating a new synthesizer and fitting it with new data.

</details>


# Constraint-Augmented Generation (CAG)

Do you have **business rules** in your dataset? These are deterministic rules that every single row in your data must follow in order to be considered valid. By default, SDV synthesizers are *probabilistic* so they may not learn to match your rule 100% of the time.

The good news is that you can input your business rules into your synthesizer using **constraints**. Constraint-augmented generation ensures that your synthetic data meets the constraint — 100% of the time.

{% hint style="success" %}
**Auto-detect business rules or add them in yourself.** SDV automatically detects common business rules that your datasets may have. You inspect what SDV has detected, and modify those constraints.
{% endhint %}

## Constraint Example

One example of a business rule is when the values in one column always have to be greater than values in another column. This is true for every single row of data.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F4s44a4kLJv2Cs9XjtFMt%2Fconstraint-augmented-generation-cag-constraint-example_June%2009%202025.png?alt=media&amp;token=3c874126-a797-4478-9248-98147241c122" alt=""><figcaption><p>In this business rule, the <code>checkout_date</code> must be greater than the <code>checkin_date</code> for all rows.</p></figcaption></figure>

**❖ Auto-detect constraints.** SDV automatically detects a constraint that corresponds to this business rule ([Inequality constraint](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/inequality)). You can add it to your synthesizer to ensure that the synthetic data follows the rule. *The auto-detection feature is available in the* [*CAG bundle*](/sdv/explore/sdv-bundles/cag)*.*

```python
from sdv.single_table import GaussianCopulaSynthesizer

my_synthesizer = GaussianCopulaSynthesizer(metadata)
constraints = my_synthesizer.detect_constraints(data)
my_synthesizer.add_constraints(constraints)
```

```
1 Constraints Detected

0. ChainedInequality(column_names=['checkin_date', 'checkout_date'], table_name='fake_hotel_guests')
```

**Add constraints directly.** You can also create the constraint yourself and supply it to your synthesizer.

```python
from sdv.cag import Inequality

# create a constraint that corresponds to your business rule
my_constraint = Inequality(
    low_column_name='checkin_date',
    high_column_name='checkout_date'
)

# add the constraint to your SDV synthseizer
my_synthesizer.add_constraints(constraints=[
    my_constraint
])
```

## Predefined Constraint Classes

Predefined constraints apply logic within a single table or between multiple tables. Predefined constraints represent common business rules that may appear in your dataset.  SDV is able to auto-detect predefined constraint classes, and you are also able to create and apply them yourself.

[Browse the predefined constraints](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints) to learn more. We also recommend going through our [tutorial](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Programmable Constraints

If your logic cannot be described by any of the predefined constraints, please [reach out to us](/sdv/support/troubleshooting/sdv-usage#other-issues) with the details of your dataset. We will track this as a feature request.

{% hint style="success" %}
**Have you purchased the CAG bundle?** For our SDV Enterprise customers who have purchased the CAG bundle, our team is able to provide additional support for constraints. After you [reach out to us](/sdv/support/troubleshooting/sdv-usage#other-issues), we will be able to provide you with some specially programmed constraints for you to test.
{% endhint %}

## Constraints API

Detect, create, and add constraint objects to your synthesizer. For more details, see the [API Reference](/sdv/modeling/constraint-augmented-generation-cag/constraints-api).

## FAQs

<details>

<summary>Is it possible to apply a constraint even if my dataset doesn't always follow it?</summary>

SDV synthesizers check to ensure that your real dataset follows the constraints 100% of the time. So it is currently **not allowed** to add a constraint if some rows of your data violate the rule. There are a rew reasons for this:

1. A constraint is supposed to represent business logic that must be met in order for the synthetic data to be considered valid. For example, it would not even be possible to load the synthetic data into your downstream application if it doesn't follow the rule 100% of the time. If your real dataset doesn't always adhere to this logic, then it indicates that the rule isn't strictly needed.
2. The synthesizer is meant to learn and emulate the patterns in the real dataset as-is. So if you present real data that violates a few of the rules, the synthesizers are designed to understand that that create similar synthetic data. (If you'd like the synthesizer to purposefully change the patterns, we recommend using [conditional sampling](/sdv/sampling/conditional) after the synthesizer is already created.)
3. We developed the internal algorithms of each constraint assuming that the rule is 100% true in the real data. Therefore, if we were to allow data that is not 100% valid, the algorithms may not run as intended. You may find that the synthesizer would crash, or produce invalid error/null values.

</details>

<details>

<summary>How is modeling &#x26; sampling performance impacted by constraints?</summary>

In most cases, the time it takes to fit the model and sample synthetic data should not be significantly affected. However, there are certain scenarios where you may notice a slow-down:

* You have a large number of constraints that overlap. That is, multiple constraints are referencing the same columns of the data.
* You are conditional sampling on the constrained columns. This requires some special processing and it may not always be possible to efficiently create conditional synthetic data.

For any questions or feature requests related to performance, please [provide feedback](/sdv/support/troubleshooting/sdv-usage#other-issues) describing your data, constraints and sampling needs.

</details>

<details>

<summary>How does the SDV handle the constraints?</summary>

Under-the-hood, the SDV uses a combination of strategies to ensure that the synthetic data always follows the constraints. These strategies are:

* **Transformation**: Most of the time, it's possible to transform the data in a way that guarantees the models will be able to learn the constraint. This is paired with a reverse transformation to ensure the synthetic data looks like the original.
* **Reject Sampling**: Another strategy is to model and sample synthetic data as usual, and then throw away any rows in the synthetic data that violate the constraints.
* **Algorithmic Injection**: Complex CAG patterns sometimes come with their own algorithms for ensuring robust and accurate modeling. These algorithms may learn additional patterns from your data. They compatible with any SDV synthesizer.

</details>

<details>

<summary>Can I use constraints to generate data from scratch (DayZSynthesizer)?</summary>

Yes, if you are an SDV Enterprise user, you have access to the [DayZSynthesizer](/sdv/modeling/single-table-synthesizers/dayzsynthesizer) which can be used to generate data from scratch. You are be able to add single-table constraints to DayZSynthesizer, including your own, programmable single-table constraint.

*Multi-table constraints are not yet supported.*

</details>


# ❖ Auto-Detect Constraints

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

SDV is able to automatically identify common business rules in your dataset and add them as constraints. Use the functionality described in this guide to auto-detect, inspect, update, and add constraints to your synthesizer.

```python
from sdv.multi_table import HSASynthesizer

# create a synthesizer
synthesizer = HSASynthesizer(metadata)

# detect constraints based on the data and add them to the synthesizer
detected_constraints = synthesizer.detect_constraints(data)
synthesizer.add_constraints(detected_constraints)

# now fit and sample the synthesizer like usual
synthesizer.fit(data)
synthetic_data = synthesizer.sample_scale(scale=1.0)
```

## Detect Constraints

SDV detects constraints using your synthesizer and data.

### \<synthesizer instance>.detect\_constraints

Use this method with an SDV synthesizer in order to detect constraints. *All single- and multi-table SDV synthesizers are supported except for DayZSynthesizer.*

```python
synthesizer = HSASynthesizer(metadata)
constraints = synthesizer.detect_constraints(data=my_datset)
```

{% hint style="warning" %}
**Running the auto-detection does not automatically apply the constraints.** This algorithm provides you a list of constraints that you can review and modify before officially adding to the synthesizer.&#x20;
{% endhint %}

**Parameters**:

* (required) `data`: Your data, a dictionary that maps each table name to a pd.DataFrame object that contains the data.
* `verbose`: A boolean that controls whether to print out information during the detection.
  * (default) `True`: Show a progress bar during detection and print out a list of all all the constraints that are found.
  * `False`: Do not print anything out during detection.
* `constraint_classes`: A list of strings describing which constraint classes to detect
  * (default) `['OneHotEncoding', 'FixedNullCombinations', 'ChainedInequality', 'DenormalizedTable']`: Detect constraints for these predefined constraint classes. *Note that DenormalizedTable will not be applied to single-table synthesizers.*
  * `[<string>]`: Provide a list of strings with the class names of the [Predefined constraints](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints). *Currently, SDV offers support for auto-detecting all predefined constraints except for ForeignToForeignKey and ReferenceTable.*&#x20;
  * `None`: Do not limit the constraint classes. SDV will detect all available constraint classes.&#x20;
* `table_names`: A list of table names describing which tables to use when detecting constraints.&#x20;
  * (default) `None`: Use all the tables names from the dataset
  * `[<string>]`: Provide a list of strings with the table names. If provided, SDV will only detect constraints within these tables.
* `detection_params`: A dictionary describing any additional parameters to use when detecting each constraint class. This advanced functionality can be used to control the exact specifications of what should be detected.
  * (default) `None`: Use the default detection capabilities for each of the constraint classes.
  * `<dictionary>`: A dictionary that maps each constraint class name (string) to a dictionary of parameters for detection. If available, these parameters are listed the API for each individual constraint class. (See [Predefined Constraints](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints).)

**Returns:** A `ConstraintList` object. This object is similar to a list of constraints. However, it has specialized methods for adding and removing constraints, as the order of constraints may impact the algorithm.

## Inspect Constraints

After detecting constraint, you can inspect them by printing out your constraint set object.

### print(\<constraint\_list\_instance>)

Print out your `ConstraintList` object to see a list of constraints. Each constraint will be printed out next to its index, which indicates the order in which it will be applied.

```
print(constraints)

ConstraintList (1 constraints)
0. ChainedInequality(column_names=['checkin_date', 'checkout_date'], table_name='guests')
```

## Remove Constraints

You can remove any of the auto-detected constraints in from your `ConstraintList` object.

### \<constraint\_list\_instance>.remove\_constraint

Use this function to remove any of the constraints in your `ConstraintList` object. Keep in mind that the constraint list is designed to be applied in order. Removing a constraint from the middle of the list may mean that some of the constraints that follow it are no longer applicable.

```python
constraints.remove_constraint(position=2)
```

**Parameters**:

* (required) `position`: An integer describing the position of the constraint that you want to remove. To find the position for a constraint, print out the list.
* &#x20;`redetect`: Whether to re-detect constraints that occur after the one you've removed.
  * (default) `False`: Remove the constraint and any other constraints that come after it if they are no longer valid.
  * `True`: Remove the constraint and any other constraints that come after it if they are no longer valid. Then, re-detect constraints in case there are more.
* `verbose`: A boolean that controls whether to print out information during removal/re-detection.
  * (default) `True`: Show a progress bar during deletion and re-detection and print out a list of all all the constraints that are found.
  * `False`: Do not print anything out during detection.

**Returns** `None`. The `ConstraintList` instance will no longer have the constraint, and may have some new constraints based on re-detection.

{% hint style="info" %}
**Why is re-detection necessary?** In SDV, each constraint might internally transform the data. For example, one constraint might merge two tables together. Then the following constraint assumes that the merged table exists, and it might use the merged table in order to perform its logic.

In this case, removing the first constraint would make the second one invalid. SDV will either remove it (if `redetect=False`) or try to re-detect it based on what the data now looks like (if `redetect=True`).
{% endhint %}

## Add Auto-Detected Constraints to Your Synthesizer

Finally, when all the auto-detected constraints look good, you can add them to a single- or multi-table SDV synthesizer.

### \<synthesizer\_instance>.add\_constraints

Use this method to add the auto-detected constraints to your synthesizer instance. *All single- and multi-table SDV synthesizers are supported except for DayZSynthesizer.*

```python
synthesizer.add_constraints(constraints)
```

**Parameters:**

* (required) `constraints`: A `ConstraintList` object containing your auto-detected constraints.

**Returns** `None`. The constraints are now added to the synthesizer. Please make sure to fit the synthesizer first. When you sample synthetic data, it is guaranteed to follow the constraints.

## FAQ

<details>

<summary>Does the order of the constraints matter?</summary>

Yes! When SDV auto-detects the constraints, they are meant to be applied in the same order that they are found in. This is why the `ConstraintList` object in an indexed, ordered list. If you had already added constraints to your synthesizer, then the auto-detected constraints are meant to be applied *after* the ones already added.

Order is important because each constraint might internally transform the data. For example, one constraint might merge two tables together. Then the following constraint assumes that the merged table exists, and it might use the merged table in order to perform its logic. This is also why it's tricky to remove a constraint. In this case, removing the first constraint would make the second one invalid.

</details>

<details>

<summary>Can I auto-detect multiple times?</summary>

Yes, you can call `detect_constraints` multiple times on the synthesizer with different constraints and parameters. The synthesizer detects new constraints based on the ones you've already added to the synthesizer. So be sure to add each batch of constraints before auto-detecting new ones.

```python
# detect the first batch of constraints and add them to the synthesizer
constraints_batch1 = synthesizer.detect_contraints(
    data,
    constraint_classes=['OneHotEncoding', 'FixedNullCombinations', 'ChainedInequality', 'DenormalizedTable'])
synthesizer.add_constraints(constraints_batch1)

# now detect the second batch of constraints on top of the ones we've already added
constraints_batch2 = synthesizer.detect_constraints(
    data,
    constraint_classes=['CarryOverColumns', 'MixedScales', 'PolymorphicRelationship'])
synthesizer.add_constraints(constraints_batch2)
```

</details>

<details>

<summary>Why do I see columns and table names that are not in my original data?</summary>

Some constraints are designed to capture the logic by transforming the data. These constraints may merge, delete, or add tables and columns. In these cases, SDV tracks the modification of your schema, including any new table and column names. It is expected for any additional constraints to be using the newly created tables and columns.&#x20;

For more information about how constraints may transform yoru data, see the [CAG FAQs](/sdv/modeling/constraint-augmented-generation-cag#how-does-the-sdv-handle-the-constraints).

</details>

<details>

<summary>Why are some constraint classes not enabled by default?</summary>

By default, SDV auto-detection looks for specific predefined constraints that are commonly found in enterprise schemas. Searching through all predefined constraints can be performance intensive, so we have found that defaulting to the most commonly-appearing constraints is a good compromise.&#x20;

Additionally, some of the predefined constraints completely cover others in terms of their logic. For example, the [ChainedInequality](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/chainedinequality) constraint completely covers all logic that could be described using the [Range](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/range) or [Inequality](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/inequality) constraints. It would be redundant to detect all of them.

If you'd like to detect constraints for another class, you can call auto-detect again and provide the constraint classes you'd like to add. Be sure to add any existing constraints to your synthesizer first before calling auto-detect again.

```python
# detect and add the default contraints
constraints = synthsizer.detect_constraints(data)
synthesizer.add_constraints(constraints)

# now specially detect some additional constraint classes that are not 
# enabled by default
additional_constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['CarryOverColumns', 'MixedScales', 'PolymorphicRelationship'])
synthesizer.add_constraints(additional_constraints)
```

</details>

<details>

<summary>What if the auto-detection missed a predefined constraint?</summary>

If auto-detection has missed a constraint, you can try calling `detect_constraints` again with the specific table names and constraint classes that correspond to it. Be sure to also update the detection parameters to allow for its detection.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['Inequality'],
    table_names=['guests'],
    detection_params={
        'Inequality': { 'sdtypes': ['numerical', 'datetime'] }
    }
)
```

If SDV is still unable to detect the constraint, there may be two things going on:

1. The constraint logic may not actually hold true for all the rows of your dataset. In order to add a constraint, SDV requires that it is always true in your dataset. (For more info, see the [CAG FAQ](/sdv/modeling/constraint-augmented-generation-cag#is-it-possible-to-apply-a-constraint-even-if-my-dataset-doesnt-always-follow-it).)
2. Or alternatively, it could be the case that you have already applied a constraint to your synthesizer that is making this one redundant or invalidating it in some way. For example, a constraint you have already added may be deleting a column; that column is no longer available to be involved in another constraint. If this is happening to you, you can try defining and adding the constraint yourself to the synthesizer. This constraint would now require reject sampling. (For more info, see the [CAG FAQ](/sdv/modeling/constraint-augmented-generation-cag#how-does-the-sdv-handle-the-constraints).)

```python
from sdv.cag import Inequality

my_constraint = Inequality(
    table_name='guests',
    low_column_name='checkin_date',
    high_column_name='checkout_date',
    strict_boundaries=True
)

synthesizer.add_constraints([my_constraint])
```

</details>

<details>

<summary>Does the synthesizer affect which constraints are auto-detected?</summary>

The constraint auto-detection is done only based on your data and metadata. The synthesizer algorithm itself does not affect the outcome of this (eg. GaussianCopula, CTGAN, HSA, etc.).

The auto-detection is done using a synthesizer because this where it fits best within the workflow. If you don't know which synthesizer you'd like to use, or you just want to know the constraints for now, we recommend creating a synthesizer just for the purposes of constraint detection.

```python
from sdv.single_table import GaussianCopulaSynthesizer

# create a synthesizer even if you don't plan to use it right now
synthesizer = GaussianCopulaSynthesizer(metadata)

# use it to detect constraints
constraints = synthesizer.detect_constraints(data)
```

You can add the constraints to a different synthesizer than the one you originally used for detection, as long as it's within the same modality (single or multi-table).

```python
from sdv.single_table import CTGANSynthesizer

# create a different synthesizer with the same single-table modality
synthesizer = CTGANSynthesizer(metadata)

# add the constraint we'd detected before
synthesizer.add_constraints(constraints)
```

</details>


# Predefined Constraints

Predefined constraint classes are available for frequently occurring business rules. The business rules can be isolated to a single table or can be applied across multiple tables.

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Single Table

These constraints can be applied within an individual table of your dataset

<table><thead><tr><th width="213.66666666666666">Constraint</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td><a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixedcombinations">FixedCombinations</a></td><td>No shuffling is allowed other than what's already observed in the data</td><td>The <code>city</code> and <code>country</code> values cannot be shuffled to create new permutations.</td></tr><tr><td><a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixedincrements">FixedIncrements</a></td><td>All the numerical values are increments of a whole number</td><td>All values in <code>salary</code> must be divisible by 1000</td></tr><tr><td><a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/inequality">Inequality</a></td><td>The value in one column must always be greater than the other</td><td>The <code>checkout_date</code> must always be after the <code>checkin_date</code></td></tr><tr><td><a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/onehotencoding">OneHotEncoding</a></td><td>The original data columns represent a one hot encoding scheme</td><td>Exactly 1 of the following columns has a <code>1</code> in each row: <code>not_subscribed</code>, <code>basic_subscriber</code>, <code>premium</code></td></tr><tr><td><a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/range">Range</a></td><td>The value in one column is bounded by the values in other columns</td><td>The <code>parent_age</code> must be in between <code>child_age</code> and <code>grandparent_age</code></td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/chainedinequality">ChainedInequality</a></td><td>A chain of 2 or more columns in an inequality</td><td><code>purchase_date</code> &#x3C; <code>start_date</code> &#x3C; <code>end_date</code> &#x3C; <code>expiration_date</code> &#x3C; <code>termination_date</code></td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/columnformula">ColumnFormula</a></td><td>One column can be formulaically determined by other columns in the same table.</td><td>In the same table, there is a simple formula that the <code>total_rate</code> column value = <code>room_rate</code> + <code>amenities+fee</code></td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixednullcombinations">FixedNullCombinations</a></td><td>No shuffling is around for the missing values, other than what's already observed in the data</td><td>The <code>city</code> and <code>country</code> columns must both either be null together or not at all.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/mixedscales">MixedScales</a></td><td>The value of one categorical column determines the scale of another numerical column.</td><td>If the value of <code>test_type</code> is <code>'blood_pressure'</code> then the value of <code>test_result</code> must be within a reasonable for this test only.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/selfreferentialhierarchy">SelfReferentialHierarchy</a></td><td>A column in the table refers to a different column in the same table.</td><td>The <code>Manager ID</code> column refers to the <code>Employee ID</code> column in the same table.</td></tr></tbody></table>

## Multi Table

These constraints can be applied between multiple different tables of your dataset.

<table><thead><tr><th width="215.859375">Constraint</th><th width="268.72265625">Description</th><th>Example</th></tr></thead><tbody><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/carryovercolumns">CarryOverColumns</a></td><td>The same columns are present in a parent table and a child table, and the values of those columns have to match up according to the connection.</td><td>The account <code>Type</code> in one table must match the corresponding account <code>Type</code> in another table.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/denormalizedtable">DenormalizedTable</a></td><td>A table that was created by denormalizing a parent and child table.</td><td>The <code>Transactions-Users</code> table was created by denormalizing the <code>Transactions</code> and <code>Users</code> table. (The user info is repeated for each transaction they've made.)</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/foreigntoforeignkey">ForeignToForeignKey</a></td><td>There are foreign keys in multiple tables but no primary key to attach them to.</td><td>The <code>Warehouse ID</code> column in multiple tables is referring to the same concept.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/foreigntoprimarykeysubset">ForeignToPrimaryKeySubset</a></td><td>There is a 1-to-many connection between tables, but only certain values are allowed to have connections.</td><td>Only accounts with <code>Type=Premium</code> are allowed to have children in another table.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/polymorphic-relationship">PolymorphicRelationship</a></td><td>The values inside a foreign key column can refer to different tables, depending on the context.</td><td>A transaction log's <code>Details</code> might sometimes reference a row in the <code>Credit Accounts</code> table and other times in the <code>Debit Accounts</code> table.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/primarytoprimarykeysubset">PrimaryToPrimaryKeySubset</a></td><td>There is a 1-to-1 connection between the primary keys of two or more tables but only certain values are allowed to have connections.</td><td>Only users with <code>Is Minor=True</code> are allowed to have an entry in another table.</td></tr><tr><td>❖ <a href="/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/referencetable">ReferenceTable</a></td><td>A table acts as an unchangeable reference. You do not want to synthesize an new information in it.</td><td>The <code>City</code> table should act as a reference; you do not want to synthesize new cities.</td></tr></tbody></table>

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}


# FixedCombinations

The **FixedCombinations** constraint enforces that the combinations between a set of columns are fixed. That is, no other permutations or shuffling is allowed other than what's already observed in the data.

## Constraint API

Create a `FixedCombinations` constraint.

**Parameters:**

* (required) `column_names`: A list of two or more columns whose combinations are fixed. The columns must be categorical. The SDV will not further shuffle the data between these column names.&#x20;
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.

```python
from sdv.cag import FixedCombinations

my_constraint = FixedCombinations(
    column_names=['city', 'country']
)
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## ❖ Auto-Detection

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Auto-detection is allowed for this constraint but it is not enabled by default. We recommend auto-detecting the [FixedNullCombinations](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixednullcombinations) constraint instead, as it covers all the logic of this constraint and more!

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['FixedCombinations']
)
```

**Detection Parameters**: None

## FAQs

<details>

<summary>Why am I only allowed to input categorical columns?</summary>

This constraint can only fix the combination of categorical columns because only categorical columns have a fixed set of possible values. For other types of columns such as numerical, datetime, or PII, it's possible to synthesize new values that never appeared in the real data. So it's not possible to fix their permutations.

In some cases, you may not want to fix the actual value of a numerical, datetime, or PII column but rather just fix whether it's null or non-null. In that case, we recommend using the [FixedNullCombinations](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixednullcombinations) constraint instead.

</details>

<details>

<summary>Why can't I apply this constraint to a single column?</summary>

This constraint ensures that the synthetic data only contains combinations that exist in the real data. If there is only one column, there are no combinations.

The SDV already guarantees that the synthetic data contains the same categorical values as the real data for a single column.

</details>

<details>

<summary>The synthetic data has the same combination multiple times. Is this intended?</summary>

Yes. This constraint prevents the SDV from creating additional permutations between columns. But the same permutations are allowed to appear multiple times.

For example, it will prevent the SDV from inventing new `city, country` pairs, but a valid pair such as `Boston, USA` may appear more than once.

</details>

<details>

<summary>What happens to null values?</summary>

This constraint will consider a null value as part of a combination. For example if a null in one column always appears next to a `True` value in another column, the constraint will learn that.

If you would to only fix the combinations of the null values (and allow synthesizer to continue creating new permutations of the non-null values), please used the [FixedNullCombinations](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixednullcombinations) constraint instead.

</details>


# FixedIncrements

The **FixedIncrements** constraint enforces that all the values in a column are increments of a particular, fixed value. That is, all the data must be divisible by the value.

## Constraint API

**Parameters**

* (required) `column_name`: The name of the column that must follow the constraint. This must be a numerical column.
* (required) `increment_value`: The size of the increment. This must be a positive integer
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.

```python
from sdv.cag import FixedIncrements

my_constraint = FixedIncrements(
    column_name='salary',
    increment_value=1000
)
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## ❖ Auto-Detection

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['FixedIncrements']
)
```

**Detection Parameters**: None

## FAQs

<details>

<summary>What happens to missing values?</summary>

This constraint ignores missing values in the dataset. The constraint is valid as long as the numerical values (non-missing values) are increments of the provided integer value.

</details>

<details>

<summary>What if I want to specify a smaller increment 1 for rounding decimal digits?</summary>

The increments in this constraint must be integers greater than 1.

The SDV already learns the number of digits in your real data and matches them to your synthetic data. For example, if you have a column that represents US dollar amounts, then the real data will only contain values with 2 decimal digits. The SDV will learn this and ensure the synthetic data also has 2 decimal digits. No constraints are needed in this case.

</details>


# Inequality

The **Inequality** constraint enforces an inequality relationship between a pair of columns. For every row, the value in one column must be greater than a value in another.

## Constraint API

**Parameters**

* (required) `low_column_name`: The name of the column whose values must be lower. *Only numerical and datetime columns are allowed.*
* (required) `high_column_name`: The name of the column whose values must be greater. *Only numerical and datetime columns are allowed.*
* `strict_boundaries`: Whether the high column must be strictly greater than the low column
  * (default) `True`: The value in the high column must be strictly greater than the value in the low column
  * `False`: The value in the high column must be greater than *or equal to* the value in the low column.
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.

```python
from sdv.cag import Inequality

my_constraint = Inequality(
    low_column_name='checkin_date',
    high_column_name='checkout_date',
    strict_boundaries=True
)
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## ❖ Auto-Detection

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Auto-detection is allowed for this constraint but it is not enabled by default. We recommend auto-detecting the [ChainedInequality](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/chainedinequality) constraint instead, as it covers all the logic of this constraint and more!

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['Inequality']
)
```

**Detection Parameters**: By default, SDV detects this constraint for `datetime` columns only. Use the the `sdtypes` parameter to update this. Provide a list of sdtypes to detect (`datetime`, `numerical`, or both).

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['Inequailty'],
    detection_params={
        'Inequality': { 'sdtypes': [ 'numerical', 'datetime' ] }
    }
)
```

## FAQs

<details>

<summary>What happens to missing values?</summary>

This constraint ignores missing values. The constraint considered is valid as long as the numerical values (non-missing values) follow the inequality.

</details>

<details>

<summary>What if I want to compare a column to a single, fixed value?</summary>

Many of our SDV synthesizers are already designed to learned the min/max values in every column and replicate the ranges in the synthetic data. This parameter is often called `enforce_min_max_values` and it applies to all numerical/datetime columns. For more information, check your [synthesizer's API guide](/sdv/modeling/single-table-synthesizers).

You can also control the enforcement on a per-column basis. Turn on/off the enforcement on individual columns by accessing and updating the transformers. For more information, see the [Preprocessing guide](broken://pages/yKD7rwpmUt1JJLCbDLqE).

Both of these options will allow you to fix the range (as observed in the real data) or expand it (by not enforcing it). If you'd like to further *restrict* the range, we encourage you to model the data as-is and use [conditional sampling](/sdv/sampling/conditional/single-table) to get the range you need.

</details>


# OneHotEncoding

The **OneHotEncoding** constraint enforces that a set of columns follow a[ one hot encoding scheme↗](https://en.wikipedia.org/wiki/One-hot#Machine_learning_and_statistics). That is, exactly one of the columns must contain a value of `1` while all the others must be `0`.

## Constraint API

Create a `OneHotEncoding` constraint.

**Parameters:**

* (required) `column_names`: A list of column names that, together, form the one hot encoding scheme. The columns must be listed as *numerical* in your metadata.
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.
* `learning_strategy`: A string that controls how SDV should ultimately enforce the constraint internally. No matter which strategy you chose, you are guaranteed to have synthetic, one-hot encoded columns. Options:
  * (default) `'one_hot'`: SDV will keep the one hot encoded columns for the underlying ML model to learn. When creating synthetic data, SDV will enforce that only 1 of the columns will be picked.
  * `'categorical'`: SDV will collapse the one hot encoded columns into a single, categorical column for the underlying ML model to learn. When creating synthetic data, SDV will expand the category value back into the original set of one hot encoded columns.

```python
from sdv.cag import OneHotEncoding

my_constraint = OneHotEncoding(
    column_names=['status_active', 'status_inactive', 'status_on_hold'],
    learning_strategy='one_hot'
)
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## ❖ Auto-Detection

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Auto-detection is allowed for this constraint and it is enabled by default. Run the detection with your data to detect all instances of this constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(data)
```

**Detection Parameters**: None

## FAQs

<details>

<summary>What is the difference between the learning strategies? Which one do I pick?</summary>

The learning strategy parameter influences how SDV will internally enforce the constraints.

* `'one_hot'`: This strategy will keep all the columns in place for the underlying ML model to learn from. Keeping all the columns may impact performance, making the fitting and the sampling slower than before. However for certain synthesizers like TVAE, this will produce higher quality data — meaning that the proportion of categories will more closely match the original.
* `'categorical'`: This strategy will collapse the one hot encoded columns into a single, categorical column for the underlying ML model to learn from. This can improve performance, as there is only 1 column to learn instead of many. For certain synthesizers like GaussianCopula, this may also produce higher quality data — meaning that the proportion of categories will more closely match the original. *When creating synthetic data, SDV will expand the value back into the original set of one hot encoded column names.*

It's important to note that both learning strategies will ultimately create 100% valid synthetic data with a set of one hot encoded columns.&#x20;

</details>


# Range

The **Range** constraint enforces that for all rows, the value of one of the columns is bounded by the values in the other two columns.

## Constraint API

Create a `Range` constraint.

* (required) `low_column_name`: The name of the column that contains the lowest value. This must be a numerical or datetime column.
* (required) `middle_column_name`: The name of the column that must be between the low and the high columns. This must be a numerical or datetime column.
* (required) `high_column_name`: The name of the column that contains the highest value. This must be a numerical or datetime column.
* `strict_boundaries`: Whether the boundaries between each of the comparisons are strict
  * (default) `True`: The middle column must be strictly greater than the low column and strictly less than the high column.
  * `False`: The middle column must be greater than *or equal to* the low column and less than *or equal to* the high column
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.

```python
from sdv.cag import Range

my_constraint = Range(
    low_column_name='child_age',
    middle_column_name='parent_age',
    high_column_name='grandparent_age',
    strict_boundaries=True
)
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## ❖ Auto-Detection

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Auto-detection is allowed for this constraint but it is not enabled by default. We recommend auto-detecting the [ChainedInequality](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/chainedinequality) constraint instead, as it covers all the logic of this constraint and more!

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['Range']
)
```

**Detection Parameters**: By default, SDV detects this constraint for `datetime` columns only. Use the the `sdtypes` parameter to update this. Provide a list of sdtypes to detect (`datetime`, `numerical`, or both).

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['Range'],
    detection_params={
        'Range': { 'sdtypes': [ 'numerical', 'datetime' ] }
    }
)
```

## FAQs

<details>

<summary>What happens to missing values?</summary>

This constraint ignores missing values. The constraint considered is valid as long as the numerical values (non-missing values) follow the logic.

</details>

<details>

<summary>What if I want to compare a column to fixed values?</summary>

Many of our SDV synthesizers are already designed to learned the min/max values in every column and replicate the ranges in the synthetic data. This parameter is often called `enforce_min_max_values` and it applies to all numerical/datetime columns. For more information, check your [synthesizer's API guide](/sdv/modeling/single-table-synthesizers).

You can also control the enforcement on a per-column basis. Turn on/off the enforcement on individual columns by accessing and updating the transformers. For more information, see the [Preprocessing guide](broken://pages/yKD7rwpmUt1JJLCbDLqE).

Both of these options will allow you to fix the range (as observed in the real data) or expand it (by not enforcing it). If you'd like to further *restrict* the range, we encourage you to model the data as-is and use [conditional sampling](/sdv/sampling/conditional/single-table) to get the range you need.

</details>


# ❖ CarryOverColumns

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **CarryOverColumns** constraint when you have the same columns in different tables, and the values of those columns have to match up according to a key (ID) column.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FGk5AgJtSya7oBJ2bWnrj%2Fcarry-over-columns_May%208%202025.png?alt=media&amp;token=995ca738-caf1-403f-8409-6782c6a14315" alt=""><figcaption><p>The 'Type' column is carried over based on the ID/Account ID information.</p></figcaption></figure>

## Constraint API

Create a `CarryOverColumns` constraint

**Parameters**:&#x20;

* (required) `common_column_info`: A list of dictionaries that describe all the columns within your dataset that are carried over. Each dictionary should have the following keys:
  * `table_name`: The name of the table that contains the column
  * `carryover_column_name`: The name of the column that is repeated across your dataset
  * `key_column_name`: The name of the column to use as the key for the carried over column. Must be a PII or ID sdtype.

Please supply 2 or more columns for this constraint.

```python
from sdv.cag import CarryOverColumns

my_constraint = CarryOverColumns(
    common_column_info=[{
        'table_name': 'Account',
        'carryover_column_name': 'Type',
        'key_column_name': 'ID'
    }, {
        'table_name': 'Transaction',
        'carryover_column_name': 'Type',
        'key_column_name': 'Account ID'
    }]
)
```

Make sure that all the tables and columns you provide are in your [Metadata](/sdv/integration/metadata).

### Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['CarryOverColumns']
)
```

**Detection Parameters**: None


# ❖ ChainedInequality

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

The **ChainedInequality** constraint enforces a chain of inequality relationships between any number of columns. For every row, the value in these columns must follow a strict ordering system.

## Constraint API

Create a `ChainedInequality` constraint.

**Parameters:**

* (required) `column_names`: A list of strings that represent the column names. They should appear in ascending order (lowest to highest). *Only numerical and datetime columns are allowed.*
* `strict_boundaries`: Whether a column must be strictly greater than previous one
  * (default) `True`: The value in the higher column must be strictly greater than the lower one
  * `False`: The value in the higher column must be greater than *or equal to* the value of the lower one
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.

```python
from sdv.cag import ChainedInequality

my_constraint = ChainedInequality(
    column_names=['purchase_date', 'start_date', 'end_date', 'expiration_date']
)   
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint and it is enabled by default. Run the detection with your data to detect all instances of this constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(data)
```

**Detection Parameters**: By default, SDV detects this constraint for `datetime` columns only. Use the the `sdtypes` parameter to update this. Provide a list of sdtypes to detect (`datetime`, `numerical`, or both).

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['ChainedInequality'],
    detection_params={
        'ChainedInequality': { 'sdtypes': [ 'numerical', 'datetime' ] }
    }
)
```


# ❖ ColumnFormula

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **ColumnFormula** constraint when one of the columns in your table can be directly, formulaically determined based on the values in some other columns.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FmoxzoTOKLJNGoohjtqwq%2F1-hotel-guests.png?alt=media&amp;token=0ced213d-2ff5-48ad-b16f-5cee8ca0b37f" alt=""><figcaption><p>This dataset contains information about guests checking into a hotel. The <code>total_rate</code> column can be formulaically determined from the <code>room_rate + amenities_fee</code>.</p></figcaption></figure>

## Formula Definition

To use the constraint, you would first need to define a formula function.

* Input: This function should take as input a pandas DataFrame containing all the columns that are needed for for the formula.
* Output: It should output a pandas Series object containing the formulaically-determined column.

In the example below, we're showing a simple example where the `total_rate = room_rate + amenities_fee`.

```python
def get_room_total(input):
  """ The total rate is the sum of room_rate and amenities_fee
  Input columns are room_rate, amenities_fee
  Output column is the total_rate
  """
  total_rate = input['room_rate'] + input['amenities_fee']
  return total_rate
```

Yo can write and save this function in a separate file, for example `my_formulas.py`. Or alternatively, you can also define it in the same script as your main SDV modeling and sampling.

## Constraint API

Create a `ColumnFormula` constraint.

**Parameters**:&#x20;

* (required) `input_column_names`: A list of strings that describe all the column names that should be inputs into the formula.\
  *In the previous SDV Enterprise version 0.47, this was called `input_columns`.*
* (required) `output_column_name`: A string that describes the final output column name. This is the column that is formulaically determined.\
  *In the previous SDV Enterprise version 0.47, this was called `output_column`.*
* (required) `formula_function_name`: A string that contains the name of the formula function. This should be defined within the Python file specified above.
* `module_name`: A string with the name of the Python module where the formula function is written. (This is typically the name of the Python file without the `.py` suffix.) \
  *In the previous SDV Enterprise version 0.47, this was called `module`.*
  * (default) `"__main__"`: The currently active module. Use this if the function is defined in the same file as the constraint.
* `table_name`: The name of the table that contains the columns. *This is required if you have multi-table data.*

```python
from sdv.cag import ColumnFormula

my_constraint = ColumnFormula(
    input_column_names=['room_rate', 'amenities_fee'],
    output_column_name='total_rate',
    formula_function_name='get_room_total',
    module_name='my_formulas',
)
```

Make sure that all the tables and columns you provide are in your [Metadata](/sdv/integration/metadata).

### Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample(num_rows=10)
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection <a href="#auto-detection" id="auto-detection"></a>

Auto-detection not available for this constraint. Please create the constraint object and add it to the synthesizer using the API described above.

## FAQs

<details>

<summary>Can I define the formula in the same file as my synthesizer and constraint?</summary>

Yes. If your formula is defined in the same file, the module name should be `"__main__"`, which is the keyword that Python uses for the currently active script. This is the default value for the parameter.

```python
my_constraint = ColumnFormula(
    input_column_names=['room_rate', 'amenities_fee'],
    output_column_name='total_rate',
    formula_function_name='get_room_total',
    module_name='__main__',
)
```

</details>

<details>

<summary>If I save a synthesizer with this constraint, can it work in other locations without the formula file?</summary>

Yes. As long as you have added the ColumnFormula constraint and fitted your synthesizer, then the saved synthesizer will always include the formula. You can load the synthesizer file in a different environment and sample synthetic data from it immediately. No need need to import or redefine the formula file.

For example, you can save the fitted synthesizer that contains the ColumnFormula constraint:

```python
synthesizer.save('my_formula_synthesizer.pkl')
```

You can then load the synthesizer later into a different environment and continue sampling synthetic data. This will work even if you're loading it in a different location without the formula file.

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer('my_synthesizer.pkl')
synthetic_data = synthesizer.sample(num_rows=10)
```

</details>

<details>

<summary>Can I define a formula that only applies partially to a subset of the data?</summary>

It's possible to have a formula that only applies to certain rows of your data. For other rows, you can allow SDV to algorithmically generate the data. For example:

* `total_rate = room_rate + amenities_fee` only if the hotel guest is a basic member
* If they are a star member, then the `total_rate` does not follow this logic. SDV can algorithmically generate it.

In this case, you can define a function that takes in the final output column (`total_rate`) as part of the input too. The input will contain the algorithmically generated values from the synthesizer, which may not necessarily match the formula. Use these values as a starting point; fix them for any rows that need it.

```python
def get_room_total(input):
  # the algorithmically-determined total is available in the total_rate column
  # this does not necessarily follow the formula for basic members
  original_rate = input['total_rate'] 

  # calculate the what the rate should be for basic members based on the sum
  membership_mask = (input['membership_type'] == 'BASIC') 
  rate_sum = input['room_rate'] + input['amenities_fee']
  
  # only update the total rate when the member is a basic member
  total_rate = original_rate.where(membership_mask, rate_sum)
  
  return total_rate
```

When defining the constraint, make sure you include the output column in the input column list as well.

```python
# Creating the constraint
from sdv.cag.sandbox import ColumnFormula

my_constraint = ColumnFormula(
    input_column_names=['room_rate', 'amenities_fee', 'total_rate', 'membership_type'],
    output_column_name='total_rate',
    module_name='my_formulas',
    formula_function_name='get_room_total'
)
```

Please be aware that while partial formulas like this are supported, this constraint is not optimized for them. You may notice some impacts to the synthetic data quality of the output column. For troubleshooting support, [please reach out](/sdv/support/troubleshooting/sdv-usage#other-issues) to us. *We are able to provide support for SDV Enterprise customers who have purchased the CAG bundle.*

</details>


# ❖ CompositeKeys

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **CompositeKeys** constraint when multiple columns together form a primary key. Optionally, you may also have multi-column foreign key that connects to the primary key.

{% hint style="warning" %}
***This constraint is deprecated.*** Instead of supplying a CompositeKey constraint, supply composite key information in your metadata. For more information, see the [Metadata API](/sdv/integration/metadata/metadata-api).

```json
{
    "tables": {
        "Patient visits": {
            "primary_key": ["Patient ID", "Date"],
            "columns": {
                "Patient ID": { "sdtype": "id", "regex_format": "P-[0-9]{5}" },
                "Date": { "sdtype": "datetime", "datetime_format": "%Y-%m-%d" },
                ...   
            }
        }
    },
    "relationships": [{
        "parent_table_name": "Patient visits",
        "parent_primary_key": ["Patient ID", "Date"],
        "child_table_name": "Lab results",
        "child_foreign_key": ["Patient ID", "Date"]
    }]
}
```

{% endhint %}

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F3flgr8LKJAK3EtUKOcbg%2Fcomposite-key_May%208%202025.png?alt=media&amp;token=86783b53-5817-405f-9c5f-3715e6eb9ddc" alt=""><figcaption><p>'Patient ID' and 'Date' together form the primary key of the table.</p></figcaption></figure>

## Constraint API

Create a `CompositeKeys` constraint that lists all the composite keys of your database (primary and foreign keys).

**Parameters**:&#x20;

* (required) `primary_keys`: A list of dictionaries that define the composite primary keys of all tables that have composite primary keys. Each dictionary should have the following keys:
  * `'table_name'`: A string with the name of the table
  * `'primary_key'`: A list of strings representing all the columns in the table that form the composite, primary key. At least 1 of these columns should be an sdtype id or PII column.
* `relationships`: A list of dictionaries that define the relationships from composite foreign keys to the composite primary keys defined above. Each dictionary should have the following keys:
  * `'parent_table_name'`: A string with the name of the parent table with the composite primary key (this should be listed in the `primary_keys` section above).
  * `'parent_primary_key'`: A list of columns that comprise the composite primary key of the table (this should be the same as the `primary_keys` section above)
  * `'child_table_name'`: A string with the name of the child table that refers to the parent
  * `'child_foreign_key'`: A list of columns that comprise the composite foreign key that refers to the parent. The length of this list should be the same as the primary key that it refers to.

```python
from sdv.cag import CompositeKeys

my_constraint = CompositeKeys(
    primary_keys=[{
        'table_name': 'Patient Visits',
        'primary_key': ['Patient ID', 'Date']
    }],
    relationships=[{
        'parent_table_name': 'Patient Visits',
        'parent_primary_key': ['Patient ID', 'Date'],
        'child_table_name': 'Test Results',
        'child_foreign_key': ['Patient ID', 'Visit Date']
    }]
)
```

Make sure that all the table and columns in you provide are in your [Metadata](/sdv/integration/metadata). If you provide any foreign key connections, make sure they align with the columns in the primary key.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).


# ❖ DenormalizedTable

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **DenormalizedTable** constraint when you have a table that was created by denormalizing a parent and child table. This happens when you collapse a parent table into the child by repeating its values.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FXjwHhD1qIsYGQG1K9TcJ%2Fcag-bundle-predefined-constraints-denormalized-table_Nov%2024%202025.png?alt=media&amp;token=39afe6ec-983e-4ffe-aaf4-12ac553d0404" alt=""><figcaption><p>In this example, the <code>Transactions-Users</code> table is a denormalized table that contains information about transactions and users. One user can have many transactions. Users have been collapsed into the table by repeating the User ID and Birthday.</p></figcaption></figure>

## Constraint API

Create a `DenormalizedTable` constraint

**Parameters**:&#x20;

* (required) `table_name`: A string containing the name of the denormalized table
* (required) `denormalized_primary_key`: A string containing the name of the column that represents the primary key of the denormalized, parent table. The values in this column are likely repeated as part of the denormalization process.
* (required) `denormalized_column_names`: A list of strings containing the column names that were also a part of the denormalized, parent table. The values in these columns are also likely repeated. They should be consistent for every denormalized primary key value.

```python
from sdv.cag import DenormalizedTable

my_constraint = DenormalizedTable(
    table_name='Transactions-Users',
    denormalized_primary_key='User ID',
    denormalized_column_names=['User Birthdate']
)
```

Make sure that the table and column names you provide are in your [Metadata](/sdv/integration/metadata). The denormalized table may be part of a broader, multi-table dataset.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint and it is enabled by default for multi-table datasets. Run the detection with your data to detect all instances of this constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(data)
```

**Detection Parameters**: None


# ❖ FixedNullCombinations

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

The **FixedNullCombinations** constraint enforces that the combinations between categorical columns are fixed. For other types of columns, it ensures that the combinations of null values are fixed. That is, no other permutations or shuffling is allowed other than what's already observed in the data.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FAVmrjZo1FnzbS5dej5cD%2Fcag-predefined-constraints-fixed-null-combinations-constraint_Dec%2015%202025.png?alt=media&amp;token=6a10974e-5691-468b-8512-0245092de165" alt=""><figcaption><p>The <code>Support Cases</code> table contains entries for each support ticket filed by customers. Only resolved cases have a resolution date — aka the <code>Resolution Status</code> column determines whether or not the <code>Resolution Date</code> can be null.</p></figcaption></figure>

## Constraint API

Create a `FixedNullCombinations` constraint.

**Parameters**:

* (required) `column_names`: A list of two or more columns whose combinations are fixed. These columns can be any sdtype. However, they cannot be listed as primary or foreign keys.
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.
* `fix_category_values`: Whether to fix the values for categorical columns
  * (default) `True`: If there are categorical columns, fix the combinations that appear with their actual value. (For other types of columns, the combinations of null vs. non-null values are fixed instead.)
  * `False`: For all column, only fix the combinations between null vs. non-null values. Allow there to be additional permutations within categorical values.

```python
from sdv.cag import FixedNullCombinations

my_constraint = FixedNullCombinations(
    column_names=['Resolution Status', 'Resolution Date'],
    table_name='Support Cases',
)
```

{% hint style="warning" %}
**Do you want to apply this constraint to PII data?** As part of its algorithm, this constraint will learn and remember some parameters in the synthesizer. While you can always share your synthetic data, we recommend caution when sharing out your synthesizer itself, as it will contain some of these parameters.
{% endhint %}

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint and it is enabled by default. Run the detection with your data to detect all instances of this constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(data)
```

**Detection Parameters**: None

## FAQs

<details>

<summary>What is the difference between FixedCombinations and FixedNullCombinations?</summary>

The [FixedCombinations](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints/fixedcombinations) constraint (available in SDV Community) fixes the combinations allowed within categorical columns. In contrast, the FixedNullCombinations constraints can be applied to other types of data such as numerical, datetime, PII etc. For these types of columns, the constraint cannot fix the actual values, as it  possible to synthesizer new, previously-unseen numerical, datetime, or categorical values. However, it can fix whether a column is null or non-null.&#x20;

The FixedNullCombinations then allows you to cover a large variety of scenarios. For example:

* If several columns need to be null altogether, or not at all
* If there should only be one non-null value within a group of columns
* If the categorical value of one column influences whether another column is null (the example shown throughout this page)

</details>

<details>

<summary>Why can't I apply this constraint to a single column?</summary>

This constraint ensures that the synthetic data only contains null combinations that exist in the real data. If there is only one column, there are no combinations.

The SDV already guarantees that the synthetic data contains a similar proportion of null  values as the real data for a single column.

</details>


# ❖ ForeignToForeignKey

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **ForeignToForeignKey** constraint when you have foreign keys in multiple tables but no primary key to attach them to. This may happen if your tables come from different domains and are linked together by the same concept.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FHaqIFnFYx4WS0F7GscLJ%2Fforeign-to-foreign-key_May%208%202025.png?alt=media&amp;token=b1cfb07c-05f3-45cd-b393-33f3977966fd" alt=""><figcaption><p>In this example, the 'Products' table and the 'Shipments' table both have a column called 'Warehouse ID' that denotes the same concept (storage warehouse). Both these columns are foreign keys, and there is no primary key for Warehouse ID that is available to link them.</p></figcaption></figure>

## Constraint API

Create a `ForeignToForeignKey` constraint.

**Parameters**:&#x20;

* (required) `columns`: A list of dictionaries representing representing all the foreign key columns that are encoding the same concept. Each dictionary should have&#x20;
  * A `'table_name'` key mapping to the string name of the table, and
  * A `'foreign_key'` key mapping to the string name of the column. (If you have a composite key, provide a tuple of multiple strings.)
* `foreign_key_generation`: A string that describes whether the synthetic data for the foreign keys should contain brand new values, or reuse the ones that exist in your database
  * (default) `'new'`: Create new values in the synthetic data. These new values will be consistent everywhere in the database. *In our example above, the synthetic data would have brand-new Warehouse IDs representing new, synthetic warehouses. These warehouses would be consistent between the Products and Suppliers table.*
  * `'reuse'`: Reuse the values from the synthetic data. *In our example above, the synthetic data would have the same set of Warehouse IDs as the real data, within both the Products and Suppliers tables. These would represent the same warehouses.*

```python
from sdv.cag import ForeignToForeignKey

my_constraint = ForeignToForeignKey(
    columns=[{
        'table_name': 'Products',
        'foreign_key': 'Warehouse ID'
    },{
        'table_name': 'Shipments',
        'foreign_key': 'Warehouse ID'
    }],
    foreign_key_generation='new'
)
```

Make sure that all the tables and columns you provide are listed in your [Metadata](/sdv/integration/metadata).

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection not available for this constraint. Please create the constraint object and add it to the synthesizer using the API described above.


# ❖ ForeignToPrimaryKeySubset

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **ForeignToPrimaryKeySubset** constraint when you have a 1-to-many connection between tables, but only certain values are allowed to have connections.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FNDP7hSnEP6enF0xY0cNc%2Fforeign-to-primary-key-subset_May%208%202025.png?alt=media&amp;token=5ef39dc2-e6af-45f0-be1d-854402197bb5" alt=""><figcaption><p>In this example, only premium type accounts are allowed to have perks.</p></figcaption></figure>

## Constraint API

Create a `ForeignToPrimaryKeySubset`  constraint.

**Parameters**:&#x20;

* (required) `parent_table_name`: A string with the name of the parent table in the relationship
* (required) `child_table_name`: A string with the name of the child table in the relationship
* (required) `child_foreign_key`: A string with the name of the foreign key column in the child. This column is only allowed to be connected to the parent table with certain values.
* (required) `conditional_column_name`: A string with the name of the conditional column in the parent table. The values in this column determine which connections are allowed.
* (required) `conditional_values`: A list of values of the conditional column. Only these particular values are allowed to have connections with the child table.

```python
from sdv.cag import ForeignToPrimaryKeySubset

my_constraint = ForeignToPrimaryKeySubset(
    parent_table_name='Accounts',
    child_table_name='Perks',
    child_foreign_key='ID',
    conditional_column_name='Type',
    conditional_values=['PREMIUM'])
```

Make sure that all the tables and columns you provide are listed in your [Metadata](/sdv/integration/metadata), as well as the relationship between the primary and foreign key.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['ForeignToPrimaryKeySubset']
)
```

**Detection Parameters**: None


# ❖ MixedScales

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

The **MixedScales** constraint enforces that the value of a categorical column (or a combination of categorical columns) determines the scale of a numerical column.&#x20;

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F8aVa9YpaUmNk5n0nauG8%2Fmixed-scales_May%208%202025.png?alt=media&amp;token=b0ca97e0-2815-46f7-bea3-b8b61b7c8013" alt=""><figcaption><p>For example, the combined value of categorical columns <code>test_type</code> and <code>units</code> will determine the value of numerical column for <code>test_results</code>. So if test type is <code>'height'</code> and units are <code>'inches'</code> then it forces the synthesizer to learn the scale specially for this segment.</p></figcaption></figure>

## Constraint API

Create a `MixedScales` constraint.

**Parameters**

* (required) `segment_column_names`: A list of one or more categorical columns that ultimately segment the data into different groups of rows. Each group will have a different scale.
* (required) `mixed_scale_column_name`: A numerical column whose scale depends on the segment.
* `table_name`: A string with the name of the table to apply this to. Required if you have a multi-table dataset.

```python
from sdv.cag import MixedScales

my_constraint = MixedScales(
    segment_column_names=['test_type', 'units'],
    'mixed_scale_column_name'='test_results'
)
```

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = GaussianCopulaSynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['MixedScales']
)
```

**Detection Parameters**: None

## FAQs

<details>

<summary>What if I only have one categorical column that determines the scale?</summary>

In this case, please provide a list of just one column name as your `segment_column_names`:

```python
my_constraint = {
    'constraint_class': 'MixeScales',
    'table_name': 'patient_test_results', # for multi table synthesizers
    'constraint_parameters': {
        'segment_column_names': ['test_type'],
        'mixed_scale_column_name': 'test_results'
    }
}
```

</details>

<details>

<summary>Why can't I input the scale min and max for each segment?</summary>

In this constraint, you will only define the column names. SDV synthesizers will use this information to automatically learn the scales (min/max values and distribution) for each different segment of the real data. There is no need to manually input this data.

</details>


# ❖ Polymorphic Relationship

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **PolymorphicRelationship** constraint when the values inside a foreign key column can refer to different tables, depending on the context.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2Fvaabe7cSoun7wpK5u3KP%2Fcag-bundle-predefined-constraints-polymorphic-relationship_Nov%2024%202025.png?alt=media&amp;token=803db8d4-4bba-4467-ad8c-ace131924489" alt=""><figcaption><p>In this example, the <code>Transaction Log</code> table has a foreign key called <code>Details</code>. Sometimes, it refers to the <code>Debit Accounts</code> table and sometimes it refers to the <code>Credit Accounts</code> tables.</p></figcaption></figure>

## Constraint API

Create a `PolymorphicRelationship`  constraint.

**Parameters**:&#x20;

* (required) `table_name`: A string with the name of the table that contains the foreign key column
* (required) `foreign_key`: A string with the name of the foreign key column inside the table
* (required) `parent_table_names`: A list of strings containing the possible different parent tables that the foreign key column could be referring to
* `type_column_name`: Sometimes, the table may have a categorical column that controls which tables the foreign key refers to. In this case, provide a string with the name of this column.
* `type_value_to_table`: If there is a type column, provide a dictionary that maps each categorical value of the column to the table it refers to.

```python
from sdv.cag import PolymorphicRelationship

my_constraint = PolymorphicRelationship(
    table_name='Transaction Log',
    foreign_key='Details',
    parent_table_names=['Debit Accounts', 'Credit Accounts'],
    type_column_name='Type',
    type_value_to_table={
        'CREDIT': 'Credit Accounts',
        'DEBIT': 'Debit Accounts'
    }
)
```

Make sure that all the tables and columns you provide are listed in your [Metadata](/sdv/integration/metadata). The table relationships will not be part of the metadata, as a polymorphic relationship cannot be inputted into the metadata.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['PolymorphicRelationship']
)
```

**Detection Parameters**: None


# ❖ PrimaryToPrimaryKey

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **PrimaryToPrimaryKey** constraint when you have an exact 1-to-1 connection between the primary keys of two or more tables.&#x20;

{% hint style="warning" %}
***This constraint is deprecated.*** Instead of supplying a PrimaryToPrimaryKey constraint, connect primary keys in your metadata via a parent/child relationship. For more information, see the [Metadata API](/sdv/integration/metadata/metadata-api).

```json
"tables": {
    "Users": {
        "primary_key": "ID",
        ...
    },
    "Supplemental Info": {
        "primary_key": "ID",
        ...
    }
},
"relationships": [{
    "parent_table_name": "Users",
    "child_table_name": "Supplemental Info",
    "parent_primary_key": "ID",
    "child_foreign_key": "ID"
}]
```

{% endhint %}

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FH689Czwah3otjfVJHs7r%2Fprimary-to-primary-key_May%208%202025.png?alt=media&amp;token=2ce50b1b-fd79-4fa7-9c99-bb5c80955c14" alt=""><figcaption><p>In this example, the table 'Users' has an exact 1-to-1 relationship with the table 'Supplemental Info'. Both tables are connected via the same primary key ID value.</p></figcaption></figure>

## Constraint API

Create a `PrimaryToPrimaryKey` constraint

**Parameters**:&#x20;

* (required) `table_names`: A list of strings, representing the table names whose primary keys are all connected. You must provide two or more table names.

```python
from sdv.cag import PrimaryToPrimaryKey

my_constraint = PrimaryToPrimaryKey(
    table_names=['Users', 'Supplemental Info'])
```

Make sure that all the tables in you provide are in your [Metadata](/sdv/integration/metadata), and have a primary key associated with them.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).


# ❖ PrimaryToPrimaryKeySubset

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **PrimaryToPrimaryKeySubset** constraint when you have a 1-to-1 connection between the primary keys of two or more tables but only certain values are allowed to have connections.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F6P9OJvn5WwalXrvSDQPO%2Fprimary-to-primary-key-subset_May%208%202025.png?alt=media&amp;token=45bb657c-0bcf-416f-938c-499996d9de40" alt=""><figcaption><p>In this example, only minors are supposed to have parental info</p></figcaption></figure>

## Constraint API

Create a `PrimaryToPrimaryKeySubset` constraint.

**Parameters**:&#x20;

* (required) `main_table_name`: A string with the name of the main table that contains all the possible rows. Only some of these rows are allowed to be connected to other tables.
* (required) `conditional_column_name`: A string with the name of a column in the main table. The values of this column controls whether connections are allowed with other tables.
* (required) `relationships`: A dictionary that maps the name of each connected table with the conditional value that is allowed for the connection.

```python
from sdv.cag import PrimaryToPrimaryKeySubset

my_constraint = PrimaryToPrimaryKeySubset(
    main_table_name='Users',
    conditional_column_name='Is Minor?',
    relationships={
        'Parental Info': [True]
    })
```

Make sure that all the table and columns in you provide are in your [Metadata](/sdv/integration/metadata), and have a primary key associated with them.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['PrimaryToPrimaryKeySubset']
)
```

**Detection Parameters**: None


# ❖ ReferenceTable

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **ReferenceTable** constraint when you have a table that acts as an unchangeable reference. That is to say, you do not want to synthesize an new information in it; the table's contents should always remain as-is.

When you apply this constraint, the reference table's contents remains as-is. At the same time, synthetic data is generated for all other tables, including ones that link to the reference table.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FFa0xI9RamUmT3kelMpEW%2Freference-table_May%208%202025.png?alt=media&amp;token=8d5ea4f9-c807-481b-b9de-5857b9ae333b" alt=""><figcaption><p>In this example, the <code>City</code> table is a reference table. You do not want to synthesize new cities, but you do want to synthesize other tables that link to the cities such as <code>Supplier</code>, <code>Product</code>, etc.</p></figcaption></figure>

## Constraint API

Create a `ReferenceTable` constraint

**Parameters**:&#x20;

* (required) `reference_table_names`: A list of strings representing all the reference tables that are in your schema.&#x20;

```python
from sdv.cag import ReferenceTable

my_constraint = ReferenceTable(
    reference_table_names=['City'])
```

Make sure that all table names you provide are in your [Metadata](/sdv/integration/metadata). Reference tables are not allowed to have any foreign key references to non-reference tables.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Auto-Detection

Auto-detection not available for this constraint. Please create the constraint object and add it to the synthesizer using the API described above.


# ❖ SelfReferentialHierarchy

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **SelfReferentialHierarchy** constraint when you have a column in the table that references the primary key column of the same table (aka a self-reference); and the self-references are not allowed to have any cycles.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2Ft5hb8V1JfCCTx7r1Aw4N%2Fself-referential-hierarchy_May%208%202025.png?alt=media&amp;token=4090efb9-5350-4fd6-b258-f5e2f263534c" alt=""><figcaption><p>In this example, the "Manger ID" column references the primary key, "Employee ID" that is in the same table. This self-reference is not allowed to have cycles, meaning that the manager/employee relationships forms a strict reporting chain all the way up to the CEO. (If person A manages person B, then B cannot manage person A.)</p></figcaption></figure>

## Constraint API

Create a `SelfReferentialHierarchy` constraint.

**Parameters**:&#x20;

* (required) `table_name`: A string with the name of the table that contains the self-reference
* (required) `base_column_name`: A string with the name of the column that is acting as the base for the reference. This may be the primary key of your table. In our example, the base is the "Employee ID".
  * *In older SDV Enterprise versions (0.33.0 and before), this parameter was called `primary_key`.*
* (required) `parent_column_name`: A string with the name of the column that is the parent of the base column. In our example, the parent column is "Manager ID".
  * *In older SDV Enterprise versions (0.33.0 and before) this parameter was called `foreign_key` .*
* `grandparent_column_name`: A string with the name of the column that is the grandparent of the base column (aka the parent's parent). Our example doesn't have a grandparent column, but if it did, it would represent the skip level manager (aka the Manger's Manager).
* `root_column_name`: A string with the name of the ultimately root of the tree. Our example doesn't have a root column, but if it did, it would represent the person at the top of the hierarchy (aka the CEO or organizational lead).
* `scaling_method`: A string with the name of the method, used when scaling up the synthetic data
  * (default) `'branch'`: Keep the original depth of the hierarchy but add more branches to it. *In our example above, this would add more reports for a given manager.*
  * `'depth'`: Add to the depth of the self-references. *In our example above, this would create new levels of managers, increasing the length of the reporting chain to the CEO.*
  * `'multiply'`: Keep the original branching factor and depth to the hierarchy, but create more trees. *In our example above, this would create additional companies with new CEOs and reporting chains.*

```python
from sdv.cag import SelfReferentialHierarchy

my_constraint = SelfReferentialHierarchy(
    table_name='Employees',
    base_column_name='Employee ID',
    parent_column_name='Manager ID'
)
```

Make sure that all the table and columns in you provide are in your [Metadata](/sdv/integration/metadata). *Note that you cannot supply a self-reference relationship in the metadata right now, so the `relationships` section of your Metadata can be blank.*

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).

## Visualize the Hierarchy

A self-referential hierarchy can be visualized as a tree-like dependency structure. For example, each row of the data (an employee) can be visualized as a node in an overall tree, pointing to the manager. The root node (topmost node) represents the CEO, followed by mangers, employees, etc. In this way, it's possible to see the overall branch factor and depth of the tree too.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FU0qCy3DvXRX17pIKBakt%2Fcag-predefined-constraints-self-referential-hierarchy-visualize-the-hierarchy_July%2014%202025.png?alt=media&amp;token=d7a7fb1c-bc2e-469e-be6e-942d7c7074d9" alt=""><figcaption></figcaption></figure>

Create a graphic that corresponds to your data using the `visualize` function.

**Parameters**:

* (required) `data`: A pd.DataFrame object containing the data you want to visualize. The data should match the constraint&#x20;
* `label_nodes`: Toggle whether to label each node of the tree the value from your dataset
  * (default) `True`: Show the value for each node.
  * `False`: Do not label the nodes. The visualization will include blank circles representing the nodes.
* `max_trees`: If there are multiple, separate trees to your hierarchy, use this parameter to control the max number of individual trees you'd like to visualize
  * (default) `None`: Visualize all the data
  * `<integer>`: Only visualize the given # of trees. the remaining rows will not be visualized.
* `max_depth`: If the trees have a very high depth, use this parameter to control the maximum depth you'd like to visualize
  * (default) `None`: Visualize all the data regardless of the depth
  * `<integers>`: Only visualize up to the given depth. Any nodes deeper than this level will not be visualized.
* `output_filepath`: If provided, save the image at the given location in the given format.&#x20;
  * (default) `None`: Do not save the visualization
  * `<filepath>`: A string with the name of the filepath. This must end with the filetype that you want to save as. Popular examples are `png`, `jpg` or `pdf`.

**Output** A [graphviz.graphs.Digraph](https://graphviz.readthedocs.io/en/stable/manual.html) containing the visualization

```python
from sdv.cag import SelfReferentialHierarchy

my_constraint = SelfReferentialHierarchy(
    table_name='Employees',
    base_column_name='Employee ID',
    parent_column_name='Manager ID')

<graph> = my_constraint.visualize(
  my_dataframe,
  label_nodes=True,
  max_trees=2,
  max_depth=3,
  output_filepath='visualizations/my_graph.png'
)
```

## Auto-Detection

Auto-detection is allowed for this constraint but it is not enabled by default.

If you'd like to auto-detect this constraint specifically, you can supply it in the auto-detection parameters. This will detect all instances of the constraint throughout the dataset.

```python
constraints = synthesizer.detect_constraints(
    data,
    constraint_classes=['SelfReferentialHierarchy']
)
```

**Detection Parameters**: None

## FAQs

<details>

<summary>How can the root node (eg. the CEO) be encoded?</summary>

In our example, the root node — the CEO — appears in the table as an employee but they do not have any manager associated with them (the value for their manager is null). This is the recommended format.&#x20;

However, the constraint also supports formats where the root node is encoded in different ways:

* Self-assignment for the root node. For example, the CEO appears as an employee and their manager is listed as themselves.
* No entry for the root node. For example, the CEO does not ever appear as an employee at all. (They only appear as a manager.)

This constraint should match either of these formats. Please make sure that if you have multiple root nodes (eg. multiple CEOs), they are all encoded consistently.

</details>


# ❖ UniqueBridgeTable

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

Use the **UniqueBridgeTable** constraint when you have a bridge table that records a many-to-many relationship between multiple tables, and the connections have to be unique. That is to say, the combinations of keys in the bridge tables must be unique.

{% hint style="warning" %}
***This constraint is deprecated.*** Instead of supplying a UniqueBridgeTable constraint, represesent the bridge table as a composite key in your metadata. For more information, see the [Metadata API](/sdv/integration/metadata/metadata-api).

```json
{
    "tables": {
        "Authors": {
            "primary_key": "ID",
            ...
        },
        "Books": {
            "primary_key": "ID",
            ...
        },
        "Author-Book": {
            "primary_key": ["Author ID", "Book ID"],
            ...
        }
    },
    "relationships": [{
        "parent_table_name": "Authors",
        "parent_primary_key": "ID",
        "child_table_name": "Author-Book",
        "child_foreign_key": "Author ID",
    },{
        "parent_table_name": "Books",
        "parent_primary_key": "ID",
        "child_table_name": "Author-Book",
        "child_foreign_key": "Book ID", 
    }]   
}
```

{% endhint %}

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FeMnrC63ULCD1vkGPnvEw%2Funique-bridge-table_May%208%202025.png?alt=media&amp;token=80f0e5c3-ad64-4b65-991f-10a1ea061686" alt=""><figcaption><p>The Author-Book table records unique links between Authors and Books.</p></figcaption></figure>

## Constraint API

Create a `UniqueBridgeTable` constraint.

**Parameters**:&#x20;

* (required) `table_name`: A string with the name of the bridge table
* (required) `foreign_key_columns`: A list of strings with the names of the foreign key columns. The combination of values in these columns must be unique.

```python
from sdv.cag import UniqueBridgeTable

my_constraint = UniqueBridgeTable(
    table_name='Author-Book',
    foreign_key_columns=['Author ID', 'Book ID'])
```

Make sure that all the tables and columns you provide are listed in your [Metadata](/sdv/integration/metadata), as well as all foreign key connections.

## Usage

Apply the constraint to any SDV synthesizer. Then fit and sample as usual.

```python
synthesizer = HSASynthesizer(metadata)
synthesizer.add_constraints([my_constraint])

synthesizer.fit(data)
synthetic_data = synthesizer.sample()
```

For more information about using predefined constraints, please see the [**Constraint-Augmented Generation tutorial**](https://colab.research.google.com/drive/1WCMQujfVKL5giULZXOPPIBoMfzR9Zj68?usp=sharing).


# Deprecated Constraints

If you were using an older version of SDV, you may have noticed that we used to support a few additional constraints. This includes:

* Constraints that used to enforced min/max boundaries on your data: Positive, Negative, ScalarInequality, and ScalarRange.
* Constraints that used to denote relationships between your data tables: CompositeKeys, PrimaryToPrimaryKey, and UniqueBridgeTable

These constraints are no longer supported in the CAG framework because the same functionality can be achieved using other features within the SDV.

## Recommended Usage

Browse below for the equivalent functionality that we recommend using instead of the deprecated constraints.

**Enforcing min/max boundaries:**

<details>

<summary>I would like my synthetic data to remain in the same min/max range as my original data</summary>

By default, SDV synthesizers learn the min/max values of your data, and ensure the synthetic data adheres to the same ranges. So there is nothing more you need to do to achieve this.

</details>

<details>

<summary>I would like to perform statistical preprocessing on my data (such as the log or logit function)</summary>

The log or logit functions mathematically impose min or max boundaries on your data. To use these functions, we recommend disabling the synthesizer's min/max enforcement. Then, you can add statistical preprocessing for individual columns.

```python
# disable the synthesizer's min/max enforcement
my_synthesizer = GaussianCopulaSynthesizer(
    metadata,
    enforce_min_max_values=False)

# add preprocessing for individual columns such as 'age' and 'salary'
my_synthesizer.auto_assign_transformers(my_data)

from rdt.transformers.numerical import LogScaler, LogitScaler

my_synthesizer.update_transformers({
    'age': LogScaler()
    'salary': LogitScaler(
        min_value=0 - 1e-10,
        max_value=1000000
    )
})
```

For more information about custom preprocessing, please see our [tutorial](https://colab.research.google.com/drive/1Z9Rff5G3eDI-LEc1LXiew9E3hF5Ob-on?usp=sharing) and [API docs](https://docs.sdv.dev/sdv/single-table-data/modeling/customizations/preprocessing). Also see the docs the [LogScaler](https://docs.sdv.dev/rdt/transformers-glossary/numerical/logscaler) and [LogitScaler](https://docs.sdv.dev/rdt/transformers-glossary/numerical/logitscaler).

</details>

<details>

<summary>I would like to control the range of my synthetic data independently of what's in my original data</summary>

The SDV software is primarily meant to learn ranges from the original data and replicate those same patterns as-is in the real data. However, we offer a few features that can help you control the ranges:

* **Further restricting the range of the synthetic data:** We recommend letting your synthesizer learn the original ranges. Then, use [conditional sampling](https://docs.sdv.dev/sdv/single-table-data/sampling/conditional-sampling) to get the exact values of synthetic data that you need.

```python
# create and fit an SDV synthesizer with the default settings
my_synthesizer = GaussianCopulaSynthesizer(metadata)
my_synthesizer.fit(my_data)

# use conditional sampling to limit the synthetic data
# in this case, we'll only create ages that are 23-25
from sdv.sampling import Condition

age_23 = Condition(num_rows=4, column_values={'age': 23})
age_24 = Condition(num_rows=4, column_values={'age': 24})
age_25 = Condition(num_rows=4, column_values={'age': 25})

synthetic_data = my_synthesizer.sample_from_conditions([age_23, age_24, age_25])
```

* **Expanding the range of the synthetic data:** Expanding the synthetic data range (beyond the the original data) is not yet fully supported within the SDV. The best approach would be to include additional rows in the original data to encompass the full range. That way, SDV can learn the full range.\
  If that is not possible, you can try disabling the synthesizer's min/max range enforcement. Depending on the synthesizer's algorithm, you may be able to achieve a slightly broader range this way (no guarantees).

```python
# disable the synthesizer's min/max enforcement
my_synthesizer = CTGANSynthesizer(
    metadata,
    enforce_min_max_values=False)
my_synthesizer.fit(my_data)

# now the synthetic data may go outside the ranges
# but this is not yet fully supported
synthetic_data = my_synthesizer.sample(num_rows=500)
```

</details>

**Denoting relationships in your data:**&#x20;

<details>

<summary>My schema contains composite keys for the primary/foreign key connections.</summary>

You can now supply composite primary and foreign keys directly in your metadata by providing a list instead of an individual column name. For example, a table may have a composite primary key that comprises of two columns, `Patient ID` and `Date`; both columns together act as the primary key.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F3flgr8LKJAK3EtUKOcbg%2Fcomposite-key_May%208%202025.png?alt=media&amp;token=86783b53-5817-405f-9c5f-3715e6eb9ddc" alt=""><figcaption></figcaption></figure>

You can set both columns as the primary key in the metadata.

```python
metadata.set_primary_key(
    table_name='Patient visits',
    column_name=['Patient ID', 'Date']
)
```

You can also supply a composite foreign key that points to a composite primary key. Make sure that the foreign key has the same number of columns and that the order of the columns you provide line up with the values they are referencing.

```python
metadata.add_relationship(
    parent_table_name='Patient visits',
    parent_primary_key=['Patient ID', 'Date'],
    child_table_name='Lab results',
    child_foreign_key=['Patient ID', 'Date']
)
```

For more information, see the [Metadata Guide](/sdv/integration/metadata).

</details>

<details>

<summary>I want to enforce uniqueness in a bridge table</summary>

A bridge table that records a many-to-many relationship between multiple tables. To enforce that the connections are unique, you can denote a composite key for the bridge table in your metadata.&#x20;

For example let's say you have a bridge table `Author-Book` that connects the `Authors` and `Books` table.&#x20;

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FeMnrC63ULCD1vkGPnvEw%2Funique-bridge-table_May%208%202025.png?alt=media&amp;token=80f0e5c3-ad64-4b65-991f-10a1ea061686" alt=""><figcaption></figcaption></figure>

To enforce that the connections are unique, set a composite key for the `Author-Book` table.

```python
metadata.set_primary_key(
    table_name='Author-Book',
    column_name=['Author ID', 'Book ID']
)
```

The individual columns in the bridge table can still be foreign keys into other tables.

```python
metadata.add_relationship(
    parent_table_name='Authors',
    parent_primary_key='ID',
    child_table_name='Author-Book',
    child_foreign_key='Author ID'
)

metadata.add_relationship(
    parent_table_name='Books',
    parent_primary_key='ID',
    child_table_name='Author-Book',
    child_foreign_key='Book ID'
)
```

For more information, see the [Metadata Guide](/sdv/integration/metadata).

</details>

<details>

<summary>My schema contains two connected primary keys</summary>

If your schema contains a relationship between two primary keys, that means you have a 1-to-1 connection between two tables. You can denote this in your metadata by adding a relationship. In the relationship, one of the primary key columns acts as a foreign key to another table. Typically the foreign key will be the table that contains extra or auxiliary information.

For example, you may have a table for `Users` and another table for their `Supplemental Info` that is connected in a 1-to-1 fashion.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FH689Czwah3otjfVJHs7r%2Fprimary-to-primary-key_May%208%202025.png?alt=media&amp;token=2ce50b1b-fd79-4fa7-9c99-bb5c80955c14" alt=""><figcaption></figcaption></figure>

In this case, we add a foreign key reference from the `Supplemental Info` table (auxiliary table) to the main `Users` table.

```python
 metadata.set_primary_key(
  table_name='Users',
  column_name='ID'
)

metadata.set_primary_key(
  table_name='Supplemental Info',
  column_name='ID'
)

metadata.add_relationship(
  parent_table_name='Users',
  parent_primary_key='ID',
  child_table_name='Supplemental Info',
  child_foreign_key='ID'
)
```

For more information, see the [Metadata Guide](/sdv/integration/metadata).

</details>

## Older Functionality

If you are using an older version of the SDV (version 1.21 or earlier), we've provided a PDF below containing the deprecated constraints API, as well as the API for custom constraints. However, please note that **we recommend upgrading to the latest version of SDV as soon as you can**. The latest SDV version contains bug fixes and additional features that you may need. Our team is only able to provide questions/debugging support for the latest SDV version.

{% hint style="success" %}
**We're here to help you upgrade.** If you need any help, please contact us via our [forum](https://forum.datacebo.com/).
{% endhint %}

{% file src="/files/Z8vNmIcxUGLcZ58bpDu7" %}

{% file src="/files/ywH9EmvPGBDrowkSjh9M" %}


# Constraints API

Apply constraints to any SDV synthesizer.

## Constraints API

The constraint API below applies to any SDV synthesizer object.

### get\_constraints

Use this function to inspect all of the constraints your synthesizer contains and save them externally as a JSON file.

**Parameters**:

* `output_filepath`: A string containing the location of a filepath. If provided, this function saves the constraints that are currently added to the synthesizer as a JSON file. Later, these constraints can be set using the [`set_constraints` function](#set_constraints).
  * (default) `None`: Do not save the constraints in a file.

**Output** A list of the constraint objects that you have supplied to your synthesizer. If the filepath is provided, then the function also saves the constraints as a JSON file.

```python
constraints = synthesizer.get_constraints(
    output_filepath='synthesizer_constraints.json'
)
```

### get\_metadata

Adding constraints to your synthesizer may internally modify the metadata. Use this function to get the modified metadata after the constraints have been added.

**Parameters**

* `version`: The version of metadata to get. Supply `'modified'` to get the modified version of the metadata (after applying the constraints).

**Output** A Metadata object with the modified metadata

```python
modified_metadata = synthesizer.get_metadata(version='modified')
```

### add\_constraints

Use this function to add constraints to your synthesizer.

**Parameters**

* (required) `constraints`: A list of constraint objects or an auto-detected `ConstraintList` object to add to your synthesizer. You can supply [**predefined constraints**](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints) or [**auto-detected constraints**](/sdv/modeling/constraint-augmented-generation-cag/auto-detect-constraints). Call this function multiple times to continue adding constraints.

**Output** (None)

```python
# create predefined constraints and add them to the synthesizer
from sdv.cag import FixedIncrements

salary_constraint = FixedIncrements(
    table_name='Users',
    column_name='salary',
    increment_value=1000
)

synthesizer.add_constraints([salary_constraint])

# auto-detect constraints and add them to the synthesizer

detected_constraints = synthesizer.detect_constraints(data)
synthesizer.add_constraints(
    constraints=detected_constraints
)
```

### load\_constraints

Use this function to load constraints that were previously saved to a file using the [`get_constraints` function](#get_constraints).

**Parameters**:

* (required) `filepath`: A string with the JSON filepath that contains all the constraint information. This file can be created using the [`get_constraints` function](#get_constraints).

**Returns:** A list of constraint objects. You can add this list directly to your synthesizer using the [`add_constraints` function](#add_constraints).

```python
from sdv.utils import load_constraints

constraints = load_constraints(filepath='synthesizer_constraints.json')
synthesizer.add_constraints(constraints)
```

### set\_constraints

*Deprecated. To add constraints from a file, use* [*load\_constraints*](#load_constraints) *to load in the constraints from the file, and then* [*add\_constraints*](#add_constraints) *to add them.*

### ❖ detect\_constraints

Use this method with an SDV synthesizer in order to detect constraints. *All single- and multi-table SDV synthesizers are supported except for DayZSynthesizer.*

```python
synthesizer = HSASynthesizer(metadata)
constraints = synthesizer.detect_constraints(data=my_datset)
```

*For API and parameters, see the* [*Auto-Detect Constraints*](/sdv/modeling/constraint-augmented-generation-cag/auto-detect-constraints) *guide.*

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **CAG Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [CAG Bundle](/sdv/explore/sdv-bundles/cag) page.
{% endhint %}

## JSON Representation

Each constraint can be represented in a JSON file format. This is useful when you want to save all the constraints to be used for a synthesizer, and add them all at once for your workflow.

To create this file format, we recommend adding the constraints to a synthesizer first and then using the [`get_constraints` function](#get_constraints). This function renders the JSON file in the required format. Alternatively, you can follow the format below.

```json
[{
    "class_name": "DenormalizedTable",
    "parameters": {
        "table_name": "Transactions-Users",
        "denormalized_primary_key": "User ID",
        "denormalized_column_names": ["User Birthdate"]
    }
},{
    "class_name": "FixedIncrements",
    "parameters": {
        "table_name": "Users",
        "column_name": "salary",
        "increment_value": 1000
    }
},{
    ...
}]
```

### JSON Spec Details

The constraints JSON file should be a list of dictionaries. Each dictionary represents a constraint that should be added to the synthesizer. The constraints should be listed in the order in which they should be added.

Each constraint dictionary should include 2 keywords:

* `"class_name"`: A string containing the name of the single-table or multi-table constraint class. This is the class name that you'd typically import from the `sdv.cag` module.
* `"parameters"`: A dictionary that contains key/value pairs that correspond to the parameters of that constraint. Each constraint has different parameters that are possible. They typically include information that identifies where in your dataset the constraint should be added, for example the table name, column names, etc. For more information, browse the [Predefined Constraints](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints).


# Program Your Own Constraint

If the predefined constraints don't meet your needs, you can program your own constraint with the logic that you need.

**Compatibility:** Any type of column *except* for primary and foreign keys

{% hint style="danger" %}
**Programming your own constraint is a last resort.** Programming a constraint requires you to specify and maintain your own logic. Our team is not able to offer debugging support for your own code.

In many cases, it's possible to achieve your result more easily with existing constraints. Please browse the [**Predefined Constraints list**](/sdv/modeling/constraint-augmented-generation-cag/predefined-constraints) to see if your logic is captured by the list.

If you have any questions, please [reach out to us](/sdv/support/troubleshooting/sdv-usage#other-issues) and we'll be happy to point you in the right direction.
{% endhint %}

## How to Program Your Own Constraint

In most cases, your business rules can be achieved by pre-processing your data before using SDV, and post-processing the data after getting the synthetic data. Programmable constraints allow you to input that pre- and post-processing logic directly into your synthesizer, so you do not have to worry about maintaining a separate script.

Learn how to program your own constraint using the tutorial below.

{% embed url="<https://colab.research.google.com/drive/1BvUGkgYaf2ohMISqWZGrVTHgR1dwaOkV?usp=sharing>" %}

Download the example constraint code used in the tutorial using the link below.

{% file src="/files/MzNjSsOdAUb1rCK7hotF" %}

## Programmable Constraints Template

Below, you will find a template code for programming a single table constraint (to be used within a single table of your dataset) and a multi-table constraint (to be used within different tables of your dataset). Select the file you need and fill out the relevant code.

{% file src="/files/iYUeGtfTT9DPl4sODa32" %}

{% file src="/files/v3jDEhgASp1NcV6XQFsQ" %}

## FAQs

<details>

<summary>Which types of columns can I use within my constraint?</summary>

You are free to use any type of column for a custom constraint *except* for primary and foreign keys.

Primary and foreign keys pay a special role in identifying rows and relationships. Their original values are needed for all our synthesizers to work — especially multi-table synthesizers that learn patterns between the relationships.

</details>

<details>

<summary>The constraint crashes. How should I fix the logic?</summary>

You can debug your custom constraint by creating the constraint object and trying to run the different functions — `transform`, `reverse_transform`, `is_valid`, etc.

```python
my_constraint = MyExampleConstraint()

transformed_data = my_constraint.transform(original_data)
print(transformed_data)

reversed_transformed_data = my_constraint.reverse_transform(transformed_data)
print(reverse_transformed_data)

validity = my_constraint.is_valid(original_data)
print(validity)
```

Ensuring that there are no crashes and printing out these results may help you understand where the logic is failing.

***Unfortunately, the SDV team is only able to offer individualized support for customers who have purchased the CAG bundle.** For these customers, we offer help with debugging and may prioritize creating a new, predefined constraint related to your logic. To learn more, visit our page about* [*SDV Enterprise features*](/sdv/explore/compare-features)*.*

</details>

<details>

<summary>Can I use my constraint to generate data from scratch (DayZSynthesizer)?</summary>

If you are an SDV Enterprise user, you have access to the [DayZSynthesizer](/sdv/modeling/single-table-synthesizers/dayzsynthesizer) which can be used to generate data from scratch.

Starting from SDV Enterprise version 0.30.0, you'll be able to add single-table constraints to DayZSynthesizer, including your own, programmable single-table constraint. To use this make sure you fill out the 2 functions that are listed as "REQUIRED FOR DayZSynthesizer" in the template. These are: `is_valid` and `fix_data`. All other functions are optional.

</details>


# Configuration


# ❖ Range Extrapolation

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **Targeted Sampling** bundle, an optional add-on to SDV Enterprise. For more information, please visit the [Targeted Sampling](/sdv/explore/sdv-bundles/targeted-sampling) page.
{% endhint %}

By default, your synthesizers learns the patterns based on the training data you provide, including the possible ranges within each column. The **range extrapolation** feature allows you to create synthetic data that is **outside the ranges of your training data**.

## Usage

This feature is meant for extrapolating outside of the ranges that your training data contains. This covers:

* Extrapolating the possible min and max for continuous attributes like numerical or datetime columns
* Creating brand new category values that were not present in your original training data
* Allowing null (or non-null) values in a column even when your training data doesn't have this scenario covered

### Specify Ranges in the Metadata

Make sure that your metadata object contains the full ranges that you want to specify. This includes the min/max values of ranges, full lists of category values, and more.&#x20;

For more information, see the [Metadata API](/sdv/integration/metadata).

### Use Your Synthesizer as Usual

The range extrapolation will automatically be applied when you create a synthesizer with the metadata.

```python
from sdv.multi_table import HSASynthesizer

synthesizer = HSASynthesizer(metadata)
```

## What's Next?

Be sure to fit your syntheizer for the extrapolation to work.

```python
my_synthesizer.fit(data)
```

After fitting, you can sample synthetic data as usual. The synthetic data will now contain the full ranges that you specified in your dictionary as opposed to the ranges from the training data.

```python
synthetic_data = my_synthesizer.sample(scale=10)
```

Note that you may need to sample many data points in order to see the extrapolated values. Alternatively, you can request the extrapolated values explicitly. For example, say your training data only included users aged 25-60 but your range extrapolation extends the ranges from 18-100. You can then request synthesizing any age between 18 and 100.

```python
from sdv.sampling import Condition

older_users = Condition(
    num_rows=10,
    table_name='users',
    column_values={'age': '90'})

younger_users = Condition(
    num_rows=10,
    table_name='users',
    column_values={'age': '18'})

synthesizer.sample_from_conditions([older_users, younger_users])
```

For more information, see the API docs for [conditional sampling](/sdv/sampling/conditional).


# Flexible Preprocessing

All synthesizers pre-process your data to prepare it for machine learning and then post-process synthetic data to convert it into the original format. This advanced feature allows you to control the transformations that are applied to each column.

{% hint style="success" %}
**Do you have any sensitive data?** The transformations are also used for anonymizing or pseudo-anonymizing sensitive data.
{% endhint %}

## Viewing the Transformations

The SDV will pre and post-process your data by using reversible data transformations for each column. The transformers are available in our [RDT library](https://docs.sdv.dev/rdt).

The methods below allow you to see which transformations will be applied to each column based on the synthesizer and data you plan to use.

### auto\_assign\_transformers

Let the synthesizer auto assign the transformations based on the data you'd like to use for modeling.

**Parameters**

* (required) `data`: Your original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.

**Output** (None)

```python
synthesizer.auto_assign_transformers(data)
```

### get\_transformers

After assigning the transformers, you can get more details about the transformers that will be used on each column.

**Parameters**

* (required for multi table) `table_name`: A string with the table name you'd like to see the transformer for

**Output** A dictionary mapping each column name to an [RDT transformer](https://docs.sdv.dev/rdt/transformers-glossary/numerical) that will pre and post-process your data.&#x20;

```python
synthesizer.get_transformers(
    table_name='guests'
)
```

```python
{
 'guest_email': AnonymizedFaker(provider_name='internet', function_name='email', enforce_uniqueness=True),
 'hotel_id': None,
 'has_rewards': LabelEncoder(add_noise=True),
 'room_type': None,
 'amenities_fee': FloatFormatter(learn_rounding_scheme=True, enforce_min_max_values=True),
 'checkin_date': UnixTimestampEncoder(datetime_format='%d %b %Y'),
 'checkout_date': UnixTimestampEncoder(datetime_format='%d %b %Y'),
 'room_rate': FloatFormatter(learn_rounding_scheme=True, enforce_min_max_values=True),
 'billing_address': AnonymizedFaker(provider_name='address', function_name='address'),
 'credit_card_number': AnonymizedFaker(provider_name='credit_card', function_name='credit_card_number', enforce_uniqueness=True)
}
```

## Modifying the Transformers

After assigning transformers, you can also modify them.&#x20;

### update\_transformers

Use this method to reassign new transformers that you'd like to use for specific columns. You can apply any transformer in the [RDT Library](https://docs.sdv.dev/rdt/transformers-glossary/numerical) to your data.

**Parameters**

* (required for multi table) `table_name`: A string representing the table name yo'd like to update the transformer for
* (required) `column_name_to_transformer`: A dictionary mapping the column name to the new RDT transformer object that you'd like to use for pre and post processing.

**Output** (None)

```python
from rdt.transformers import FloatFormatter

synthesizer.update_transformers(
    table_name='guests',
    column_name_to_transformer={
        'amenities_fee': FloatFormatter(missing_value_replacement=0.0)
    }
)
```

{% hint style="warning" %}
**Be careful with this step!** Make sure you are updating transformers that are compatible with each column's sdtype and make sense for the underlying machine learning model.
{% endhint %}

### ＊  update\_transformers\_by\_sdtype

Use this method to reassign new transformers for all columns of a specific sdtype. For example, reassign the processing for all numerical or categorical columns. You can apply any transformer in the [RDT Library](https://docs.sdv.dev/rdt/transformers-glossary/numerical) to your data.

{% hint style="info" %}
**＊SDV Enterprise Feature.** This feature is only available for licensed, enterprise users. For more information, visit our page to [Compare SDV Features](/sdv/explore/compare-features).
{% endhint %}

**Parameters**

* (required) `sdtype`: A string with the name of the sdtype that you want to change the transformers for
* (required) `transformer_name`: A string with the name of the transformer to use.
* `transformer_parameters`: A dictionary that maps the name of the transformer parameter (string) to the parameter value. Use this if you want to override the default settings.
  * (default) `None`: Do not override the default settings
* `table_names`: A list of strings describing the specific table names to apply the transformers to. This is applicable only for multi table data.
  * (default) `None`: Update the transformers for all tables
  * `<list>`: Only update the transformers for the given table names

```python
synthesizer.update_transformers_by_sdtype(
    sdtype='numerical',
    transformer_name='FloatFormatter',
    transformer_parameters={
        'missing_value_replacement': 'random',
        'missing_value_generation': 'from_column',
    },
    table_names=['guests'])
```

### Anonymizing Sensitive Data

Use the [AnonymizedFaker](https://docs.sdv.dev/rdt/transformers-glossary/browse-transformers/anonymizedfaker) or [PsuedoAnonymizedFaker](https://docs.sdv.dev/rdt/transformers-glossary/browse-transformers/pseudoanonymizedfaker) transformers for sensitive data.

```python
from rdt.transformers import PseudoAnonymizedFaker 

synthesizer.update_transformers(column_name_to_transformer={
    'credit_card_number': PsuedoAnonymizedFaker(provider_name='credit_card', function_name='credit_card_number'),
})
```

{% hint style="info" %}
**Anonymization vs. Pseudo-anonymization**

Pseudo-anonymization preserves a mapping between the real, sensitive values and the fake, synthetic data. Use this if you'd like to trace back the synthetic data to real values.

Anonymization is not reversible. Anyone with access to the fake, synthetic data will not be able to it back to any value of real data.&#x20;
{% endhint %}

## Applying the Transformations

After modifying the transformations, you can apply them to the real data in a step-wise fashion or all at once.

### preprocess

Use this function to preprocess the data according to the transformations. After preprocessing, you should have numerical data that is ready for modeling.

```python
processed_data = synthesizer.preprocess(data)
```

**Parameters**

* (required) `data`: Your original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.

**Output** A preprocessed version of the real data. Based on the transformations, some columns may be converted, added or removed. The output type is the same as the data input.

### fit\_processed\_data

Use this function to perform model training on preprocessed data. Note that this step may take a while, as it requires machine learning to learn trends from your the dataset.

```python
synthesizer.fit_processed_data(processed_data)
```

**Parameters**

* (required) `processed_data`: The processed data that is ready for machine learning. This is the output of the `preprocess` method, in the form of a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.

**Output** (None). This steps trains the model using machine learning. After this step, you are ready to create synthetic data. For more details, see [Sampling](/sdv/sampling/sample-realistic-data).

### fit

Use this function to perform the preprocessing and modeling training all in one step. Internally, this uses both the `preprocess` and `fit_processed_data` functions in succession.

```python
synthesizer.fit(real_data)
```

**Parameters**

* (required) `data`: Your original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.

**Output** (None) After this step, you are ready to create synthetic data. For more details, see [Sampling](/sdv/sampling/sample-realistic-data).

## FAQ

<details>

<summary>Why am I seeing column names that are different from my original data?</summary>

If you have added constraints, then the SDV may add or delete columns to accommodate the business logic before it goes through preprocessing.

</details>

<details>

<summary>Can I remove a transformer if I don't want any processing?</summary>

In some cases, your data may already be processed such that the synthesizer does not need to perform additional transformations. You can remove transformers by assigning them to `None`.

```python
synthesizer.update_transformers(
  table_name='hotels',
  column_name_to_transformer={
    'rating': None
  })
```

</details>

<details>

<summary>Why are some of the transformers <code>None</code>?</summary>

Based on the data you're using, the synthesizer may determine that no transformations are necessary for machine learning. In this case, you will see that the column is not assigned to `None`, meaning that the data will not undergo any transformation for the machine learning model.

</details>


# Sample Realistic Data

Create realistic synthetic data data that follows the same format and mathematical properties as the real data. In the multi-table setting, you can also scale up (or down) the sizes of all tables in your dataset.

{% hint style="info" %}
**Sampling prerequisites.** In order to create synthetic data, make sure you have created a synthesizer and trained it on your data using the `fit` function. For more information, see the [Modeling](/sdv/modeling/single-table-synthesizers) docs.
{% endhint %}

### sample

Use this function to create synthetic data that mimics the real data. This function is available for single- and multi-table synthesizers. For sequential syntheizers, please see the sample\_sequences function below.

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(filepath='my_synthesizer.pkl')

synthetic_data = synthesizer.sample(num_rows=100)
```

**Parameters**:

* (required) `num_rows`: An integer >0 that specifies the number of rows to synthesize
* (required for multi-table) `main_table_name`: A string containing the name of the main table from the dataset. This is require for multi-table datasets. In this case, the `num_rows` you specify will apply to the main table, and all the other table will be scaled up or down algorithmically in relation to this table.
* `batch_size`: An integer >0, describing the number of rows to sample at a time. If you are sampling a large number of rows, setting a smaller batch size allows you to see and save incremental progress.&#x20;
  * (default) Same as `num_rows`.
* `max_tries_per_batch`: An integer >0, describing the number of sampling attempts to make per batch. If you have included constraints, it may take multiple batches to create valid data.
  * (default)  `100`.
* `output_folder_path`: A string describing the name of a folder for writing the synthetic data. If provided, the synthetic data will be written as CSV files inside the folder. Each CSV file will correspond to a table, named as `<table_name>.csv`.
  * (deafult) `None`: Do not write the synthetic data to files.

### sample\_scale

Use this function to create synthetic data that is scaled up or down for multi-table datasets. This function is only available for multi-table synthesizers.

```python
from sdv.utils import load_synthesizer

synthesizer = load_synthesizer(filepath='my_synthesizer.pkl')

synthetic_data = synthesizer.sample_scale(scale=2.5)
```

**Parameters**:

* (required) `scale`: A float >0.0 that describes how much to scale the data by
  * (default) `1`: Don't scale the data. The model will create synthetic data that is roughly the same size as the original data.
  * `>1`: Scale the data by the specified factor. For example, `2.5` will create synthetic data that is roughly  2.5x the size of the original data.
  * `<1`: Shrink the data by the specified percentage. For example, `0.9` will create synthetic data that is roughly 90% of the size of the original data.

**Returns:** Synthetic data. The synthetic data is the same format as the original data, which is dictionary mapping the table name to a pandas DataFrame containing the data for it.

{% hint style="info" %}
**How does scaling work for multi-table data?** During the fitting process, your SDV synthesizer learns the size of each data table. This is assumed to be a scale of 1. Scaling the entire dataset up or down means that the size of each table will change proportionally based on the original data size.&#x20;

Note that some synthesizers may perform small, additional algorithmic calculations to determine the final size of each table. However, you can still expect the final synthetic data to approximately follow the scale of the real data (with some minor deviations).
{% endhint %}

### sample\_sequences

Use this function to create synthetic data for sequential data. This function is only available for sequential synthesizers.

**Parameters:**

* (required) `num_sequences`: An integer >0 describing the number of sequences to sample
* `sequence_length`: An integer >0 describing the length of each sequence. If you provide `None`, the synthesizer will determine the lengths algorithmically, and the length may be different for each sequence. Defaults to `None`.

### reset\_sampling

Use this function to reset any randomization in sampling. After calling this, your synthesizer will generate the same data as before. For example in the code below, `synthetic_data1` and `synthetic_data2` are the same.

```python
synthesizer.reset_sampling()
synthetic_data1 = synthesizer.sample(num_rows=100)

synthesizer.reset_sampling()
synthetic_data2 = synthesizer.sample(num_rows=100)
```

**Parameters** None

**Returns** None. Resets the synthesizer.


# Conditional Sampling

Do you have exact values that you'd like to include in the synthetic data? Using **conditional sampling** to provide this information. Conditional sampling allows you to target the exact data you need, while still preserving correlations with other, related variables.

Use cases include:

* Generating hypothetical scenarios by fixing the values to correspond to extreme cases
* De-biasing your data by requesting an equal balance of labels
* Imputing unknown data by requesting the data that you already know&#x20;


# Single Table Conditional Sampling

Conditional sampling allows you to fix any values you'd like to see in the synthetic data. The rest of the synthetic data adapts to the values you provide.

```python
from sdv.sampling import Condition

# Step 1: Create Your Conditions
suite_guests_with_rewards = Condition(
    num_rows=250,
    column_values={'room_type': 'SUITE', 'has_rewards': True}
)

# Step 2: Sample Synthetic Data
synthetic_data = synthesizer.sample_from_conditions([suite_guests_with_rewards])
```

Follow the 2-step workflow below to create your conditions and then ask SDV to sample with them.

{% hint style="info" %}
**Sampling prerequisites.** In order to create synthetic data, make sure you have created a synthesizer and trained it on your data using the `fit` function. For more information, see the [Modeling](/sdv/modeling/single-table-synthesizers) docs.
{% endhint %}

## Step 1: Create Your Conditions

Use conditions to specify which exact, fixed values you'd like to appear in your synthetic data. SDV creates synthetic data with the fixed value and creates other variables based on it. The other variables will contain the same patterns and correlations with respect to the fixed value.&#x20;

You can create as many conditions as you need to fully describe the synthetic data that you'd like to create.

### Condition

Create a `Condition` object to specify exact values you want to include in the synthetic data.

*In this example, we want to create 250 guests staying in suites with rewards, and an additional 250 guests staying in suites without rewards.*

```python
from sdv.sampling import Condition

suite_guests_with_rewards = Condition(
    num_rows=250,
    column_values={'room_type': 'SUITE', 'has_rewards': True}
)

suite_guests_without_rewards = Condition(
    num_rows=250,
    column_values={'room_type': 'SUITE', 'has_rewards': False}
)
```

**Parameters**

* (required) `num_rows`: The number of rows that need to be included in the scenario
* (required) `column_values`: A dictionary with the scenario. The keys should be a column names and the values should be the exact data that the column should have. *You can fix any columns, as long as they are not primary or foreign keys.*

{% hint style="info" %}
**You may require multiple conditions.** Define as many Condition objects as you need to construct your hypothetical scenario. For example, you may want to create a 50/50 mix of active an inactive users of various tiers.
{% endhint %}

### DataFrameCondition

Create a `DataFrameCondition` if you already have a DataFrame of fixed-value columns that you'd like to include in the synthetic data. This is a shortcut you can use instead of creating multiple `Condition` objects.

```python
import pandas as pd
from sdv.sampling import DataFrameCondition

my_fixed_datapoints = pd.DataFrame(data={
    'room_type': ['SUITE', 'SUITE', 'SUITE', 'SUITE', 'SUITE'],
    'has_rewards': [True, True, True, False, False]
})

type_rewards_condition = DataFrameCondition(dataframe=my_fixed_datapoints)
```

**Parameters**:

* (required) `dataframe`: A pandas.DataFrame object containing the columns whose values you want to fix

## Step 2: Sample Synthetic Data

Once you have the synthetic data, you can use any single-table SDV synthesizer to sample conditions. Your synthesizer must have already been trained (fit) on the data.

### sample\_from\_conditions

Use this function to create synthetic data based on the conditions.

```python
synthetic_data = synthesizer.sample_from_conditions(
    conditions=[suite_guests_with_rewards, suite_guests_without_rewards, type_rewards_condition],
    output_folder_path='synthetic_data/scenarios/'
)
```

**Parameters**

* (required) `conditions`: A list of Condition or DataFrameCondition objects that specify the exact values that you want to fix.
* `batch_size`: An integer >0, describing the number of rows to sample at a time. If you are sampling a large number of rows, setting a smaller batch size allows you to see and save incremental progress.
  * (default) The same as the total number of rows
* `max_tries_per_batch`: An integer >0, describing the number of sampling attempts to make per batch. If you have included constraints, it may take multiple batches to create valid data.&#x20;
  * (default) `100`
* `output_folder_path`: A string describing a CSV folder for writing the synthetic data. The data will be written inside this folder as `<table_name>.csv`.
  * (default) `None`: Skip writing to a file

**Returns** A [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) object with synthetic data. The synthetic data is simulated based on the conditions.

{% hint style="success" %}
**The quality of your synthetic data is preserved.** When you provide conditions, the SDV synthesizers will match your conditions and create the remaining data based on the patterns that it learned. Your synthetic data will continue to have the same statistical patterns between columns.
{% endhint %}

## Troubleshooting

{% hint style="warning" %}
**Conditional sampling is a complex feature.** In some cases, your synthesizer may not be able to create all rows of synthetic data that you request. Let's walk through some areas that you can investigate.
{% endhint %}

#### Which synthesizer are you using?

The SDV synthesizers have different conditional sampling capabilities. If you are using the CTGAN, TVAE or CopulaGAN synthesizers, the SDV may unable to complete your conditional sampling request in some instances.&#x20;

Neural network-based synthesizers use a *reject sampling* approach: They sample synthetic data freely, keep the rows that match your conditions and repeat the process as needed. This may not be efficient if the conditional values are extremely rare.

Some suggestions:

* Use a larger `batch_size` or `max_tries_per_batch`. The more rare your conditions, the more attempts the SDV will have to make.
* Try using the [GaussianCopulaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) instead. This synthesizer can sample conditions mathematically instead of reject sampling, which is more efficient.

#### Are you including constraints?

Any synthesizer that has [constraints](/sdv/modeling/constraint-augmented-generation-cag) may have to use *reject sampling* to ensure the rows are valid. This can slow down the process.

Suggestions:&#x20;

* Use a larger `batch_size` or `max_tries_per_batch`. The more constraints you have, the more attempts the SDV will have to make.
* Consider removing constraints and refitting your synthesizer. This will help if you are conditioning on desired columns that were involved in a constraint.

#### Are you requesting data that is out of range?

The SDV synthesizers are designed to learn patterns from the input data that you've added during `fit`, including the the min and max ranges of each column. If you are requesting conditional values that are outside of the ranges, the synthesizer may not be able to accommodate your request.

Check to see whether you values at out of range. If so, you may have more success if you fit your synthesizer without enforcing min max values. For more details, see the [Modeling API](/sdv/modeling/single-table-synthesizers).

```python
synthesizer = GaussianCopulaSynthesizer(
    metadata,
    enforce_min_max_values=False
)
```

#### Need more help?

If you encounter any problems, please [visit our forum](https://forum.datacebo.com/). You can browse existing issues to see if there's a solution. If you cannot find what you're looking for, create an account and start new thread.


# ❖ Multi Table Conditional Sampling

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **Targeted Sampling** bundle, an optional add-on to SDV Enterprise. For more information, please visit the [Targeted Sampling](/sdv/explore/sdv-bundles/targeted-sampling) page.
{% endhint %}

Conditional sampling allows you to fix any values you'd like to see in the synthetic data. The rest of the synthetic data adapts to the values you provide. In a multi-table setting, you can fix the values in one or more tables. *This feature is currently available for* [*HSA*](/sdv/modeling/multi-table-synthesizers/hsasynthesizer) *and* [*Independent*](/sdv/modeling/multi-table-synthesizers/independentsynthesizer) *Synthesizers.*

{% hint style="success" %}
**The SDV Guarantee.** SDV will always sample synthetic data for all tables in your dataset. Based on your conditions, it will connect tables with the appropriate values, and provide synthetic data with referential integrity.
{% endhint %}

```python
from sdv.sampling import Condition, MultiTableCondition

# Step 1: Create Single-Table Conditions
resort_hotels = Condition(
    num_rows=10,
    table_name='hotels',
    column_values={'classification': 'RESORT'})

suite_guests_with_rewards = Condition(
    table_name='guests'
    column_values={'room_type': 'SUITE', 'has_rewards': True})

# Step 2: Compose Multi-Table Conditions    
suites_in_resorts = MultiTableCondition(
    conditions=[resort_hotels, suite_guests_with_rewards])

# Step 3: Sample Synthetic Data
synthetic_data = synthesizer.sample_from_conditions([suite_guests_with_rewards])
```

Follow the 3-step workflow below to create your conditions and then ask SDV to sample with them.

{% hint style="info" %}
**Sampling prerequisites.** In order to create synthetic data, make sure you have created a synthesizer and trained it on your data using the `fit` function. For more information, see the [Modeling](/sdv/modeling/single-table-synthesizers) docs.
{% endhint %}

## Step 1: Create Single-Table Conditions

Use conditions to specify which exact, fixed values you'd like to appear in each individual table of your synthetic data. SDV creates synthetic tables with the fixed values and creates other variables (and tables) based on it. The other values will contain the same patterns and correlations with respect to the fixed value.

You can create as many conditions as you need to fully describe the synthetic data that you'd like to create. Create them for 1 more more different tables in your dataset.

### Condition

Create a `Condition` object to specify exact values you want to include in the synthetic data for a specific table.

*In this example, we want to create 10 resort hotels, and some guests that are staying in suites.*

```python
from sdv.sampling import Condition

resort_hotels = Condition(
    num_rows=10,
    table_name='hotels',
    column_values={'classification': 'RESORT'})

suite_guests_with_rewards = Condition(
    table_name='guests'
    column_values={'room_type': 'SUITE', 'has_rewards': True})
```

**Parameters**

* (required) `table_name`: A string with the name of the table for which you'd like to fix the values
* `column_values`: A dictionary with the values to fix. The keys should be a column names and the values should be the exact data that the column should have. You can fix any columns, as long as they are not primary or foreign keys. *In a multi-table setting, column values may not be needed. It may the be case that you want to only specify a number of rows (`num_rows`) rather than the column values.*
* `num_rows`: The number of rows you would like to create for the table. *In a multi-table setting, this may not be needed. SDV is able to determine the number of rows for a table based on other tables that is is connected to. For example, if your original data had roughly 20 guests for every hotel, then SDV can automatically determine there should be 200 guests for 10 hotels.*

{% hint style="info" %}
**You may require multiple conditions.** Define as many Condition objects as you need to construct your hypothetical scenario. For example, you may want to create a 50/50 mix of active an inactive users of various tiers.
{% endhint %}

### DataFrameCondition

Create a `DataFrameCondition` if you already have a DataFrame of fixed-value columns that you'd like to include in the synthetic data. This is a shortcut you can use instead of creating multiple `Condition` objects.

```python
import pandas as pd
from sdv.sampling import DataFrameCondition

my_fixed_datapoints = pd.DataFrame(
    table_name='guests',
    data={
    'room_type': ['SUITE', 'SUITE', 'SUITE', 'SUITE', 'SUITE'],
    'has_rewards': [True, True, True, False, False]
    })

type_rewards_condition = DataFrameCondition(dataframe=my_fixed_datapoints)
```

**Parameters**:

* (required) `table_name`: A string with the name of the table for which you'd like to fix values
* (required) `dataframe`: A pandas.DataFrame object containing the columns whose values you want to fix

## Step 2: Compose Multi-Table Conditions

Creating multi-table conditions tells SDV to ensure that the individual tables are connected to each other. Without this step, SDV can still create individual tables with the conditions, but they may not be connected to each other.

For example in our case, we may want to ensure that the suite rewards guests are all staying in resort hotels. We can use the multi-table condition to ensure this connection will exist.

```python
from sdv.sampling import MultiTableCondition
 
suites_in_resorts = MultiTableCondition(
    conditions=[resort_hotels, suite_guests_with_rewards])
```

**Parameters**:

* (required) `conditions`: A list of the single-table Condition or DataFrameCondition objects. SDV ensures that these tables are connected to each other.

{% hint style="info" %}
**What are the sizes of the tables?** For at least one of the tables, the conditions must define the number of rows. SDV can then algorithmically determine the number of rows to generate for each of the other tables.
{% endhint %}

## Step 3: Sample Synthetic Data

Once you have the synthetic data, you can use any single-table SDV synthesizer to sample conditions. Your synthesizer must have already been trained (fit) on the data.

### sample\_from\_conditions

Use this function to create synthetic data based on the conditions.

```python
synthetic_data = synthesizer.sample_from_conditions(
    conditions=[suite_guests_with_rewards, suite_guests_without_rewards, type_rewards_condition],
    output_folder_path='synthetic_data/scenarios/'
)
```

**Parameters**

* (required) `conditions`: A list of Condition, DataFrameCondition, or MultiTableCondition objects that specify the exact values that you want to fix.
* `batch_size`: An integer >0, describing the number of rows to sample at a time. If you are sampling a large number of rows, setting a smaller batch size allows you to see and save incremental progress.
  * (default) The total number of rows
* `max_tries_per_batch`: An integer >0, describing the number of sampling attempts to make per batch. If you have included constraints, it may take multiple batches to create valid data.
  * (default) `100`
* `output_folder_path`: A string describing a CSV folder for writing the synthetic data. The data will be written inside this folder as `<table_name>.csv` for each table.
  * (default) `None`: Skip writing to a file

**Returns** A [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) object with synthetic data. The synthetic data is simulated based on the conditions.

{% hint style="success" %}
**The quality of your synthetic data is preserved.** When you provide conditions, the SDV synthesizers will match your conditions and create the remaining data based on the patterns that it learned. Your synthetic data will continue to have the same statistical patterns between columns.
{% endhint %}

## Troubleshooting

{% hint style="warning" %}
**Conditional sampling is a complex feature.** In some cases, your synthesizer may not be able to create all rows of synthetic data that you request. Let's walk through some areas that you can investigate.
{% endhint %}

#### Which synthesizer are you using?

The SDV synthesizers have different conditional sampling capabilities. Under-the-hood, HSA and Independent Synthesizers are set to use a single-table synthesizer for each individual table. If you are using the CTGAN, TVAE or CopulaGAN synthesizers, the SDV may unable to complete your conditional sampling request in some instances.&#x20;

Neural network-based synthesizers use a *reject sampling* approach: They sample synthetic data freely, keep the rows that match your conditions and repeat the process as needed. This may not be efficient if the conditional values are extremely rare.

Some suggestions:

* Use a larger `batch_size` or `max_tries_per_batch`. The more rare your conditions, the more attempts the SDV will have to make.
* Try using the [GaussianCopulaSynthesizer](/sdv/modeling/single-table-synthesizers/gaussiancopulasynthesizer) instead. This synthesizer can sample conditions mathematically instead of reject sampling, which is more efficient.

#### Are you including constraints?

Any synthesizer that has [constraints](/sdv/modeling/constraint-augmented-generation-cag) may have to use *reject sampling* to ensure the rows are valid. This can slow down the process.

Suggestions:&#x20;

* Use a larger `batch_size` or `max_tries_per_batch`. The more constraints you have, the more attempts the SDV will have to make.
* Consider removing constraints and refitting your synthesizer. This will help if you are conditioning on desired columns that were involved in a constraint.

#### Are you requesting data that is out of range?

The SDV synthesizers are designed to learn patterns from the input data that you've added during `fit`, including the the min and max ranges of each column. If you are requesting conditional values that are outside of the ranges, the synthesizer may not be able to accommodate your request.

Check to see whether you values at out of range. If so, you may have more success if you fit your synthesizer without enforcing min max values. For more details, see the [Modeling API](/sdv/modeling/multi-table-synthesizers).

```python
synthesizer = HSASynthesizer(metadata)

synthesizer.set_table_parameters(
    table_name='guests',
    table_synthesizer='GaussianCopulaSynthesizer',
    table_parameters={
        'enforce_min_max_values': False
    }
)
```

#### Need more help?

If you encounter any problems, please [visit our forum](https://forum.datacebo.com/). You can browse existing issues to see if there's a solution. If you cannot find what you're looking for, create an account and start new thread.


# Sequential Conditional Sampling

Conditional sampling allows you to fix any values you'd like to see in the synthetic data. The rest of the synthetic data adapts to the values you provide. For sequential data, you can fix the *context* columns that remain static for the duration of the sequence.

## Condition on Known Context

{% hint style="info" %}
**Sampling prerequisites.** In order to create synthetic data, make sure you have created a synthesizer and trained it on your data using the `fit` function. For more information, see the [Modeling](/sdv/modeling/single-table-synthesizers) docs.
{% endhint %}

### sample\_sequential\_columns

Use this function to sample the sequences based on known context columns that do not change.

**Parameters**

* (required) `context_columns`: A [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) that contains the sequence key and all the context columns of your data that do not vary with respect to time. Each row corresponds to a sequence that you want to synthesize.
* `sequence_length`: An integer >0 describing the length of each sequence. If you provide `None`, the synthesizer will determine the lengths algorithmically, and the length may be different for each sequence. Defaults to `None`.

**Returns** A [pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) object with synthetic data. The synthetic data is based on the referenced, context columns.


# Exporting Synthetic Data

Export your synthetic data back into a similar format as the original data or into a new format using the functions describe on this page.

## ❖ Export to a Database

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **AI Connectors Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [AI Connectors Bundle](/sdv/explore/sdv-bundles/ai-connectors) page.
{% endhint %}

Write your synthetic data into a database to use for downstream applications. Many users choose to write to a database in a lower environment for testing and development. You can also append the synthetic data to an existing database.

We recommending using the same AI Connector you used for importing training data. This connector already knows your original database schema, so it will create the same schema when exporting synthetic data. Be sure to set your export config, so the connector knows where to write the synthetic data.

For more information, see [Connect Your Database](/sdv/integration/db) docs.&#x20;

```python
connector.set_export_config(...)
connector.export(data=synthetic_data, mode='write')
```

{% hint style="info" %}
**Exporting to a different type of database?** Create a new connector based on the type of database. Then, set its export config and complete the export.
{% endhint %}

## Write Locally

We recommend using the same data handler that you used for importing training data. This writes multiple files corresponding to each table you have synthesized. For more information, see the [Import Local Data](/sdv/integration/local) docs.

```python
connector.write(
  synthetic_data,
  folder_name='project/synthetic_data',
  to_csv_parameters={
      'encoding': 'latin-1',
      'index': False
  }
  mode='x')
)
```

{% hint style="info" %}
**Write to a different type of file?** Create a new handler based on the desired file format and then perform the write.&#x20;
{% endhint %}

{% hint style="info" %}
**Write other file formats directly from pandas.** The SDV uses the [pandas library](https://pandas.pydata.org/) for data manipulation and synthesizing. Pandas offers native support for many other types of files. For more information, see the [pandas reference](https://pandas.pydata.org/docs/reference/io.html).

If you have a multi-table dataset, be sure to iterate through all the tables of your multi-table dataset. Pandas typically writes one table at a time.

```python
guests_table = synthetic_data['guests']
guests_table.to_parquet('guests.parquet')

hotels_table = synthetic_data['hotels']
hotels_table.to_parquet('hotels.parquet')
```

{% endhint %}


# Diagnostic

Diagnostic checks are meant to validate the basic data format and validity of the synthetic data. Run through these to make sure that the basics are met. We expect the score to be a perfect, 100%.

## Diagnostic Report

We recommend running the Diagnostic Report to run all basic checks for data format and validity on your synthetic data at once.

### run\_diagnostic

Use this function to run a diagnostic on the synthetic data.

```python
from sdv.evaluation import run_diagnostic

diagnostic_report = run_diagnostic(
    real_data=real_data,
    synthetic_data=synthetic_data,
    metadata=metadata,
    constraints=constraints_list)
```

```
Generating report ...

(1/4) Evaluating Data Validity: |██████████| 15/15 [00:00<00:00, 603.69it/s]|
Data Validity Score: 100.0%

(2/4) Evaluating Data Structure: |██████████| 2/2 [00:00<00:00, 151.49it/s]|
Data Structure Score: 100.0%

(3/4) Evaluating Relationship Validity: |██████████| 1/1 [00:00<00:00, 68.51it/s]|
Relationship Validity Score: 100.0%

(4/4) Evaluating Constraint Validity: |██████████| 2/2 [00:00<00:00, 45.62it/s]|
Constraint Validity Score: 100.0%

Overall Score (Average): 100.0%
```

**Parameters**:

* (required) `real_data`: The original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.
* (required) `synthetic_data`: Your synthetic data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.
* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object with your metadata
* `constraints`: A list of dictionaries representing the business rules in your dataset. For more information, see [Constraint-Augmented Generation](/sdv/modeling/constraint-augmented-generation-cag). *In most cases, this constraints list should be the result of calling the `synthesizer.get_constraints()` after you have fitted. See the* [*Constraints API*](/sdv/modeling/constraint-augmented-generation-cag/constraints-api#get_constraints)*.*
* `verbose`: A boolean describing whether or not to print the report progress and results. Defaults to `True`. Set this to `False` to run the report silently.

**Returns**: An [SDMetrics DiagnosticReport](https://docs.sdv.dev/sdmetrics/data-metrics/diagnostic/diagnostic-report) object generated with your real and synthetic data

### Interpreting the Score

{% hint style="success" %}
**The score should be 100%.** The diagnostic report checks for basic data validity and data structure issues. You should expect the score to be perfect for any of the default SDV synthesizers.
{% endhint %}

The basic diagnostic checks are summarized in the table below.

<table><thead><tr><th width="158">Property</th><th>Description</th></tr></thead><tbody><tr><td>Data Validity</td><td><p>Basic validity checks for each of the columns:</p><ol><li>Primary keys must always be unique and non-null</li><li>Continuous values in the synthetic data must adhere to the min/max range in the real data</li><li>Discrete values in the synthetic data must adhere to the same categories as the real data.</li><li>ID columns must adhere to the provided Regex format.</li><li>Datetime columns must adhere to the provided datetime format.</li></ol></td></tr><tr><td>Constraint Validity</td><td>Checks to ensure that the synthetic data adheres to all of the relevant business logic that you provide. Business logic is provided as contraints.</td></tr><tr><td>Structure</td><td>Checks to ensure the real and synthetic data have the same column names</td></tr><tr><td>Relationship Validity</td><td><p><em>For multi-table data only.</em> Basic validity checks for each relationship between a parent table and a child table:</p><ol><li>Each primary key in the parent table must have an appropriate number of children (i.e. cardinality) based on the min/max of the real data.</li><li>Each foreign key in the child table must reference a primary key that exists in the parent (i.e. referential integrity).</li></ol></td></tr></tbody></table>

### get\_details

This function returns details about the report's properties. Use it to pinpoint the exact columns or tables that are causing issues.

**Parameters**:

* (required) `property_name`: A string with the name of the property. One of: `'Data Validity'`, `'Constraint Validity'`, `'Structure'`, or `'Relationship Validity'`
* `table_name`: For multi table data, a string with the name of the table. If provided, you'll receive filtered results for the table.

**Returns** A pandas.DataFrame object with the detailed scores

```python
diagnostic_report.get_details(property_name='Data Validity')
```

```python
Table     Column	        Metric                   Score
guests    guest_email     KeyUniqueness            1.0
guests    had_rewards     CategoryAdherence	       1.0
guests    room_type       CategoryAdherence	       1.0
guests    amenities_fee   BoundaryAdherence	       1.0
```

## Utilities

### print\_referential\_integrity

Use this function to manually check for referential integrity across a few randomly-selected values from the synthetic data. This is useful for demo purposes.

*To run a full referential integrity check for all values across all relationships, we recommend running the* [*Diagnostic Report*](#diagnostic-report) *instead.*

```python
from sdv.evaluation.utils import print_referential_integrity

print_referential_integrity(
  metadata=metadata,
  synthetic_data=synthetic_data,
  table_name='transaction',
  foreign_key_name='user_id',
  num_rows=3
)
```

```
Picking random transaction row: 1468732365
✅ Found user row! user_id: 86-25-19730100-15 

Picking random transaction row: 1988317645
✅ Found user row! user_id: 0-35-79964375-26 

Picking random transaction row: 1507984147
❌ Unable to find the linked user row
```

**Parameters**:

* (required) `metadata`: The [metadata](/sdv/integration/metadata) object that corresponds to your data schema.
* (required) `synthetic_data`: A dictionary that maps each table name to a pandas.DataFrame containing the synthetic data for it
* (required) `table_name`: A string containing the table name that has the foreign key to check
* (required)  `foreign_key_name`: A string with the column of the foreign key to check. For composite keys, provide a tuple of strings.
* `num_rows`: An int containing the number of columns to check; defaults to 10.

**Returns**: None. The function chooses a few foreign key values random from the synthetic data and checks to make sure the references are found in the parent table. It prints out the result for each column.


# Data Quality

Quality checks compare the statistical similarity between the real and synthetic data. Run through these to evaluate whether the synthetic data captures patterns from the real data. We don't expect the scores to be a perfect, 100%, but the results can point you towards the strengths and weaknesses of your synthesizer.

## Quality Report

The Quality Report checks for statistical similarity between the real and the synthetic data for all the tables. Use this to discover which patterns the synthetic data has captured from the real data

### evaluate\_quality

Use this function to evaluate the quality of your synthetic data.

```python
from sdv.evaluation import evaluate_quality

quality_report = evaluate_quality(
    real_data=real_data,
    synthetic_data=synthetic_data,
    metadata=metadata)
```

```
Generating report ...

(1/4) Evaluating Column Shapes: |██████████| 15/15 [00:00<00:00, 564.15it/s]|
Column Shapes Score: 85.61%

(2/4) Evaluating Column Pair Trends: |██████████| 55/55 [00:00<00:00, 110.40it/s]|
Column Pair Trends Score: 71.97%

(3/4) Evaluating Cardinality: |██████████| 1/1 [00:00<00:00, 53.27it/s]|
Cardinality Score: 70.0%

(4/4) Evaluating Intertable Trends: |██████████| 50/50 [00:00<00:00, 86.54it/s]|
Intertable Trends Score: 68.49%

Overall Score (Average): 74.02%
```

**Parameters**:

* (required) `real_data`: The original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.
* (required) `synthetic_data`: Your synthetic data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data.
* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object with your metadata
* `verbose`: A boolean describing whether or not to print the report progress and results. Defaults to `True`. Set this to `False` to run the report silently.

**Returns**: An [SDMetrics QualityReport](https://docs.sdv.dev/sdmetrics/data-metrics/quality/quality-report) object generated with your real and synthetic data

### Interpreting the Score

{% hint style="success" %}
**Your score will vary from 0% to 100%.** This value tells you how similar the synthetic data is to the real data.

* A 100% score means that the patterns are exactly the same. For example, if you compared the real data with itself (identity), the score would be 100%.
* A 0% score means the patterns are as different as can be. This would entail that the synthetic data purposefully contains anti-patterns that are opposite from the real data.
* Any score in the middle can be interpreted along this scale. For example, a score of 80% means that the synthetic data is about 80% similar to the real data — about 80% of the trends are similar.

The quality score is expected to vary, and **you may never achieve exactly 100% quality**. That's ok! The SDV synthesizers are designed to estimate patterns, meaning that they may smoothen, extrapolate, or noise certain parts of the data.  For more information, see the [FAQs](#faqs).
{% endhint %}

The different types of data quality are summarized in the table below.

<table><thead><tr><th width="158">Property</th><th>Description</th></tr></thead><tbody><tr><td>Column Shapes</td><td>The statistical similarity between the real and synthetic data for single columns of data. This is often called the <em>marginal distribution</em> of each column.</td></tr><tr><td>Column Pair Trends</td><td>The statistical similarity between the real and synthetic data for pairs of columns (within the same table). This is often called the <em>correlation</em> or <em>bivariate distributions</em> of the columns.</td></tr><tr><td>Cardinality</td><td><em>For multi-table data only.</em> Within each parent/child relationship, the cardinality refers to the number of children that each parent has.</td></tr><tr><td>Intertable Trends</td><td><em>For multi-table data only.</em> This is similar to column pair trends, but instead refers to columns between different tables. For example a column between a parent table and a different column in a child table.</td></tr></tbody></table>

### get\_details

This function returns details about the report's properties. Use it to pinpoint the exact columns or tables that are causing issues.

**Parameters**:

* (required) `property_name`: A string with the name of the property. One of: `'Column Shapes'`, `'Column Pair Trends'`, `'Cardinality'` or `'Intertable Trends'`.
* `table_name`: For multi table data, a string with the name of the table. If provided, you'll receive filtered results for the table.

**Returns** A pandas.DataFrame object with the detailed scores

```python
quality_report.get_details(property_name='Column Shapes', table_name='guests')
```

```python
Table        Column            Metric             Score
guests       amenities_fee     KSComplement       0.921127
guests       checkin_date      KSComplement       0.926000
...    
```

### save

Use this function to save the report object

{% hint style="success" %}
The report **does not save the full real and synthetic datasets.** But we still recommend using caution when deciding when to store the report and who to share it with. It does save the metadata along with the score for each property, breakdown and metric.
{% endhint %}

**Parameters**:

* (required) `filepath`: The name of file to save the object. This must end with `.pkl`

**Returns** (None) Saves the report as a file

```python
quality_report.save(filepath='results/quality_report.pkl')
```

### QualityReport.load

Use this function to load in a previously-saved quality report.

**Parameters**:

* (required) `filepath`: The name of the file where the report is stored

**Returns** An [SDMetrics QualityReport](https://docs.sdv.dev/sdmetrics/reports/quality-report/multi-table-api) object

```python
from sdmetrics.reports import QualityReport

quality_report = QualityReport.load('results/quality_report.pkl')
```

## FAQs

See the [SDMetrics QualityReport](https://docs.sdv.dev/sdmetrics/reports/quality-report/whats-included) for even more details about the metrics and properties included in the report.

<details>

<summary>What can I do to improve the quality score?</summary>

We recommend using the report to get more detailed insight.

1. Identify which properties have low score
2. Use the [`get_details`](#get_details) method for those properties to identify which particular data columns or tables have the lowest scores.
3. If possible, [visualize](/sdv/evaluation/visualization) the data to see how the synthetic data compares to the real data.

Using this information, you can update parameters or the data processing steps for the relevant columns. Refer to the [API docs](/sdv/modeling/single-table-synthesizers) corresponding to your synthesizer and check the available [configurations](/sdv/modeling/configuration).

*Note that it's ok — and even expected — to have a quality score that is not exactly 100%. Many of our users find that the synthetic data is still effective for downstream use.*

</details>

<details>

<summary>If my score is very high, does that mean the synthetic data will have high utility?</summary>

A high score indicates a high level of statistical similarity between the real and the synthetic data in terms of the properties we've tested (column shapes and column pair trends). This is a proxy of the overall utility the synthetic data may have for your project, but it is not a guarantee.

The only way to capture true data utility is to use your synthetic data for its intended purpose (downstream application). We recommend trying this as soon as possible, iterating to improve your synthetic data.

If you need help with this, please contact us via our [forum](https://forum.datacebo.com/).

</details>

<details>

<summary>This reports checks for patterns in 1 and 2-dimensions. Why not higher dimensions?</summary>

Higher order distributions of 3 or more columns are not included in the Quality Report. We have found that very high order similarity may have an adverse effect on the synthetic data. After a certain point, it indicates that the synthetic data is just a copy of the real data. (For more information, see the [NewRowSynthesis](https://docs.sdv.dev/sdmetrics/metrics/metrics-glossary/newrowsynthesis) metric.)&#x20;

If higher order similarity is a requirement, you likely have a targeted use case for synthetic data (eg. machine learning efficacy). Until we add these reports, you may want to explore other metrics in the [SDMetrics library](https://docs.sdv.dev/sdmetrics/metrics/metrics-glossary). You may also want to try directly using your synthetic data for the downstream application.

</details>


# Visualization

Use these functions to visualize your actual data in 1 or 2-dimensional space. This can help you see what kind of patterns the synthetic data has learned, and identify differences between the real and synthetic data.

### get\_column\_plot

Use this function to visualize a real column against the same synthetic column. You can plot any column of type: `boolean`, `categorical`, `datetime` or `numerical`.&#x20;

```python
from sdv.evaluation import get_column_plot

fig = get_column_plot(
    real_data=real_data,
    synthetic_data=synthetic_data,
    metadata=metadata,
    table_name='guests',
    column_name='amenities_fee'
)
    
fig.show()
```

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2Flw30XrnR62mvfM4WO8ej%2FColumn%20Comparison.png?alt=media&amp;token=d000af60-1580-4e57-9fe5-2b5ea9b135d9" alt=""><figcaption></figcaption></figure>

**Parameters**

* (required) `real_data`: The original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data. (To skip plotting the real data, input `None`.)
* (required) `synthetic_data`: Your synthetic data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data. (To skip plotting the synthetic data, input `None`.)
* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object that describes the columns
* (required for multi table data) `table_name`: The name of the table
* (required) `column_name`: The name of the column you want to plot
* `plot_type`: The type of plot to create
  * (default) `None`: Determine an appropriate plot type based on your data type, as specified in the metadata.
  * `'bar'`: Plot the data as distinct bar graphs
  * `'displot'`: Plot the data as a smooth, continuous curves

**Output** A [plotly Figure](https://plotly.com/python-api-reference/generated/plotly.graph_objects.Figure.html) object that plots the distribution. This will change based on the sdtype.

{% hint style="info" %}
Use `fig.show()` to see the plot in an iPython notebook. The plot is interactive, allowing you to zoom, scroll and take screenshots.
{% endhint %}

### get\_column\_pair\_plot

Use this utility to visualize the trends between a pair of columns for real and synthetic data. You can plot any 2 columns of type: `categorical`, `ordinal`, `numerical`, `datetime` or `boolean`. The columns do not have to the be the same type.

```python
from sdv.evaluation import get_column_pair_plot

fig = get_column_pair_plot(
    real_data=real_data,
    synthetic_data=synthetic_data,
    metadata=metadata,
    table_name='guests',
    column_names=['room_rate', 'room_type'],
    )
    
fig.show()
```

**Parameters**

* (required) `real_data`: The original training data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data. (To skip plotting the real data, input `None`.)
* (required) `synthetic_data`: Your synthetic data, a dictionary mapping each table name (string) to a pandas.DataFrame containing the data. (To skip plotting the synthetic data, input `None`.)
* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object that describes the columns
* (required for multi table data) `table_name`: The name of the table
* (required) `column_names`: A list with the names of the 2 columns you want to plot
* `plot_type`: The type of plot to create
  * (default) `None`: Determine an appropriate plot type based on your data type, as specified in the metadata.
  * `'box'`: Create a box plot showing the quartiles, broken down by different attributes
  * `'violin'`: Create a violin plot to show distributions, broken down by different attributes. This is an alternative to using `'box'`
  * `'heatmap'`: Create a side-by-side heatmap showing the frequency of each pair of values
  * `'scatter'`: Create a scatter plot that plots each point on a 2D axis
* `sample_size`: The number of data points to plot
  * (default) `None`: Plot all the data points
  * `<integer>`: Subsample rows from both the real and synthetic data before plotting. Use this if you have a lot of data points.

**Output** A [plotly Figure](https://plotly.com/python-api-reference/generated/plotly.graph_objects.Figure.html) object that plots the 2D distribution. This will change based on the sdtype.

{% hint style="info" %}
Use `fig.show()` to see the plot in an iPython notebook. The plot is interactive, allowing you to zoom, scroll and take screenshots.
{% endhint %}

### get\_cardinality\_plot

Use this utility to visualize the cardinality of a multi-table relationship. The *cardinality* refers to the number of child rows that each parent row has. This could be 0 or more.

```python
# for multi table data
from sdv.evaluation import get_cardinality_plot

fig = get_cardinality_plot(
    real_data=real_data,
    synthetic_data=synthetic_data,
    child_table_name='sessions',
    parent_table_name='users',
    child_foreign_key='user_id',
    metadata=metadata)
    
fig.show()
```

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2F4FQ8QOTjqBMXA62p8ic0%2FVisualization%20Cardinality.png?alt=media&amp;token=7223cc0a-ee92-48a9-ab2b-ffcbea90ff29" alt=""><figcaption></figcaption></figure>

**Parameters**

* (required) `real_data`: The original training data. This is a dictionary mapping each table name (string) to a pandas.DataFrame containing the data. (To skip plotting the real data, input `None`.)
* (required) `synthetic_data`: Your synthetic data. This is a dictionary mapping each table name (string) to a pandas.DataFrame containing the data. (To skip plotting the synthetic data, input `None`.)
* (required) `child_table_name`: A string describing the name of the child table in the relationship
* (required) `parent_table_name`: A string describing the name of the parent table in the relationship
* (required) `child_foreign_key`: A string describing the name of the foreign key column of the child table that references the parent table
* (required) `metadata`:  A [Metadata](/sdv/integration/metadata) object that describes the data

**Output** A [plotly Figure](https://plotly.com/python-api-reference/generated/plotly.graph_objects.Figure.html) object that plots the cardinality of the real vs. the synthetic data for the provided relationship.

{% hint style="info" %}
Use `fig.show()` to see the plot in an iPython notebook. The plot is interactive, allowing you to zoom, scroll and take screenshots.
{% endhint %}


# Privacy

Evaluate the privacy of your data and synthesizer before sharing it with others.

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="https://docs.sdv.dev/sdmetrics/metrics/privacy-metrics"><strong>Compare Real vs. Synthetic data</strong></a></td><td>Apply any of the privacy metrics from our SDMetrics library to compare the real and synthetic data. SDMetrics measures disclosure protection, distance to closest record, and more.</td><td><a href="https://docs.sdv.dev/sdmetrics/metrics/privacy-metrics">https://docs.sdv.dev/sdmetrics/metrics/privacy-metrics</a></td></tr><tr><td>❖ <a href="/sdv/evaluation/privacy/empirical-differential-privacy"><strong>Empirical Differential Privacy</strong></a></td><td>Empirically calculate the amount of privacy that your synthesizer algorithm provides. Compare different synthesizer algorithms to see their tradeoffs.</td><td></td></tr></tbody></table>

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **Differential Privacy Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [Differential Privacy Bundle](/sdv/explore/sdv-bundles/differential-privacy) page.
{% endhint %}


# Empirical Differential Privacy

{% hint style="info" %}
❖ **SDV Enterprise Bundle**. This feature is available as part of the **Differential Privacy Bundle**, an optional add-on to SDV Enterprise. For more information, please visit the [Differential Privacy Bundle](/sdv/explore/sdv-bundles/differential-privacy) page.
{% endhint %}

[Differential privacy](/sdv/explore/sdv-bundles/differential-privacy) is a mathematically-rigorous framework that you can use to create private synthetic data. Using our evaluation tool, you can empirically verify the differential privacy that a synthesizer algorithm is offering for a dataset.

## How does it work?

In the differential privacy setup, we are interested in measuring the impact that 1 row of training data has on the overall parameters that a synthesizer learns. Depending on the synthesizer's exact algorithm, the parameters may not be easily accessible or interpretable. Instead, we can create synthetic data using the synthesizer and assume that the patterns exhibited by the synthetic data reflect the parameters.

&#x20;Our evaluation setup creates multiple synthesizers:

* First, we train a synthesizer on all of the real, training data,
* Then, we remove a single row of training data, and use it to train a new synthesizer

We can compare the synthetic data that the synthesizers produce. An algorithm with *high differential privacy* will produce similar synthetic data despite the removal of a row — no matter which row is removed.

<figure><img src="https://1967107441-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfNxEeZzl9uFiJ4Zf4BRZ%2Fuploads%2FpCUsVpB83aZlEZtSzoTB%2FDP-evaluation-docs-graphic_May%2015%202025.png?alt=media&amp;token=44028476-50fd-410e-91ae-89042d5473f6" alt=""><figcaption><p>The differential privacy setup measure the effect that 1 row of training data has on the synthesizer's parameters. We proxy the synthesizer's parameters by producing synthetic data instead.</p></figcaption></figure>

In the SDV's setup, we compare the statistical differences in the different synthetic datas using the [quality score](broken://pages/tsReTbZId8vlj4pm5fvV). (But in reality, we could use any statistical measure.) We repeated this process many times, leaving out a different row each time. The differential privacy score represents the *worst case scenario* that we measure when leaving out a row of real data.

## API & Usage

### Verify your DP synthesizer

If you are using a synthesizer specifically designed to offer differential privacy — such as [DPGCSynthesizer](/sdv/modeling/single-table-synthesizers/dpgcsynthesizer), or [DPGCFlexSynthesizer](/sdv/modeling/single-table-synthesizers/dpgcflexsynthesizer) — it's important to verify the privacy that the synthesizer is able to offer on your dataset. Use the `verify_differential_privacy` method on your synthesizer object and pass in the original data you used during fit.

```python
privacy_score = my_dpgc_synthesizer.verify_differential_privacy(
    data=my_dataframe,
    num_rows_synthetic_data=1000000,
    num_rows_test=10,
    test_data_seed=42,
    verbose=True
)
```

{% hint style="warning" %}
**Measuring differential privacy may take some time.** This empirical measure trains multiple synthesizers. Depending on the synthesizer algorithm, the size of the dataset, and the number of rows you'd like to test, the overall differential privacy measure may take significant time and computing resources. We recommend starting with a smaller dataset and smaller set of test rows.
{% endhint %}

**Parameters**:&#x20;

* (required) `data`: A dictionary that maps the table name to a pandas.DataFrame containing the data.
* `num_rows_synthetic_data`: The number of rows of synthetic data to produce before doing the differential privacy computations. We recommend using a large number of rows to get a stable representation of what the synthesizer has learned.
  * (default) `1000000`: Create 1 million rows of synthetic data each time we train a synthesizer
* `num_rows_test`: The number of rows of real data to test in a leave-one-out fashion. Each row represents an iteration of leaving the row out, training a synthesizer on the remaining data, and creating synthetic data. *The evaluation tool optimizes the rows to leave out by purposefully choosing rows with outliers and other interesting patterns.*
  * (default) `20`: Choose 20 rows to leave out (1 at a time) and measure differential privacy.
* `test_data_seed`: A seed to use to deterministically pick the rows to test
  * (default) `None`: Do not set a seed. Different rows may be left out each time you call this evaluation tool
* `verbose`: Whether to show progress.
  * (default) `True`: Show a progress bar for each row that is tested
  * `False`: Do not show a progress bar

**Returns**: A privacy score representing the empirical differential privacy using the synthesizer algorithm for the given dataset. The score ranges from 0 to 1, describing the impact that 1 row of training data has on the synthesizer.

* **(best) 1.0**: The synthesizer offers the best possible differential privacy protection. A single row of training data has no impact on what the synthesizer learns.
* **(worst) 0.0**: The synthesizer offers the worst possible differential privacy protection. A single row of training data has a massive impact on what the synthesizer learns.

{% hint style="success" %}
**SDVerified stamp of approval.** After running this function, your synthesizer will have recorded the fact that you have verified it.

```python
my_dpgc_synthesizer.is_verified()
```

```python
{
    'differential_privacy_verified': True
}
```

At this point, you can save your synthesizer object as a file and share it with others.<br>

```python
my_dpgc_synthesizer.save('my_dpgc_synthesizer.pkl')
```

{% endhint %}

### Measure DP on any synthesizer

Use the `measure_differential_privacy` tool to empirically measure the differential privacy of any synthesizer algorithm on a dataset. You can supply any [single-table SDV synthesizer](/sdv/modeling/single-table-synthesizers) for this evaluation.

```python
from sdv.evaluation.single_table import measure_differential_privacy

privacy_score = measure_differential_privacy(
    data=my_dataframe,
    metadata=my_metadata,
    synthesizer_name='GaussianCopulaSynthesizer',
    synthesizer_parameters={ 'default_distribution': 'norm' },
    num_rows_synthetic_data=1000000,
    num_rows_test=10,
    test_data_seed=42,
    verbose=True
)
```

{% hint style="warning" %}
**Measuring differential privacy may take some time.** This empirical measure trains multiple synthesizers. Depending on the synthesizer algorithm, the size of the dataset, and the number of rows you'd like to test, the overall differential privacy measure may take significant time and computing resources. We recommend starting with a smaller dataset and smaller set of test rows.
{% endhint %}

**Parameters**:

* (required) `data`: A dictionary that maps the table name to the pandas.DataFrame containing the real data for training the synthesizer
* (required) `metadata`: A [Metadata](/sdv/integration/metadata) object that describes your data
* (required) `synthesizer_name`: A string with the name of the synthesizer algorithm to use. You can choose from any of the [single-table SDV synthesizers](/sdv/modeling/single-table-synthesizers) that you have access to.
* `synthesizer_parameters`: A dictionary with the parameters to pass into the synthesizer. Use this to fine-tune the synthesizer algorithm.
  * (default) `None`: Use the default parameters for the given synthesizer
  * `<dict>`: A dictionary of parameters to use to fine-tune the synthesizer algorithm. The keys represent the parameter names, and the values are the parameter values.
* `num_rows_synthetic_data`: The number of rows of synthetic data to produce before doing the differential privacy computations. We recommend using a large number of rows to get a stable representation of what the synthesizer has learned.
  * (default) `1000000`: Create 1 million rows of synthetic data each time we train a synthesizer
* `num_rows_test`: The number of rows of real data to test in a leave-one-out fashion. Each row represents an iteration of leaving the row out, training a synthesizer on the remaining data, and creating synthetic data. *The evaluation tool optimizes the rows to leave out by purposefully choosing rows with outliers and other interesting patterns.*
  * (default) `20`: Choose 20 rows to leave out (1 at a time) and measure differential privacy.
* `test_data_seed`: A seed to use to deterministically pick the rows to test
  * (default) `None`: Do not set a seed. Different rows may be left out each time you call this evaluation tool
* `verbose`: Whether to show progress.
  * (default) `True`: Show a progress bar for each row that is tested
  * `False`: Do not show a progress bar

**Returns**: A privacy score representing the empirical differential privacy using the synthesizer algorithm for the given dataset. The score ranges from 0 to 1, describing the impact that 1 row of training data has on the synthesizer.

* **(best) 1.0**: The synthesizer offers the best possible differential privacy protection. A single row of training data has no impact on what the synthesizer learns.
* **(worst) 0.0**: The synthesizer offers the worst possible differential privacy protection. A single row of training data has a massive impact on what the synthesizer learns.


# Anonymizing PII Values

An important aspect of privacy is to ensure that the synthetic data isn't repeating original PII values from the real data. This also includes combinations of other types of columns that can, together, identify original entities in the real data. Use the the functions listed in this page to verify that PII values are newly created in synthetic data.

### get\_pii\_overlap

Use this function when you have a PII column in the real data and you'd like to ensure that the synthetic data does not repeat those PII values.

```python
from sdv.evaluation.utils import get_pii_overlap

overlap_amt = get_pii_overlap(
    real_data=real_data,
    synthetic_data=synthetic_data,
    table_name=table_name,
    pii_column_name='ssn',
    verbose=True
)
```

```
Number of common data points: 0 (0.0%)
✅ The synthetic data does not contain any PII values from the real data
```

**Parameters**:

* (required) `real_data`: A dictionary mapping a table name to a pandas.DataFrame containing real data
* (required) `synthetic_data`: A dictionary mapping a table name to a pandas.DataFrame containing synthetic data
* (required) `table_name`: The name of the table that contains the PII column to check
* (required) `pii_column_name`: The name of the column that contains PII values to check
* `verbose`: Whether to print out the interpretation of the result
  * (default) `True`: Print out the interpretation of the result
  * `False`: Do not print anything out. Only return the final result.

**Returns**: The # of PII data points in the synthetic data that have the exact same value as the real data.

### get\_combination\_overlap

Use this function when a few columns together could be used to identify an entity from the real data — for example a person's zipcode, gender, and date of birth. In this case, you'd like to ensure that the synthetic data doesn't repeat the exact combinations that occur in the real data.

```python
from sdv.evaluation.utils import get_combination_overlap

overlap_amt = get_combination_overlap(
    real_data=real_data,
    synthetic_data=synthetic_data,
    table_name=table_name,
    column_names=['zipcode', 'gender', 'date_of_birth'],
    verbose=True
)
```

```
Number of common combinations: 0 (0.0%)
✅ The synthetic data does not contain any of the same combinations from the real data
```

**Parameters**:

* (required) `real_data`: A dictionary mapping a table name to a pandas.DataFrame containing real data
* (required) `synthetic_data`: A dictionary mapping a table name to a pandas.DataFrame containing synthetic dat
* (required) `table_name`: The name of the table that contains the columns to check
* (required) `column_names`: A list of strings represent the column names to check. Combinations of these columns will be checked.
* `verbose`: Whether to print out the interpretation of the results

**Returns**: The # of rows in the synthetic data that contain the exact same combination of values as the real data (for the given columns).

## Interpreting the Overlap

In many cases, you'll see that there are 0 common PII values or combinations which indicates that the values are anonymized in the synthetic data. But **having a few overlapping values is not necessarily a bad thing** — in fact, it can indicate that good privacy practices if you have large amounts of synthetic data.

This happens when there are relatively few values possible, and/or when you have sampled a very large amount of synthetic data. In this case, it's possible to have an overlap due to random chance. For example, a US social security number is a 9-digit number such as `123-45-6789`; if you generate billions of synthetic data points, the synthetic data may end up up containing a real value due to random chance. This is actually a good thing: If the synthetic data purposefully avoided creating any of the real values, then we might be able to deduce the real values by looking up what *does not appear* in the synthetic data. This would violate privacy.

For this reason, if the overlap is relatively small (<2% of synthetic data points), then the functions will interpret this as a warning so that you're aware of it, but it's not necessarily something to worry about. Use your judgement when interpreting this result and [get in touch with us](https://forum.datacebo.com/) if you need help.&#x20;

```python
from sdv.evaluation.utils import get_pii_overlap

overlap_amt = get_pii_overlap(
    real_data=real_data,
    synthetic_data=synthetic_data,
    table_name=table_name,
    pii_column_name='ssn',
    verbose=True
)
```

```
Number of common combinations: 2 (0.000005%)
⚠️ The synthetic data contains a few PII values from the real data. This might be due to random chance.
```


# Troubleshooting


# Help with Installation

{% hint style="info" %}
**Are you an SDV Enterprise user?** Check out the [SDV Enterprise troubleshooting docs](https://docs.sdv.dev/sdv-enterprise/installation/troubleshooting).&#x20;
{% endhint %}

## Error in pip's Dependency Resolver

This error indicates that there are conflicting requirements in the Python libraries you are trying to install. You are likely to encounter this if you already have libraries installed in your environment.

To prevent conflicts, we recommend using a virtual environment (such as [virtualenv](https://virtualenv.pypa.io/en/latest/)). Ensure that you have the correct requirements and try the installation again in a fresh Python environment dedicated to the SDV.

{% hint style="info" %}
**Are you using a Google Colab notebook?** Sometimes Google Colab comes with different default libraries than the ones that SDV requires. Generally, you can ignore the printed errors.

In the top menu bar, click **Runtime** and then **Restart session**. After this, you can continue running the rest of your code.
{% endhint %}

## ModuleNotFoundError or AttributeError

These errors may indicate a problem with installation or usage. To debug this, start by verifying that you have the latest SDV version.

```python
import sdv
print(sdv.__version__)
```

When a version number prints, it should be the latest one found in our [Release Notes](https://github.com/sdv-dev/SDV/releases).&#x20;

**Installation problems:** If this code produces an error or if the version is old, then the newest SDV is not properly installed. Ensure that you have the correct Requirements and try the installation again in a fresh Python environment.&#x20;

**Usage problems:** If everything looks ok, then ensure that you are using the correct API and spelling. Starting from version `1.0.0`, some modules now use different names. Browse this website to see the latest API.

## Other Issues?

For other problems, please [visit our forum](https://forum.datacebo.com/). You can browse existing issues to see if there's a solution. If you cannot find what you're looking for, create an account and start new thread.


# Help with SDV

## ConstraintsNotMetError

A constraint should describe a rule that is true for every row in your real data. If any rows in the real data violate the rule, the SDV will return a `ConstraintsNotMetError`. Since the constraint is not true in your real data, the model will not be able to learn it.

If you see this error, you have two options:

* (recommended) **Remove the constraint.** This ensures the model learns patterns that exist in the real data. You can always use conditional sampling later to generate synthetic data with specific values.
* **Clean your input dataset.** If you remove the violative rows in the real data, then you will be able to apply the constraint with the cleaned data. This is not recommended because even if the synthetic data follows the rule, the model is not truly representative of the original data.

## Unable to load & use a previously-saved synthesizer

If you've previously trained and saved an SDV synthesizer, you should be able to load it into SDV in the future for sampling synthetic data. If this isn't working, we recommend **checking your Python environment** to see if any of the underlying library versions have changed.

<table><thead><tr><th width="245">Scenario</th><th>Guidance</th></tr></thead><tbody><tr><td>I've upgraded my version of SDV (or other libraries)</td><td>Upgrading your version should not affect your synthesizer. If this isn't working, please <a href="/sdv/support/troubleshooting/sdv-usage#other-issues">reach out</a> providing more details.<br><br><em>Please note that older synthesizers, creating before SDV v1.10.0 may not allow upgrades. In this case, you would need to train and save a new synthesizer on the latest version.</em></td></tr><tr><td>I've downgraded my version of SDV (or other libraries)</td><td>Downgrading versions is not supported. We recommend creating a new Python environment with the same version of SDV and other libraries.</td></tr></tbody></table>

{% hint style="info" %}
**Latest bug fixes and features will not be available for older synthesizers.** If you load in an old synthesizer into the latest version of SDV, that synthesizer may not be able to support all the latest SDV updates. To capture these, we recommend training and saving a new synthesizer on the latest version.
{% endhint %}

## Plotly charts not rendering in VSCode, JupyterLab, etc

In some environments, Plotly charts created by the SDV don't render inline and may instead be saved out to as images to a folder. To force Plotly to render charts inline, we recommend setting the renderer manually when you call `fig.show()`. We recommend reading [this Plotly article](https://plotly.com/python/renderers/) on how to set renderers and display figures in your preferred IDE.

<table><thead><tr><th>IDE</th><th>Code</th><th data-hidden></th></tr></thead><tbody><tr><td>Jupyter Notebook inside Visual Studio Code</td><td><code>fig.show('vscode')</code></td><td></td></tr><tr><td>JupyterLab</td><td><code>fig.show('jupyterlab')</code></td><td></td></tr></tbody></table>

## Other Issues?

For other problems, please [visit our forum](https://forum.datacebo.com/). You can browse existing issues to see if there's a solution. If you cannot find what you're looking for, create an account and start new thread.

{% hint style="danger" %}
**Important!** If you are encountering an Error, please provide as much detail as possible to help us replicate it. This includes:

* A code snippet to help us replicate the problem. The more easily we are able to recreate the error, the faster we can understand what is going wrong.
* The full stack trace of the code. That is, please copy-paste everything that is printed on your screen as part of the error, including the lines of code.
* Any other information that will allow us to replicate, especially the *metadata* you are using as well as any snippets of *training data*.
  {% endhint %}


# Versioning & Backwards Compatibility Policy

*Last Major Update: May 25, 2023*

As our ecosystem matures, we are aware that many of you are relying on the SDV synthesizers for your projects. To support your endeavors, we aim to maintain a high standard for SDV software releases.

Many Python libraries follow different approaches to managing software releases. We feel that it is important to communicate our standards with you upfront, so that you can make the best decisions for your project. While we may tweak or amend these in the future, we will do so with the goal of creating a more reliable and stable software for you.

{% hint style="success" %}
This versioning and backwards compatibility policy applies to the [SDV library](https://github.com/sdv-dev/SDV) that serves SDV Community and SDV Enterprise.
{% endhint %}

## Versioning

Each SDV software release has a version in the form of **`<MAJOR>.<MINOR>.<PATCH>`**, for example, 1.1.0. You can find a version history [GitHub releases page](https://github.com/sdv-dev/SDV/releases).

We aim to follow the general [SemVer](https://semver.org/) guidelines with a few exceptions, which we'll explain in this page.

* **MAJOR** releases include significant changes to the API, functionality or other parts of the workflow. These releases may not be backwards compatible so you may need to update your scripts when upgrading. We do not expect to make major releases often. (We have only made 1 so far.)
* **MINOR** releases include new features and potentially some deprecations. Minor releases will be backwards compatible for supported SDV features (see next section). We expect to make frequent minor releases to provide you with improved usage.
* **PATCH** releases are small updates that do not impact the API. They typically include internal code changes, logical updates and bug fixes. We expect to make patch releases ad-hoc for small but important updates.

## Supported Features

Everything described in [this docs website](https://docs.sdv.dev/sdv) is a supported feature. We maintain a high standard for these features:

* Backwards Compatibility: The feature will not crash on the next SDV release
* Documentation: The functionality and detailed API will be available in this documentation site
* Support: If you ask a question on our [forum](https://forum.datacebo.com/), a member of the SDV team will help you to debug the issue. To solve your problem, we may make fixes and ask you to upgrade to a newer version of the SDV.

{% hint style="info" %}
**Python Parameter Names.** When using the SDV library, we encourage you to provide named keywords when providing your parameters, just as shown in the docs.

For example, supply the keywords `modality` and `dataset_name` when [downloading a demo dataset](/sdv/integration/overview#download_demo).

```python
from sdv.datasets.demo import download_demo

data, metadata = download_demo(
    modality='multi_table',
    dataset_name='fake_hotels'
)
```

Keywords can be inputted in any order. If you do not provide the keywords, we cannot guarantee the functions will work.
{% endhint %}

Please note that the SDV source code may contain additional public features that are not described in our documentation. These features are unsupported. While you are welcome to experiment with such features, we will not be able to provide you with the same software standard.

## Backwards Compatibility

Backwards compatibility means that if your SDV script runs on today's SDV version, it will continue to run without crashing on the next release. The duration of backwards compatibility depends on the type of feature, as shown below.

<table><thead><tr><th width="496">Type of Change</th><th>Backwards Compatibility</th></tr></thead><tbody><tr><td>Modifying the API for a <strong>supported feature</strong>, i.e. one that is listed in these docs</td><td>Yes, until next major release</td></tr><tr><td>Modifying the API for an <strong>unsupported feature</strong>, i.e. one that is available in the codebase but is not listed in these docs</td><td>None. (We'll make our best effort to support larger changes for a few months.)</td></tr><tr><td>Internal, logical changes to any feature that does not affect the API, for eg. bug fixes, internal workflows and Python maintenance updates.</td><td>None</td></tr></tbody></table>

While we aim to provide this standard for all features, know that exceptional situations may arise. For example, a typo may force us to make an update without enforcing compatibility for the misspelled version.

{% hint style="info" %}
For security purposes, SDV Enterprise packages may expire and reach end of life sooner. For more information, please [SDV Enterprise IT Security](https://docs.sdv.dev/sdv-enterprise/installation/it-security) page.
{% endhint %}

## Communication

When we deprecate a feature, you will receive FutureWarning with more details about the changes and a recommendation for updating your script. The software will continue to produce the warning (without crashing) according to the compatibility standard.

When we deprecate a supported feature, we'll cover it in the corresponding release notes.

## Questions

For any questions, please [visit our forum](https://forum.datacebo.com/).