Constraint-Augmented Generation (CAG)

Do you have business rules in your dataset? These are deterministic rules that every single row in your data must follow in order to be considered valid. By default, SDV synthesizers are probabilistic so they may not learn to match your rule 100% of the time.

The good news is that you can input your business rules into your synthesizer using constraints. Constraint-augmented generation ensures that your synthetic data meets the constraint — 100% of the time.

Auto-detect business rules or add them in yourself. SDV automatically detects common business rules that your datasets may have. You inspect what SDV has detected, and modify those constraints.

Constraint Example

One example of a business rule is when the values in one column always have to be greater than values in another column. This is true for every single row of data.

In this business rule, the checkout_date must be greater than the checkin_date for all rows.

❖ Auto-detect constraints. SDV automatically detects a constraint that corresponds to this business rule (Inequality constraint). You can add it to your synthesizer to ensure that the synthetic data follows the rule. The auto-detection feature is available in the CAG bundle.

from sdv.single_table import GaussianCopulaSynthesizer

my_synthesizer = GaussianCopulaSynthesizer(metadata)
constraints = my_synthesizer.detect_constraints(data)
my_synthesizer.add_constraints(constraints)

1 Constraints Detected

0. ChainedInequality(column_names=['checkin_date', 'checkout_date'], table_name='fake_hotel_guests')

Add constraints directly. You can also create the constraint yourself and supply it to your synthesizer.

from sdv.cag import Inequality

# create a constraint that corresponds to your business rule
my_constraint = Inequality(
    low_column_name='checkin_date',
    high_column_name='checkout_date'
)

# add the constraint to your SDV synthseizer
my_synthesizer.add_constraints(constraints=[
    my_constraint
])

Predefined Constraint Classes

Predefined constraints apply logic within a single table or between multiple tables. Predefined constraints represent common business rules that may appear in your dataset. SDV is able to auto-detect predefined constraint classes, and you are also able to create and apply them yourself.

Browse the predefined constraints to learn more. We also recommend going through our tutorial.

Programmable Constraints

If your logic cannot be described by any of the predefined constraints, please reach out to us with the details of your dataset. We will track this as a feature request.

Have you purchased the CAG bundle? For our SDV Enterprise customers who have purchased the CAG bundle, our team is able to provide additional support for constraints. After you reach out to us, we will be able to provide you with some specially programmed constraints for you to test.

Constraints API

Detect, create, and add constraint objects to your synthesizer. For more details, see the API Reference.

FAQs

Is it possible to apply a constraint even if my dataset doesn't always follow it?

SDV synthesizers check to ensure that your real dataset follows the constraints 100% of the time. So it is currently not allowed to add a constraint if some rows of your data violate the rule. There are a rew reasons for this:

1. A constraint is supposed to represent business logic that must be met in order for the synthetic data to be considered valid. For example, it would not even be possible to load the synthetic data into your downstream application if it doesn't follow the rule 100% of the time. If your real dataset doesn't always adhere to this logic, then it indicates that the rule isn't strictly needed.

2. The synthesizer is meant to learn and emulate the patterns in the real dataset as-is. So if you present real data that violates a few of the rules, the synthesizers are designed to understand that that create similar synthetic data. (If you'd like the synthesizer to purposefully change the patterns, we recommend using conditional sampling after the synthesizer is already created.)

3. We developed the internal algorithms of each constraint assuming that the rule is 100% true in the real data. Therefore, if we were to allow data that is not 100% valid, the algorithms may not run as intended. You may find that the synthesizer would crash, or produce invalid error/null values.

How is modeling & sampling performance impacted by constraints?

In most cases, the time it takes to fit the model and sample synthetic data should not be significantly affected. However, there are certain scenarios where you may notice a slow-down:

• You have a large number of constraints that overlap. That is, multiple constraints are referencing the same columns of the data.

• You are conditional sampling on the constrained columns. This requires some special processing and it may not always be possible to efficiently create conditional synthetic data.

For any questions or feature requests related to performance, please provide feedback describing your data, constraints and sampling needs.

How does the SDV handle the constraints?

Under-the-hood, the SDV uses a combination of strategies to ensure that the synthetic data always follows the constraints. These strategies are:

• Transformation: Most of the time, it's possible to transform the data in a way that guarantees the models will be able to learn the constraint. This is paired with a reverse transformation to ensure the synthetic data looks like the original.

• Reject Sampling: Another strategy is to model and sample synthetic data as usual, and then throw away any rows in the synthetic data that violate the constraints.

• Algorithmic Injection: Complex CAG patterns sometimes come with their own algorithms for ensuring robust and accurate modeling. These algorithms may learn additional patterns from your data. They compatible with any SDV synthesizer.

Can I use constraints to generate data from scratch (DayZSynthesizer)?

Yes, if you are an SDV Enterprise user, you have access to the DayZSynthesizer which can be used to generate data from scratch. You are be able to add single-table constraints to DayZSynthesizer, including your own, programmable single-table constraint.

Multi-table constraints are not yet supported.