Conditional Sampling
Do you have exact values that you'd like to include in the synthetic data? Using conditional sampling to provide this information. Conditional sampling allows you to target the exact data you need, while still preserving correlations with other, related variables.
Generate hypothetical scenarios, by fixing the values to correspond to extreme cases
De-bias your data, by requesting an equal balance of labels
Impute unknown data, by requesting the data that you already know
from sdv.sampling import Condition
# Step 1: Create Your Conditions
suite_guests_with_rewards = Condition(
num_rows=250,
column_values={'room_type': 'SUITE', 'has_rewards': True}
)
# Step 2: Sample Synthetic Data
synthetic_data = synthesizer.sample_from_conditions([suite_guests_with_rewards])Follow the 2-step workflow below to create your conditions and then ask SDV to sample with them.
Step 1: Create Your Conditions
Use conditions to specify which exact, fixed values you'd like to appear in your synthetic data. SDV creates synthetic data with the fixed value and creates other variables based on it. The other variables will contain the same patterns and correlations with respect to the fixed value.
You can create as many conditions as you need to fully describe the synthetic data that you'd like to create.
Condition
Create a Condition object to specify exact values you want to include in the synthetic data.
In this example, we want to create 250 guests staying in suites with rewards, and an additional 250 guests staying in suites without rewards.
from sdv.sampling import Condition
suite_guests_with_rewards = Condition(
num_rows=250,
column_values={'room_type': 'SUITE', 'has_rewards': True}
)
suite_guests_without_rewards = Condition(
num_rows=250,
column_values={'room_type': 'SUITE', 'has_rewards': False}
)Parameters
(required)
num_rows: The number of rows that need to be included in the scenario(required)
column_values: A dictionary with the scenario. The keys should be a column names and the values should be the exact data that the column should have. You can fix any columns, as long as they are not primary or foreign keys.
DataFrameCondition
Create a DataFrameCondition if you already have a DataFrame of fixed-value columns that you'd like to include in the synthetic data. This is a shortcut you can use instead of creating multiple Condition objects.
import pandas as pd
from sdv.sampling import DataFrameCondition
my_fixed_datapoints = pd.DataFrame(data={
'room_type': ['SUITE', 'SUITE', 'SUITE', 'SUITE', 'SUITE'],
'has_rewards': [True, True, True, False, False]
})
type_rewards_condition = DataFrameCondition(dataframe=my_fixed_datapoints)Parameters:
(required)
dataframe: A pandas.DataFrame object containing the columns whose values you want to fix
Step 2: Sample Synthetic Data
Once you have the synthetic data, you can use any single-table SDV synthesizer to sample conditions. Your synthesizer must have already been trained (fit) on the data.
sample_from_conditions
Use this function to create synthetic data based on the conditions.
synthetic_data = synthesizer.sample_from_conditions(
conditions=[suite_guests_with_rewards, suite_guests_without_rewards, type_rewards_condition],
output_file_path='synthetic_simulated_scenario.csv'
)Parameters
(required)
conditions: A list of Condition or DataFrameCondition objects that specify the exact values that you want to fix.batch_size: An integer >0, describing the number of rows to sample at a time. If you are sampling a large number of rows, setting a smaller batch size allows you to see and save incremental progress. Defaults to the same asnum_rows.max_tries_per_batch: An integer >0, describing the number of sampling attempts to make per batch. If you have included constraints, it may take multiple batches to create valid data. Defaults to100.output_file_path: A string describing a CSV filepath for writing the synthetic data. Specify toNoneto skip writing to a file. Defaults toNone.
Returns A pandas DataFrame object with synthetic data. The synthetic data is simulated based on the conditions.
The quality of your synthetic data is preserved. When you provide conditions, the SDV synthesizers will match your conditions and create the remaining data based on the patterns that it learned. Your synthetic data will continue to have the same statistical patterns between columns.
Troubleshooting
Conditional sampling is a complex feature. In some cases, your synthesizer may not be able to create all rows of synthetic data that you request. Let's walk through some areas that you can investigate.
Which synthesizer are you using?
The SDV synthesizers have different conditional sampling capabilities. If you are using the CTGAN, TVAE or CopulaGAN synthesizers, the SDV may unable to complete your conditional sampling request in some instances.
Neural network-based synthesizers use a reject sampling approach: They sample synthetic data freely, keep the rows that match your conditions and repeat the process as needed. This may not be efficient if the conditional values are extremely rare.
Some suggestions:
Use a larger
batch_sizeormax_tries_per_batch. The more rare your conditions, the more attempts the SDV will have to make.Try using the GaussianCopulaSynthesizer instead. This synthesizer can sample conditions mathematically instead of reject sampling, which is more efficient.
Are you including constraints?
Any synthesizer that has constraints may have to use reject sampling to ensure the rows are valid. This can slow down the process.
Suggestions:
Use a larger
batch_sizeormax_tries_per_batch. The more constraints you have, the more attempts the SDV will have to make.Consider removing constraints and refitting your synthesizer. This will help if you are conditioning on desired columns that were involved in a constraint.
Are you requesting data that is out of range?
The SDV synthesizers are designed to learn patterns from the input data that you've added during fit, including the the min and max ranges of each column. If you are requesting conditional values that are outside of the ranges, the synthesizer may not be able to accommodate your request.
Check to see whether you values at out of range. If so, you may have more success if you fit your synthesizer without enforcing min max values. For more details, see the Modeling API.
synthesizer = GaussianCopulaSynthesizer(
metadata,
enforce_min_max_values=False
)Need more help?
Raise an issue on GitHub with more details about your usage. To help us replicate your issue, please provide us with as much detail as possible about your data, the synthesizer you're using and any parameters or features you're using with it.
Last updated