This metric computes the similarity of a real column vs. a synthetic column in terms of the column shapes. You can think of the shape as what you observe when you plot a bar graph of the column.
- Categorical: This metric is meant for discrete, categorical data
- Boolean: This metric works on boolean data
This metric does not accept missing values
(best) 1.0: The p-value is high, indicating that the synthetic data is not very different from the real data
(worst) 0.0: The p-value is low, indicating that the synthetic data is significantly different than the real data
This test normalizes the real and synthetic data in order to compute the category frequencies. Then, it applies the Chi-squared test  to test the null hypothesis that the synthetic data comes from the same distribution as the real data.
The test returns the p-value , where a smaller p-value indicates that the synthetic data is significantly different from the real data, rejecting the null hypothesis and leading to a worse overall score.
Access this metric from the
single_columnmodule and use the
from sdmetrics.single_column import CSTest
real_data: A pandas.Series containing a single column
synthetic_data: A similar pandas.Series object with the synthetic version of the column
This metric is in Beta. Be careful when using the metric and interpreting its score.
- This test is invalid when the category frequencies are too small . Because this test normalizes all frequencies, they are all <1, affecting the overall results.
- The p-value may be hacked by supplying data of different sizes
- The p-value interpretation may not be useful. Most users are interested in quantifying the differences between real and synthetic data, not testing the null hypothesis. In fact, if the synthetic data model is a simplification of the real data, the null hypothesis will nearly always be always be false with enough data.