# ContingencySimilarity

This metric computes the similarity of a pair of categorical columns between the real and synthetic datasets -- aka it compares 2D distributions.

## Data Compatibility

**Categorical**: This metric is meant for discrete, categorical data**Boolean**: This metric works well on boolean data

To use this metric, both of the columns must be compatible. If there are missing values in the columns, the metric will treat them as an additional, single category.

## Score

**(best) 1.0**: The contingency table is exactly the same between the real vs. synthetic data

**(worst) 0.0**: The contingency table is as different as can be

The plots below show an example of fictitious real and synthetic data with ContingencySimilarity=0.92.

## How does it work?

For a pair of columns, *A* and *B*, the test computes a normalized contingency table [1] for the real and synthetic data. This table describes the proportion of rows that have each combination of categories in *A* and *B*.

Then, it computes the difference between the contingency tables using the Total Variation Distance [2]. Finally, we subtract the distance from 1 to ensure that a high score means high similarity. The process is summarized by the formula below.

In the formula, α describes all the possible categories in column A and β describes all the possible categories in column B. Meanwhile, R and S refer to the real and synthetic frequencies for those categories.

## Usage

**Recommended Usage:** The Quality Report applies this metric to every pair of compatible columns and provides visualizations to understand the score.

To manually run this metric, access the `column_pairs`

module and use the `compute`

method.

**Parameters**

(required)

`real_data`

: A pandas.DataFrame object containing 2 columns of real data(required)

`synthetic_data`

: A pandas.DataFrame object containing 2 columns of synthetic data

## FAQs

**Technical Notes: Association**

It is possible to use an association measure to quantify whether a contingency table is biased towards certain categories. There are many ways to compute association, a common coefficient being Cramer's V [3].

The association score measures the degree of bias but not its direction. This can lead to misleading results when comparing real and synthetic data. In the example below, both the real and synthetic data have the same, high association score even though they are biased towards different categories. If we simply compared association scores, we would erroneously conclude that the similarity is high, at 1.0.

This is why SDMetrics does *not* use association metrics to compare real and synthetic data.

## References

[1] https://en.wikipedia.org/wiki/Contingency_table

[2] https://en.wikipedia.org/wiki/Total_variation_distance_of_probability_measures

Last updated