> For the complete documentation index, see [llms.txt](https://docs.sdv.dev/sdv/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sdv.dev/sdv/evaluation/privacy/anonymizing-pii-values.md). # Anonymizing PII Values An important aspect of privacy is to ensure that the synthetic data isn't repeating original PII values from the real data. This also includes combinations of other types of columns that can, together, identify original entities in the real data. Use the the functions listed in this page to verify that PII values are newly created in synthetic data. ### get\_pii\_overlap Use this function when you have a PII column in the real data and you'd like to ensure that the synthetic data does not repeat those PII values. ```python from sdv.evaluation.utils import get_pii_overlap overlap_amt = get_pii_overlap( real_data=real_data, synthetic_data=synthetic_data, table_name=table_name, pii_column_name='ssn', verbose=True ) ``` ``` Number of common data points: 0 (0.0%) ✅ The synthetic data does not contain any PII values from the real data ``` **Parameters**: * (required) `real_data`: A dictionary mapping a table name to a pandas.DataFrame containing real data * (required) `synthetic_data`: A dictionary mapping a table name to a pandas.DataFrame containing synthetic data * (required) `table_name`: The name of the table that contains the PII column to check * (required) `pii_column_name`: The name of the column that contains PII values to check * `verbose`: Whether to print out the interpretation of the result * (default) `True`: Print out the interpretation of the result * `False`: Do not print anything out. Only return the final result. **Returns**: The # of PII data points in the synthetic data that have the exact same value as the real data. ### get\_combination\_overlap Use this function when a few columns together could be used to identify an entity from the real data — for example a person's zipcode, gender, and date of birth. In this case, you'd like to ensure that the synthetic data doesn't repeat the exact combinations that occur in the real data. ```python from sdv.evaluation.utils import get_combination_overlap overlap_amt = get_combination_overlap( real_data=real_data, synthetic_data=synthetic_data, table_name=table_name, column_names=['zipcode', 'gender', 'date_of_birth'], verbose=True ) ``` ``` Number of common combinations: 0 (0.0%) ✅ The synthetic data does not contain any of the same combinations from the real data ``` **Parameters**: * (required) `real_data`: A dictionary mapping a table name to a pandas.DataFrame containing real data * (required) `synthetic_data`: A dictionary mapping a table name to a pandas.DataFrame containing synthetic dat * (required) `table_name`: The name of the table that contains the columns to check * (required) `column_names`: A list of strings represent the column names to check. Combinations of these columns will be checked. * `verbose`: Whether to print out the interpretation of the results **Returns**: The # of rows in the synthetic data that contain the exact same combination of values as the real data (for the given columns). ## Interpreting the Overlap In many cases, you'll see that there are 0 common PII values or combinations which indicates that the values are anonymized in the synthetic data. But **having a few overlapping values is not necessarily a bad thing** — in fact, it can indicate that good privacy practices if you have large amounts of synthetic data. This happens when there are relatively few values possible, and/or when you have sampled a very large amount of synthetic data. In this case, it's possible to have an overlap due to random chance. For example, a US social security number is a 9-digit number such as `123-45-6789`; if you generate billions of synthetic data points, the synthetic data may end up up containing a real value due to random chance. This is actually a good thing: If the synthetic data purposefully avoided creating any of the real values, then we might be able to deduce the real values by looking up what *does not appear* in the synthetic data. This would violate privacy. For this reason, if the overlap is relatively small (<2% of synthetic data points), then the functions will interpret this as a warning so that you're aware of it, but it's not necessarily something to worry about. Use your judgement when interpreting this result and [get in touch with us](https://forum.datacebo.com/) if you need help. ```python from sdv.evaluation.utils import get_pii_overlap overlap_amt = get_pii_overlap( real_data=real_data, synthetic_data=synthetic_data, table_name=table_name, pii_column_name='ssn', verbose=True ) ``` ``` Number of common combinations: 2 (0.000005%) ⚠️ The synthetic data contains a few PII values from the real data. This might be due to random chance. ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.sdv.dev/sdv/evaluation/privacy/anonymizing-pii-values.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.