Ctrlk
ForumDataCebo
For the complete documentation index, see llms.txt. This page is also available as Markdown.

⭐Diagnostic Report API

The Diagnostic Report is designed to capture basic diagnostic measurements across your entire dataset at once, reporting areas that may be problematic. Use this as a first step to ensuring that you have created valid synthetic data.

from sdmetrics.reports import DiagnosticReport

report = DiagnosticReport()
report.generate(real_data, synthetic_data, metadata)
Generating report ...

(1/3) Evaluating Data Validity: |██████████| 15/15 [00:00<00:00, 603.69it/s]|
Data Validity Score: 100.0%

(2/3) Evaluating Data Structure: |██████████| 2/2 [00:00<00:00, 151.49it/s]|
Data Structure Score: 100.0%

(3/3) Evaluating Relationship Validity: |██████████| 1/1 [00:00<00:00, 68.51it/s]|
Relationship Validity Score: 100.0%

Overall Score (Average): 100.0%

💯 The score should be 100%. The diagnostic report checks for basic data validity and data structure issues. If you want to create synthetic data that looks and feels similar to the real data, you should expect the score to be perfect. If you are using any of the default SDV synthesizers, the score should always be 1.0.

The only exception is if you have made an explicit choice to deviate from the real data in some way. For example, you purposely want the synthetic data to go out-of-bounds.

Usage

This section describes the API for running the report.

Generating the report

DiagnosticReport()

Create your report object by importing it from the reports module.

generate(real_data, synthetic_data, metadata)

Generate your report by passing in the data and metadata.

  • (required) real_data: A dictionary mapping the name of each table to a pandas.DataFrame containing the real data for that table.

  • (required) synthetic_data: A dictionary mapping the name of each table to a pandas.DataFrame containing the synthetic data for that table

  • (required) metadata: A dictionary describing the format, types of data and relationship between the tables. See Multi Table Metadata for more details.

  • verbose: A boolean describing whether or not to print the report progress and results. Defaults to True. Set this to False to run the report silently.

You'll see a progress bar as the report is generated. Once completed, the diagnostic results are printed out.

Getting & explaining the results

get_score()

Use this method at any point to retrieve the overall score.

Returns: A floating point value between 0 and 1 that summarizes the quality of your synthetic data.

💯 The score should be 100% or very close to it. The diagnostic report checks for basic data validity and data structure issues. If you want to create synthetic data that looks and feels similar to the real data, you should expect to score to be perfect. If you are using any of the default SDV synthesizers, the score should always be 1.0.

get_properties()

Use this method at any point to retrieve each property that the report evaluated

Returns: A dictionary that lists each property name and its associated score

get_details(property_name)

Use this method to get more details about a particular property.

  • (required) property_name: A string with the name of the property. One of: 'Data Validity', 'Data Structure' or 'Relationship Validity'.

  • table_name: A string with the name of the table. If provided, you'll receive filtered results for the table.

Returns: A pandas.DataFrame object that returns more details about the property for the given table

For example, the details for 'Data Validity' shows the name of each individual column, the metric that was used to compute it and the overall score for that column.

Visualizing the report

You can visualize the properties and use the SDMetrics utilities to visualize the raw data too.

get_visualization(property_name, table_name)

Use this method to visualize the details about a property.

  • (required) property_name: A string with the name of the property. Currently, 'Data Validity' or 'Relationship Validity' are supported.

  • (required) table_name: A string with the name of the table

Returns: A plotly.Figure object

For example, the 'Data Validity' property visualizes the score for every column as well as the metric used to compute it.

Other visualizations are available! Use the SDMetrics Visualization Utilities to get more insights into your data. Tip! All visualizations returned in this report are interactive. If you're using an iPython notebook, you can zoom, pan, toggle legends and take screenshots.

Saving & loading the report

You can save your report if you want to share or access it in the future.

save(filepath)

Save the Python report object

  • (required) filepath: The name of file to save the object. This must end with .pkl

The report does not save the full real and synthetic datasets, but it does save the metadata along with the score for each property, breakdown and metric.

The score information may still leak sensitive details about your real data. Use caution when deciding where to store the report and who to share it with.

DiagnosticReport.load(filepath)

Load the report from the file

  • (required) filepath: The name of the file where the report is stored

Returns: A DiagnosticReport object.

FAQs

What is the best way to see the visualizations? Can I save them?

This report returns all visualizations as plotly.Figure object, which are integrated with most iPython notebooks (eg. Colab, Jupyter).

Tip! You can interact with the visualizations when you're viewing them in a notebook. You can zoom, pan and take screenshots.

It's also possible to programmatically save a static image export. See the Plotly Guide for more details.

PreviousDiagnosticNextBoundaryAdherence

Last updated