> For the complete documentation index, see [llms.txt](https://docs.sdv.dev/sdgym/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sdv.dev/sdgym/benchmarking/interpreting-results/explore-the-artifacts.md). # Explore the Artifacts Along with providing the Results Summary, SDGym also saves the intermediary artifacts it creates along the way — this includes the synthesizer objects themselves and the synthetic data they created for each dataset/synthesizer pair. ## Folder Structure When running the SDGym benchmark locally or on AWS, you can include an `output_destination` folder to save the artifacts. The benchmark splits the results into single-table and multi-table runs. Within each, it adds a sub-folder corresponding to the day you've run the benchmark, denoted as `SDGym_results__
_`. Inside the sub-folder, you'll see one folder per dataset, labeled as `__
_` and per synthesizer. Inside it are all the artifacts for that synthesizer/dataset combination. * Synthesizers are stored as `.pkl` * The synthetic data is stored as `_synthetic_data.csv` * Individual results for a synthesizer/dataset pair are stored in a single row in the `_benchmark_result.csv` file. An example of this folder structure is shown below on an SDGym benchmark run on June 24, 2025: ``` output_destination/ |--- single-table/ |--- SDGym_results_06_24_2025/ |--- census_06_24_2025/ |--- CTGANSynthesizer/ |--- CTGANSynthesizer.pkl |--- CTGANSynthesizer_synthetic_data.csv |--- CTGANSynthesizer_benchmark_result.csv |--- GaussianCopulaSynthesizer/ |--- ... |--- expedia_hotel_logs_06_24_2025/ |--- CTGANSynthesizer/ |--- ... |--- TVAESynthesizer/ |--- ... |--- metainfo.yaml |--- results.csv |--- multi-table/ |--- SDGym_results_06_24_2025/ |--- ... ``` You'll also notice two additional files in the folder: * The `results.csv` file contains the [Results Summary](/sdgym/benchmarking/interpreting-results/results-summary.md). This is the consolidation of all the individual `_benchmark_result.csv` files. * The `metainfo.yaml` file contains information about the benchmarking run, including the version of SDGym, version of SDV, date the run was finished, and any other relevant information. *This file is useful for debugging purposes.* {% hint style="info" %} **Are you running a benchmark regularly?** We recommend writing to the same `output_destination` folder every time. Each benchmark run will be stored in a different folder, ready for you to explore and compare results. For example: ``` output_destination/ |--- single-table/ |--- SDGym_results_06_24_2025/ |--- ... |--- metainfo.yaml |--- results.csv |--- SDGym_results_07_24_2025/ |--- ... |--- metainfo.yaml |--- results.csv ``` {% endhint %} ## Results Explorer Use the `ResultsExplorer` class to programmatically navigate the output destination folder and access the artifacts. ```python from sdgym import ResultsExplorer my_results_explorer = ResultsExplorer( path='my_files/output_destination/', modality='single_table' ) ``` **Parameters**: * (required) `path`: The filepath to the top-level folder containing all the results * (required) `modality`: Choose either `'single_table'` or `'multi_table'` to view the right results Once you've created the object, you can use any of the functions below to navigate and access the artifacts. ### list Use this function to list out all the different SDGym results that exist within the folder. Each result corresponds to a sub-folder named `SDGym_results_
__`. **Parameters**: (None) **Output:** A list of strings corresponding to each of the SDGym benchmarking results ```python my_results_explorer.list() [ 'SDGym_results_06_24_2025', 'SDGym_results_07_24_2025', ... ] ``` ### load\_metainfo Use this function to load the `metainfo.yaml` file contains information about the benchmarking run, including the version of SDGym, version of SDV, date the run was finished, and any other relevant information. **Parameters:** * `results_folder_name`: A string with the name of folder that contains the run. *Use the `list` function to get a list of possible options.* * (default) `None`: Use the results from the latest benchmark **Output**: A Python dictionary containing the contents of the metainfo from all the runs that occurred during that day. *This information can be useful for debugging.* ```python metainfo = my_results_explorer.load_metainfo( results_folder_name='SDGym_results_06_24_2025') ``` ``` { 'run_06_24_2025_0': { 'sdgym_version': '0.10.0', 'sdv_version': '1.23.0', 'starting_date': '06_24_2025 18:05:35', 'completed_date': '06_24_2025 18:06:23', 'jobs': [ ('alarm', 'GaussianCopulaSynthesizer'), ('census', 'GaussianCopulaSynthesizer'), ('census', 'CTGANSynthesizer'), ...]}} ``` ### load\_results Use this function to load the results summary from a benchmarking run on a given day. **Parameters**: * `results_folder_name`: A string with the name of folder that contains the run. *Use the `list` function to get a list of possible options.* * (default) `None`: Use the results from the latest benchmark * `dataset_name`: A string with the name of the dataset. If provided, the results are filtered to the dataset name. * (default) `None`: Return the results from all the datasets that were run * `synthesizer_names`: A list of strings with the names of the synthesizers. (See [Synthesizers](/sdgym/customization/synthesizers.md).) If provided, the results are filtered to only the synthesizers in the list. * (default) `None`: Return the results from all the synthesizers that were run * `summary`: A boolean that describes whether to return all the columns in results or just a few important columns. For more information, see the [Results Summary](/sdgym/benchmarking/interpreting-results/results-summary.md) guide. * (default) `False`: Return all the columns in the results * `True`: Only return a few key columns that are important for analysis. These columns are: `Synthesizer`, `Dataset`, `Adjusted_Total_Time`, `Adjusted_Quality_Score`, and `Diagnostic_Score` **Output:** A pandas DataFrame object containing the results from that day. For more information about the results, see the [Results Summary](/sdgym/benchmarking/interpreting-results/results-summary.md) guide. ```python results_summary = my_results_explorer.load_results( results_folder_name='SDGym_results_06_24_2025') ``` ``` Synthesizer Dataset Dataset_Size_MB Train_Time ... GaussianCopulaSynthesizer alarm 34.5 123.56 ... GaussianCopulaSynthesizer census 130.2 2356.12 ... CTGANSynthesizer alarm 34.5 NaN ... CTGANSynthesizer census 130.2 3140.4 ... UniformSynthesizer alarm 34.5 1.1 ... UniformSynthesizer census 130.2 15.5 ... ``` {% hint style="info" %} In most cases, the results summary is the same as the `results.csv` file. However, if you've performed multiple runs on the same day, then SDGym will concatenate the results from all runs on that day. For more information, see the [FAQ](#faqs) below. {% endhint %} ### load\_synthesizer Use this function to load a synthesizer object created for a dataset during a particular benchmark. **Parameters**: * (required) `dataset_name`: A string with the name of the dataset that the synthesizer was trained on. * (required) `synthesizer_name`: A string with the name of the synthesizer that was used.\ \&#xNAN;*Use the results summary to get a list of all possible dataset and synthesizer options.* * `results_folder_name`: A string with the name of the sub-folder that corresponds to a benchmarking result. *Use the* [*list*](#list) *function to get the possible options.* * (default) `None`: Use the results from the latest benchmark **Output**: The synthesizer object that was created during the provided SDGym benchmarking run. The synthesizer has already been fitted on the given dataset. ```python my_synthesizer = my_results_explorer.load_synthesizer( results_folder_name='SDGym_results_06_24_2025', dataset_name='alarm', synthesizer_name='GaussianCopulaSynthesizer' ) ``` ### load\_synthetic\_data Use this function to load the synthetic data created for a dataset during a particular benchmark. **Parameters:** * (required) `dataset_name`: A string with the name of the dataset that the benchmark created synthetic data for. * (required) `synthesizer_name`: A string with the name of the synthesizer that was used.\ \&#xNAN;*Use the results summary to get a list of all possible dataset and synthesizer options.* * `results_folder_name`: A string with the name of the sub-folder that corresponds to a benchmarking result. *Use the* [*list*](#list) *function to get the possible options.* * (default) `None`: Use the results from the latest benchmark **Output**: * For a single-table dataset, this is a pandas.DataFrame object containing the synthetic data that corresponds to the given dataset for the given SDGym benchmarking run. * For a multi-table dataset, this is a dictionary that maps each table name (string) to the pandas.DataFrame object containing the synthetic data. ```python my_synthetic_data = my_results_explorer.load_synthetic_data( results_folder_name='SDGym_results_06_24_2025', dataset_name='alarm', synthesizer_name='GaussianCopulaSynthesizer' ) ``` ### load\_real\_data Use this function to load the original dataset that is used for benchmarking SDGym. The dataset is the same regardless of the SDGym benchmark. **Parameters**: * (required) `dataset_name`: A string with the name of the dataset. For more information, see the [Datasets reference](/sdgym/customization/datasets.md). **Output**: * For a single-table dataset, this is a pandas.DataFrame object containing the real dat * For a multi-table dataset, this is a dictionary that maps each table name (string) to the pandas.DataFrame object containing the real data. ```python original_dataset = my_results_explorer.load_real_data( dataset_name='alarm' ) ``` ## FAQs
If I run SDGym multiple times in a day, which artifacts are stored? Synthesizer and synthetic data artifacts are always written to the same folder during a given day. If there are multiple runs within a day, SDGym will not override the files. Instead, it will generate new files with suffixes `(1)`, `(2)`, `(3)` and so on. The example below shows what happens if you run SDGym twice — first with the CTGANSynthesizer and then with the TVAESynthesizer. The synthesizers and synthetic data are stored within their respective tables. But you'll see multiple `metainfo.yaml` and `results.csv` files, corresponding to each run. ``` output_destination/ |--- single-table/ |--- SDGym_results_06_24_2025/ |--- census_06_24_2025/ |--- CTGANSynthesizer/ |--- ... |--- TVAEynthesizer/ |--- ... |--- expedia_hotel_logs_06_24_2025/ |--- CTGANSynthesizer/ |--- ... |--- TVAESynthesizer/ |--- ... |--- metainfo.yaml |--- results.csv |--- metainfo(1).yaml |--- results(1).csv ```
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.sdv.dev/sdgym/benchmarking/interpreting-results/explore-the-artifacts.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.