AWS Integration

SDGym offers a native, AWS integration for compute and storage.

Authentication

Authenticate into your Amazon account first using environment variables.

import os
import sdgym

# use environment variables to authenticate into your AWS account
os.environ['AWS_ACCESS_KEY_ID'] = 'XXX'
os.environ['AWS_SECRET_ACCESS_KEY'] = 'XXX'
os.environ['AWS_SESSION_TOKEN'] = 'XXX' # optional

Usage

Once authenticated, you can supply parameters to SDGym for:

• compute: run your script on EC2 instances instead of your local machine,

• storage: read from datasets stored in S3 and write results back into S3

import sdgym

sdgym.benchmark_single_table(
    additional_datasets_folder='s3://my-demo-bucket', # read datasets from S3
    run_on_ec2=True, # use EC2 for compute
    output_filepath='s3://my-demo-bucket/results.csv', # store results in S3
    detailed_results_folder='s3://my-demo-bucket/details/' # store details in S3
)

For more information, see the docs for Running a Benchmark.