❖ PostgreSQL

This page describes how to integrate with your Postgres database.

Supported versions: PostgreSQL 18+

❖ SDV Enterprise Bundle. This feature is available as part of the AI Connectors Bundle, an optional add-on to SDV Enterprise. For more information, please visit the AI Connectors Bundle page.

Installation

To use this feature, please make sure you have installed the bundle with the optional db-postgresql dependency. For more information, see the SDV Enterprise Installation docs.

pip install sdv-installer --upgrade
sdv-installer install --package bundle-ai-connectors --options db-postgresql

Connector

Use this connector to create a connection to your Postgres database.

from sdv.io.database import PostgreSQLConnector

connector = PostgreSQLConnector()

Parameters (None)

Output A connector object that you can use to import data and metadata

Config File

Create a YAML file that corresponds to the database to connect to. You will need separate config files for your source dataset (for importing training data) and a destination dataset (for exporting synthetic data).

The config file has the following structure:

connection:
  host: <string>
  port: <string>
  database_name: <string> 
  schema_name: <string>

auth:
  username: <string>
  password: <string>

runtime:
  connection_timeout: <int>
  
options.PostgreSQLConnector:
  application_name: <string>
  sslmode: <disable | allow | prefer | require | verify-ca | verify-full>

Parameters:

connection: This section defines the database tables you want to connect to. It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

• (required) host: The network address of the system running the database service. Based on your db setup, this can be a hostname (eg. db.example.com) or an IP address (eg. 10.0.0.1)

• (required) port: The network endpoint on the host where the database service listens for client connections. This is a numeric identifier but please provide it as a string.

• (required) database_name: A top-level logical container for data and database objects within a database system.

• (required) schema_name: A named logical grouping of database objects within a database, used for organization and access control.

auth: This section provides information about your identity, which is required to connect to the database. It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

• (required) username: The identifier of who is authenticating the connection.

• (required) password: A secret credential associated with the username, used to verify its identity during authentication

runtime: This section contains other parameters that come into play during runtime, either when establishing the connection or running additional queries. It is common to all traditional, self-managed databases that are not managed by a Cloud vendor.

• connection_timeout: The maximum amount of time a client will wait while attempting to establish a connection to the database before stopping the attempt. This applies when first establishing the connection, not for any subsequent queries or timeouts.

  • (default) Driver and environment dependent, typically between ~10-30 seconds, but potentially unbounded.

options.PostgreSQLConnector: This section includes information that is specific for Postgres databases

• application_name: An optional string with the name of the application that is connecting to the database (for example "SDV_Enterprise"). This is useful for reporting and monitoring.

• sslmode: Specifies whether the connection between the client (SDV) and the database server is encrypted and if so, whether the client (SDV) verifies the identity of the database server when establishing an encrypted connection.

  • disable: Send over the data and queries as plain text. (Do not perform any encryption or validation.)

  • allow: Try to send over the data as plain text first. If that fails, then encrypt as a fallback.

  • (default) prefer: Try to send over the data using encryption first. If that fails, send plain text as a fallback.

  • require: Always send over the data using encryption

  • verify-ca: In addition to sending the data using encryption, the client (SDV) also validates the certificate. The hostname is not validated.

  • verify-full: In addition to sending the data using encryption, the client (SDV) also validates the certificate and hostname.

Setting Your Config File

Use the set_import_config and set_export_config functions to add your config YAML file to your connector.

connector.set_import_config(filepath='source_db_config.yaml')
connector.set_export_config(filepath='destination_db_config.yaml')

Use the connector to create metadata, import a subset of training data, and export synthetic data. For more information, see the Usage API.

FAQ

Which permissions are needed for importing and exporting?

Importing data requires read access. For importing data, you'll need the following permissions: CONNECT (to the database), USAGE (on the schema), SELECT (on the tables)

Exporting data requires write access. For exporting data, you'll need the same access as importing plus access for following commands:

• CREATE (on the database, needed to write to a new database schema)

• INSERT (on individual tables, needed to append to an existing database schema)

• CREATE (on the schema, needed to overwrite an existing database schema)

• Additionally, to overwrite an existing database schema, the user must be an owner of the table.

For more information, see the Postgres reference guide.