Skip to main content
Feedback

PostgreSQL walkthrough

Working with PostgreSQL

Step 1: establishing a connection

Make sure you have appropriate credentials for establishing a connection with a PostgreSQL database server.

Step 2: choosing the River mode

Once you have successfully established a connection to PostgreSQL, select the River mode to load data into a Target. You can choose from three available options:

note

By clicking on the highlighted modes, you are redirected to a relevant page that offers a comprehensive explanation and presents a Product Tour example showcasing the available features in PostgreSQL.

  • Multi-Tables: Simultaneously load multiple tables from PostgreSQL to your desired Target.
    • Standard Extraction - This River mode maps, transforms, and loads data from multiple tables into a unified schema. It uses SQL queries for transformations and can be scheduled or manually triggered.
    • Change Data Capture (CDC) - This mode monitors the Source database logs, capturing and transforming changes in real-time. This ensures minimal data loss and low-latency transfer when loading the transformed data into the target database, keeping it in sync with the Source.
note
  • TimescaleDB is an open-source database specifically crafted to enhance the scalability of SQL for time-series data.

    Data Integration does not provide support for Timescale.

  • For both Standard extraction and CDC, the default TIMESTAMP format is TIMESTAMP without a Timezone. Depending on your requirements, you can opt for TIMESTAMP with a Timezone offset in UTC. This option is available in the Table Settings tab under TIMESTAMP Format.
  • Custom Query: Create a customized query and import it into your designated Target.
  • Legacy River: Select a sole Source table for loading into a Target.

Step 3: running the River

Once you have successfully configured the River mode, selected the extraction method, and set up the scheduling, you are ready to initiate the River and its operation.

Partitioned tables in PostgreSQL

Data Integration supports PostgreSQL partitioned tables in both Standard Extraction and Change Data Capture (CDC).

On this Page