java.lang.Object
- org.apache.flink.cdc.connectors.postgres.source.PostgresSourceBuilder<T>

@Experimental
public class PostgresSourceBuilder<T>
extends Object

The source builder for PostgresIncrementalSource.

Nested Class Summary

Nested Classes
Modifier and Type Class Description

static class PostgresSourceBuilder.PostgresIncrementalSource<T>
The Postgres source based on the incremental snapshot framework.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`PostgresSourceBuilder<T>`	`assignUnboundedChunkFirst(boolean assignUnboundedChunkFirst)`	Whether the `PostgresSourceEnumerator` should assign the unbounded chunks first or not during snapshot reading phase.
`PostgresSourceBuilder.PostgresIncrementalSource<T>`	`build()`	Build the `PostgresSourceBuilder.PostgresIncrementalSource`.
`PostgresSourceBuilder<T>`	`chunkKeyColumn(String chunkKeyColumn)`	The chunk key of table snapshot, captured tables are split into multiple chunks by the chunk key column when read the snapshot of table.
`PostgresSourceBuilder<T>`	`closeIdleReaders(boolean closeIdleReaders)`	scan.incremental.close-idle-reader.enabled
`PostgresSourceBuilder<T>`	`connectionPoolSize(int connectionPoolSize)`	The connection pool size.
`PostgresSourceBuilder<T>`	`connectMaxRetries(int connectMaxRetries)`	The max retry times to get connection.
`PostgresSourceBuilder<T>`	`connectTimeout(java.time.Duration connectTimeout)`	The maximum time that the connector should wait after trying to connect to the Postgres database server before timing out.
`PostgresSourceBuilder<T>`	`database(String database)`	The name of the PostgreSQL database from which to stream the changes.
`PostgresSourceBuilder<T>`	`debeziumProperties(Properties properties)`	The Debezium Postgres connector properties.
`PostgresSourceBuilder<T>`	`decodingPluginName(String name)`	The name of the Postgres logical decoding plug-in installed on the server.
`PostgresSourceBuilder<T>`	`deserializer(DebeziumDeserializationSchema<T> deserializer)`	The deserializer used to convert from consumed `SourceRecord`.
`PostgresSourceBuilder<T>`	`distributionFactorLower(double distributionFactorLower)`	The lower bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.
`PostgresSourceBuilder<T>`	`distributionFactorUpper(double distributionFactorUpper)`	The upper bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.
`PostgresSourceBuilder<T>`	`fetchSize(int fetchSize)`	The maximum fetch size for per poll when read table snapshot.
`PostgresSourceConfigFactory`	`getConfigFactory()`
`PostgresSourceBuilder<T>`	`heartbeatInterval(java.time.Duration heartbeatInterval)`	The heartbeat interval for the Postgres server.
`PostgresSourceBuilder<T>`	`hostname(String hostname)`	The hostname of the database to monitor for changes.
`PostgresSourceBuilder<T>`	`includeDatabaseInTableId(boolean includeDatabaseInTableId)`	Whether to include database in the generated Table ID.
`PostgresSourceBuilder<T>`	`includeLogicalMessages(List<String> prefixes)`	Enable emitting Postgres logical decoding messages (records produced by `pg_logical_emit_message`) to the deserializer.
`PostgresSourceBuilder<T>`	`includePartitionedTables(boolean includePartitionedTables)`	Whether the connector should read partitioned tables via partition root or not.
`PostgresSourceBuilder<T>`	`includeSchemaChanges(boolean includeSchemaChanges)`	Whether to infer schema change event on relation message.
`PostgresSourceBuilder<T>`	`lsnCommitCheckpointsDelay(int lsnCommitDelay)`	Set the `LSN` checkpoints delay number for Postgres to commit the offsets.
`PostgresSourceBuilder<T>`	`password(String password)`	Password to use when connecting to the Postgres database server.
`PostgresSourceBuilder<T>`	`port(int port)`	Integer port number of the Postgres database server.
`PostgresSourceBuilder<T>`	`releaseSnapshotMetadataEnabled(boolean releaseSnapshotMetadataEnabled)`	Whether to release the finished snapshot split metadata from the coordinator after entering the stream phase, to reduce JobManager memory pressure (FLINK-40697).
`PostgresSourceBuilder<T>`	`scanNewlyAddedTableEnabled(boolean scanNewlyAddedTableEnabled)`	Whether the `PostgresSourceEnumerator` should scan the newly added tables or not.
`PostgresSourceBuilder<T>`	`schemaList(String... schemaList)`	An required list of regular expressions that match database names to be monitored; any database name not included in the whitelist will be excluded from monitoring.
`PostgresSourceBuilder<T>`	`serverTimeZone(String timeZone)`	The session time zone in database server, e.g.
`PostgresSourceBuilder<T>`	`skipSnapshotBackfill(boolean skipSnapshotBackfill)`	Whether to skip backfill in snapshot reading phase.
`PostgresSourceBuilder<T>`	`slotName(String slotName)`	The name of the PostgreSQL logical decoding slot that was created for streaming changes from a particular plug-in for a particular database/schema.
`PostgresSourceBuilder<T>`	`splitMetaGroupSize(int splitMetaGroupSize)`	The group size of split meta, if the meta size exceeds the group size, the meta will be divided into multiple groups.
`PostgresSourceBuilder<T>`	`splitSize(int splitSize)`	The split size (number of rows) of table snapshot, captured tables are split into multiple splits when read the snapshot of table.
`PostgresSourceBuilder<T>`	`startupOptions(StartupOptions startupOptions)`	Specifies the startup options.
`PostgresSourceBuilder<T>`	`tableList(String... tableList)`	An required list of regular expressions that match fully-qualified table identifiers for tables to be monitored; any table not included in the list will be excluded from monitoring.
`PostgresSourceBuilder<T>`	`username(String username)`	Name of the Postgres database to use when connecting to the Postgres database server.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Method Detail
  - decodingPluginName
```
public PostgresSourceBuilder<T> decodingPluginName(String name)
```
    The name of the Postgres logical decoding plug-in installed on the server. Supported values are decoderbufs, wal2json, wal2json_rds, wal2json_streaming, wal2json_rds_streaming and pgoutput.
  - hostname
```
public PostgresSourceBuilder<T> hostname(String hostname)
```
    The hostname of the database to monitor for changes.
  - port
```
public PostgresSourceBuilder<T> port(int port)
```
    Integer port number of the Postgres database server.
  - database
```
public PostgresSourceBuilder<T> database(String database)
```
    The name of the PostgreSQL database from which to stream the changes.
  - schemaList
```
public PostgresSourceBuilder<T> schemaList(String... schemaList)
```
    An required list of regular expressions that match database names to be monitored; any database name not included in the whitelist will be excluded from monitoring.
  - tableList
```
public PostgresSourceBuilder<T> tableList(String... tableList)
```
    An required list of regular expressions that match fully-qualified table identifiers for tables to be monitored; any table not included in the list will be excluded from monitoring. Each identifier is of the form <schemaName>.<tableName>.
  - username
```
public PostgresSourceBuilder<T> username(String username)
```
    Name of the Postgres database to use when connecting to the Postgres database server.
  - password
```
public PostgresSourceBuilder<T> password(String password)
```
    Password to use when connecting to the Postgres database server.
  - slotName
```
public PostgresSourceBuilder<T> slotName(String slotName)
```
    The name of the PostgreSQL logical decoding slot that was created for streaming changes from a particular plug-in for a particular database/schema. The server uses this slot to stream events to the connector that you are configuring. Default is "flink".
    Slot names must conform to PostgreSQL replication slot naming rules, which state: "Each replication slot has a name, which can contain lower-case letters, numbers, and the underscore character."
  - serverTimeZone
```
public PostgresSourceBuilder<T> serverTimeZone(String timeZone)
```
    The session time zone in database server, e.g. "America/Los_Angeles". It controls how the TIMESTAMP type in MYSQL converted to STRING. See more https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-temporal-types
  - splitSize
```
public PostgresSourceBuilder<T> splitSize(int splitSize)
```
    The split size (number of rows) of table snapshot, captured tables are split into multiple splits when read the snapshot of table.
  - splitMetaGroupSize
```
public PostgresSourceBuilder<T> splitMetaGroupSize(int splitMetaGroupSize)
```
    The group size of split meta, if the meta size exceeds the group size, the meta will be divided into multiple groups.
  - distributionFactorUpper
```
public PostgresSourceBuilder<T> distributionFactorUpper(double distributionFactorUpper)
```
    The upper bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.
  - distributionFactorLower
```
public PostgresSourceBuilder<T> distributionFactorLower(double distributionFactorLower)
```
    The lower bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.
  - fetchSize
```
public PostgresSourceBuilder<T> fetchSize(int fetchSize)
```
    The maximum fetch size for per poll when read table snapshot.
  - connectTimeout
```
public PostgresSourceBuilder<T> connectTimeout(java.time.Duration connectTimeout)
```
    The maximum time that the connector should wait after trying to connect to the Postgres database server before timing out.
  - connectMaxRetries
```
public PostgresSourceBuilder<T> connectMaxRetries(int connectMaxRetries)
```
    The max retry times to get connection.
  - connectionPoolSize
```
public PostgresSourceBuilder<T> connectionPoolSize(int connectionPoolSize)
```
    The connection pool size.
  - startupOptions
```
public PostgresSourceBuilder<T> startupOptions(StartupOptions startupOptions)
```
    Specifies the startup options.
  - chunkKeyColumn
```
public PostgresSourceBuilder<T> chunkKeyColumn(String chunkKeyColumn)
```
    The chunk key of table snapshot, captured tables are split into multiple chunks by the chunk key column when read the snapshot of table.
  - debeziumProperties
```
public PostgresSourceBuilder<T> debeziumProperties(Properties properties)
```
    The Debezium Postgres connector properties. For example, "snapshot.mode".
  - closeIdleReaders
```
public PostgresSourceBuilder<T> closeIdleReaders(boolean closeIdleReaders)
```
    scan.incremental.close-idle-reader.enabled
    Whether to close idle readers at the end of the snapshot phase. This feature depends on FLIP-147: Support Checkpoints After Tasks Finished. The flink version is required to be greater than or equal to 1.14, and the configuration 'execution.checkpointing.checkpoints-after-tasks-finish.enabled' needs to be set to true.
    See more https://cwiki.apache.org/confluence/display/FLINK/FLIP-147%3A+Support+Checkpoints+After+Tasks+Finished.
  - deserializer
```
public PostgresSourceBuilder<T> deserializer(DebeziumDeserializationSchema<T> deserializer)
```
    The deserializer used to convert from consumed SourceRecord.
  - heartbeatInterval
```
public PostgresSourceBuilder<T> heartbeatInterval(java.time.Duration heartbeatInterval)
```
    The heartbeat interval for the Postgres server.
  - skipSnapshotBackfill
```
public PostgresSourceBuilder<T> skipSnapshotBackfill(boolean skipSnapshotBackfill)
```
    Whether to skip backfill in snapshot reading phase.
    If backfill is skipped, changes on captured tables during snapshot phase will be consumed later in stream reading phase instead of being merged into the snapshot.
    WARNING: Skipping backfill might lead to data inconsistency because some stream events happened within the snapshot phase might be replayed (only at-least-once semantic is promised). For example updating an already updated value in snapshot, or deleting an already deleted entry in snapshot. These replayed stream events should be handled specially.
  - scanNewlyAddedTableEnabled
```
public PostgresSourceBuilder<T> scanNewlyAddedTableEnabled(boolean scanNewlyAddedTableEnabled)
```
    Whether the PostgresSourceEnumerator should scan the newly added tables or not.
  - assignUnboundedChunkFirst
```
public PostgresSourceBuilder<T> assignUnboundedChunkFirst(boolean assignUnboundedChunkFirst)
```
    Whether the PostgresSourceEnumerator should assign the unbounded chunks first or not during snapshot reading phase.
  - releaseSnapshotMetadataEnabled
```
public PostgresSourceBuilder<T> releaseSnapshotMetadataEnabled(boolean releaseSnapshotMetadataEnabled)
```
    Whether to release the finished snapshot split metadata from the coordinator after entering the stream phase, to reduce JobManager memory pressure (FLINK-40697).
  - lsnCommitCheckpointsDelay
```
public PostgresSourceBuilder<T> lsnCommitCheckpointsDelay(int lsnCommitDelay)
```
    Set the LSN checkpoints delay number for Postgres to commit the offsets.
  - includePartitionedTables
```
public PostgresSourceBuilder<T> includePartitionedTables(boolean includePartitionedTables)
```
    Whether the connector should read partitioned tables via partition root or not.
  - includeDatabaseInTableId
```
public PostgresSourceBuilder<T> includeDatabaseInTableId(boolean includeDatabaseInTableId)
```
    Whether to include database in the generated Table ID.
  - includeLogicalMessages
```
public PostgresSourceBuilder<T> includeLogicalMessages(List<String> prefixes)
```
    Enable emitting Postgres logical decoding messages (records produced by pg_logical_emit_message) to the deserializer.
    Unlike normal table changes, logical messages are not bound to any table and are not filtered by PUBLICATION. They are written as standalone LOGICAL MESSAGE records in the WAL and streamed to all replication slots created with the pgoutput plugin. Both transactional (pg_logical_emit_message(true, ...)) and non-transactional (pg_logical_emit_message(false, ...)) messages are delivered.
    Once enabled, this connector performs client-side prefix filtering: only messages whose prefix starts with one of the given prefixes are forwarded to the deserializer; messages with non-matching prefixes are silently dropped.
    Requirements:
    
    PostgreSQL 14+ with wal_level=logical configured in postgresql.conf (the pgoutput plugin started supporting the messages streaming option since PG 14).
    The decoding plugin must be pgoutput (configured via decodingPluginName(java.lang.String)).
    prefixes must be non-null and non-empty; otherwise no logical message will be emitted.
    Parameters:
    
    prefixes - the list of message prefixes to be included; messages whose prefix matches any of these will be emitted, all others are dropped
  - includeSchemaChanges
```
public PostgresSourceBuilder<T> includeSchemaChanges(boolean includeSchemaChanges)
```
    Whether to infer schema change event on relation message.
  - build
```
public PostgresSourceBuilder.PostgresIncrementalSource<T> build()
```
    Build the PostgresSourceBuilder.PostgresIncrementalSource.
    
    Returns:
    
    a PostgresParallelSource with the settings made for this builder.
  - getConfigFactory
```
public PostgresSourceConfigFactory getConfigFactory()
```

Class PostgresSourceBuilder<T>

Nested Class Summary

Method Summary

Methods inherited from class java.lang.Object

Method Detail

decodingPluginName

hostname

port

database

schemaList

tableList

username

password

slotName

serverTimeZone

splitSize

splitMetaGroupSize

distributionFactorUpper

distributionFactorLower

fetchSize

connectTimeout

connectMaxRetries

connectionPoolSize

startupOptions

chunkKeyColumn

debeziumProperties

closeIdleReaders

deserializer

heartbeatInterval

skipSnapshotBackfill

scanNewlyAddedTableEnabled

assignUnboundedChunkFirst

releaseSnapshotMetadataEnabled

lsnCommitCheckpointsDelay

includePartitionedTables

includeDatabaseInTableId

includeLogicalMessages

includeSchemaChanges

build

getConfigFactory