Class JdbcSourceConfigFactory
- java.lang.Object
-
- org.apache.flink.cdc.connectors.base.config.JdbcSourceConfigFactory
-
- All Implemented Interfaces:
Serializable,SourceConfig.Factory<JdbcSourceConfig>
- Direct Known Subclasses:
Db2SourceConfigFactory,OracleSourceConfigFactory,PostgresSourceConfigFactory,SqlServerSourceConfigFactory
@Internal public abstract class JdbcSourceConfigFactory extends Object implements SourceConfig.Factory<JdbcSourceConfig>
ASourceConfig.Factoryto provideSourceConfigof JDBC data source.- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description protected booleanassignUnboundedChunkFirstprotected StringchunkKeyColumnprotected booleancloseIdleReadersprotected intconnectionPoolSizeprotected intconnectMaxRetriesprotected java.time.DurationconnectTimeoutprotected List<String>databaseListprotected PropertiesdbzPropertiesprotected doubledistributionFactorLowerprotected doubledistributionFactorUpperprotected intfetchSizeprotected Stringhostnameprotected booleanincludeSchemaChangesprotected Stringpasswordprotected intportprotected booleanreleaseSnapshotMetadataEnabledprotected booleanscanNewlyAddedTableEnabledprotected StringserverTimeZoneprotected booleanskipSnapshotBackfillprotected intsplitMetaGroupSizeprotected intsplitSizeprotected StartupOptionsstartupOptionsprotected List<String>tableListprotected Stringusername
-
Constructor Summary
Constructors Constructor Description JdbcSourceConfigFactory()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description JdbcSourceConfigFactoryassignUnboundedChunkFirst(boolean assignUnboundedChunkFirst)Whether to assign the unbounded chunks first during snapshot reading phase.JdbcSourceConfigFactorychunkKeyColumn(String chunkKeyColumn)The chunk key of table snapshot, captured tables are split into multiple chunks by the chunk key column when read the snapshot of table.JdbcSourceConfigFactorycloseIdleReaders(boolean closeIdleReaders)Whether to close idle readers at the end of the snapshot phase.JdbcSourceConfigFactoryconnectionPoolSize(int connectionPoolSize)The connection pool size.JdbcSourceConfigFactoryconnectMaxRetries(int connectMaxRetries)The max retry times to get connection.JdbcSourceConfigFactoryconnectTimeout(java.time.Duration connectTimeout)The maximum time that the connector should wait after trying to connect to the database server before timing out.abstract JdbcSourceConfigcreate(int subtask)JdbcSourceConfigFactorydatabaseList(String... databaseList)An optional list of regular expressions that match database names to be monitored; any database name not included in the whitelist will be excluded from monitoring.JdbcSourceConfigFactorydebeziumProperties(Properties properties)The Debezium connector properties.JdbcSourceConfigFactorydistributionFactorLower(double distributionFactorLower)The lower bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.JdbcSourceConfigFactorydistributionFactorUpper(double distributionFactorUpper)The upper bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.JdbcSourceConfigFactoryfetchSize(int fetchSize)The maximum fetch size for per poll when read table snapshot.JdbcSourceConfigFactoryhostname(String hostname)Integer port number of the database server.JdbcSourceConfigFactoryincludeSchemaChanges(boolean includeSchemaChanges)Whether theSourceConfigshould output the schema changes or not.JdbcSourceConfigFactorypassword(String password)Password to use when connecting to the database server.JdbcSourceConfigFactoryport(int port)Integer port number of the database server.JdbcSourceConfigFactoryreleaseSnapshotMetadataEnabled(boolean releaseSnapshotMetadataEnabled)Whether to release the snapshot split metadata held by the source coordinator once the stream split is assembled and checkpoint-covered.JdbcSourceConfigFactoryscanNewlyAddedTableEnabled(boolean scanNewlyAddedTableEnabled)Whether theSourceConfigshould scan the newly added tables or not.JdbcSourceConfigFactoryserverTimeZone(String timeZone)The session time zone in database server, e.g.JdbcSourceConfigFactoryskipSnapshotBackfill(boolean skipSnapshotBackfill)Whether to skip backfill in snapshot reading phase.JdbcSourceConfigFactorysplitMetaGroupSize(int splitMetaGroupSize)The group size of split meta, if the meta size exceeds the group size, the meta will be divided into multiple groups.JdbcSourceConfigFactorysplitSize(int splitSize)The split size (number of rows) of table snapshot, captured tables are split into multiple splits when read the snapshot of table.JdbcSourceConfigFactorystartupOptions(StartupOptions startupOptions)Specifies the startup options.JdbcSourceConfigFactorytableList(String... tableList)An optional list of regular expressions that match fully-qualified table identifiers for tables to be monitored; any table not included in the list will be excluded from monitoring.JdbcSourceConfigFactoryusername(String username)Name of the user to use when connecting to the database server.
-
-
-
Field Detail
-
port
protected int port
-
hostname
protected String hostname
-
username
protected String username
-
password
protected String password
-
startupOptions
protected StartupOptions startupOptions
-
includeSchemaChanges
protected boolean includeSchemaChanges
-
closeIdleReaders
protected boolean closeIdleReaders
-
distributionFactorUpper
protected double distributionFactorUpper
-
distributionFactorLower
protected double distributionFactorLower
-
splitSize
protected int splitSize
-
splitMetaGroupSize
protected int splitMetaGroupSize
-
fetchSize
protected int fetchSize
-
serverTimeZone
protected String serverTimeZone
-
connectTimeout
protected java.time.Duration connectTimeout
-
connectMaxRetries
protected int connectMaxRetries
-
connectionPoolSize
protected int connectionPoolSize
-
dbzProperties
protected Properties dbzProperties
-
chunkKeyColumn
protected String chunkKeyColumn
-
skipSnapshotBackfill
protected boolean skipSnapshotBackfill
-
scanNewlyAddedTableEnabled
protected boolean scanNewlyAddedTableEnabled
-
assignUnboundedChunkFirst
protected boolean assignUnboundedChunkFirst
-
releaseSnapshotMetadataEnabled
protected boolean releaseSnapshotMetadataEnabled
-
-
Method Detail
-
hostname
public JdbcSourceConfigFactory hostname(String hostname)
Integer port number of the database server.
-
port
public JdbcSourceConfigFactory port(int port)
Integer port number of the database server.
-
databaseList
public JdbcSourceConfigFactory databaseList(String... databaseList)
An optional list of regular expressions that match database names to be monitored; any database name not included in the whitelist will be excluded from monitoring. By default all databases will be monitored.
-
tableList
public JdbcSourceConfigFactory tableList(String... tableList)
An optional list of regular expressions that match fully-qualified table identifiers for tables to be monitored; any table not included in the list will be excluded from monitoring. Each identifier is of t、he form databaseName.tableName. By default the connector will monitor every non-system table in each monitored database.
-
username
public JdbcSourceConfigFactory username(String username)
Name of the user to use when connecting to the database server.
-
password
public JdbcSourceConfigFactory password(String password)
Password to use when connecting to the database server.
-
serverTimeZone
public JdbcSourceConfigFactory serverTimeZone(String timeZone)
The session time zone in database server, e.g. "America/Los_Angeles". It controls how the TIMESTAMP type converted to STRING. See more https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-temporal-types
-
splitSize
public JdbcSourceConfigFactory splitSize(int splitSize)
The split size (number of rows) of table snapshot, captured tables are split into multiple splits when read the snapshot of table.
-
splitMetaGroupSize
public JdbcSourceConfigFactory splitMetaGroupSize(int splitMetaGroupSize)
The group size of split meta, if the meta size exceeds the group size, the meta will be divided into multiple groups.
-
distributionFactorUpper
public JdbcSourceConfigFactory distributionFactorUpper(double distributionFactorUpper)
The upper bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.
-
distributionFactorLower
public JdbcSourceConfigFactory distributionFactorLower(double distributionFactorLower)
The lower bound of split key evenly distribution factor, the factor is used to determine whether the table is evenly distribution or not.
-
fetchSize
public JdbcSourceConfigFactory fetchSize(int fetchSize)
The maximum fetch size for per poll when read table snapshot.
-
connectTimeout
public JdbcSourceConfigFactory connectTimeout(java.time.Duration connectTimeout)
The maximum time that the connector should wait after trying to connect to the database server before timing out.
-
connectionPoolSize
public JdbcSourceConfigFactory connectionPoolSize(int connectionPoolSize)
The connection pool size.
-
connectMaxRetries
public JdbcSourceConfigFactory connectMaxRetries(int connectMaxRetries)
The max retry times to get connection.
-
includeSchemaChanges
public JdbcSourceConfigFactory includeSchemaChanges(boolean includeSchemaChanges)
Whether theSourceConfigshould output the schema changes or not.
-
debeziumProperties
public JdbcSourceConfigFactory debeziumProperties(Properties properties)
The Debezium connector properties. For example, "snapshot.mode".
-
chunkKeyColumn
public JdbcSourceConfigFactory chunkKeyColumn(String chunkKeyColumn)
The chunk key of table snapshot, captured tables are split into multiple chunks by the chunk key column when read the snapshot of table.
-
startupOptions
public JdbcSourceConfigFactory startupOptions(StartupOptions startupOptions)
Specifies the startup options.
-
closeIdleReaders
public JdbcSourceConfigFactory closeIdleReaders(boolean closeIdleReaders)
Whether to close idle readers at the end of the snapshot phase. This feature depends on FLIP-147: Support Checkpoints After Tasks Finished. The flink version is required to be greater than or equal to 1.14, and the configuration'execution.checkpointing.checkpoints-after-tasks-finish.enabled'needs to be set to true.See more https://cwiki.apache.org/confluence/display/FLINK/FLIP-147%3A+Support+Checkpoints+After+Tasks+Finished.
-
skipSnapshotBackfill
public JdbcSourceConfigFactory skipSnapshotBackfill(boolean skipSnapshotBackfill)
Whether to skip backfill in snapshot reading phase.If backfill is skipped, changes on captured tables during snapshot phase will be consumed later in stream reading phase instead of being merged into the snapshot.
WARNING: Skipping backfill might lead to data inconsistency because some change log events happened within the snapshot phase might be replayed (only at-least-once semantic is promised). For example updating an already updated value in snapshot, or deleting an already deleted entry in snapshot. These replayed change log events should be handled specially.
-
scanNewlyAddedTableEnabled
public JdbcSourceConfigFactory scanNewlyAddedTableEnabled(boolean scanNewlyAddedTableEnabled)
Whether theSourceConfigshould scan the newly added tables or not.
-
assignUnboundedChunkFirst
public JdbcSourceConfigFactory assignUnboundedChunkFirst(boolean assignUnboundedChunkFirst)
Whether to assign the unbounded chunks first during snapshot reading phase. Defaults to false.
-
releaseSnapshotMetadataEnabled
public JdbcSourceConfigFactory releaseSnapshotMetadataEnabled(boolean releaseSnapshotMetadataEnabled)
Whether to release the snapshot split metadata held by the source coordinator once the stream split is assembled and checkpoint-covered. Defaults to false.
-
create
public abstract JdbcSourceConfig create(int subtask)
- Specified by:
createin interfaceSourceConfig.Factory<JdbcSourceConfig>
-
-