Package org.apache.flink.cdc.connectors.postgres.source.reader

Class PostgresSourceReader

java.lang.Object

org.apache.flink.connector.base.source.reader.SourceReaderBase<E,T,SplitT,SplitStateT>

org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase<SourceRecords,T,SourceSplitBase,SourceSplitState>

org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceReader

org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceReaderWithCommit

org.apache.flink.cdc.connectors.postgres.source.reader.PostgresSourceReader

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SourceReader

@Experimental
public class PostgresSourceReader
extends IncrementalSourceReaderWithCommit

The multi-parallel postgres source reader for table snapshot phase from SnapshotSplit and then single-parallel source reader for table stream phase from StreamSplit.

If scan newly added table is enable, postgres reader will not commit offset until all new added tables' snapshot splits are finished.

Field Summary

Fields inherited from class org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceReaderWithCommit

lastCheckpointOffsets

Fields inherited from class org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceReader

dialect, incrementalSourceReaderContext, sourceConfig, suspendedStreamSplit, uncompletedStreamSplits

Fields inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

config, context, eofRecordEvaluator, options, recordEmitter, splitFetcherManager

Constructor Summary

Constructors

Constructor

Description

PostgresSourceReader(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue elementQueue, java.util.function.Supplier supplier, org.apache.flink.connector.base.source.reader.RecordEmitter recordEmitter, org.apache.flink.configuration.Configuration config, IncrementalSourceReaderContext incrementalSourceReaderContext, SourceConfig sourceConfig, SourceSplitSerializer sourceSplitSerializer, DataSourceDialect dialect)

Method Summary

Modifier and Type	Method	Description
void	handleSourceEvents(org.apache.flink.api.connector.source.SourceEvent sourceEvent)
protected SourceSplitState	initializedState(SourceSplitBase split)
void	notifyCheckpointComplete(long checkpointId)
protected void	onSplitFinished(Map finishedSplitIds)
List<SourceSplitBase>	snapshotState(long checkpointId)
protected void	updateStreamSplitFinishedSplitsSize(LatestFinishedSplitsNumberEvent sourceEvent)

Methods inherited from class org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceReader

addSplits, getFinishedUnackedSplits, getNextMetaGroupId, start, toSplitType

Methods inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

close, getNumberOfCurrentlyAssignedSplits, isAvailable, notifyNoMoreSplits, pauseOrResumeSplits, pollNext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Constructor Detail

PostgresSourceReader

public PostgresSourceReader(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue elementQueue,
                            java.util.function.Supplier supplier,
                            org.apache.flink.connector.base.source.reader.RecordEmitter recordEmitter,
                            org.apache.flink.configuration.Configuration config,
                            IncrementalSourceReaderContext incrementalSourceReaderContext,
                            SourceConfig sourceConfig,
                            SourceSplitSerializer sourceSplitSerializer,
                            DataSourceDialect dialect)

Method Detail

handleSourceEvents

public void handleSourceEvents(org.apache.flink.api.connector.source.SourceEvent sourceEvent)

Specified by:

handleSourceEvents in interface org.apache.flink.api.connector.source.SourceReader

Overrides:

handleSourceEvents in class IncrementalSourceReader

initializedState

protected SourceSplitState initializedState(SourceSplitBase split)

Overrides:

initializedState in class IncrementalSourceReader

updateStreamSplitFinishedSplitsSize

protected void updateStreamSplitFinishedSplitsSize(LatestFinishedSplitsNumberEvent sourceEvent)

Overrides:

updateStreamSplitFinishedSplitsSize in class IncrementalSourceReader

snapshotState

public List<SourceSplitBase> snapshotState(long checkpointId)

Specified by:

snapshotState in interface org.apache.flink.api.connector.source.SourceReader

Overrides:

snapshotState in class IncrementalSourceReaderWithCommit

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.common.state.CheckpointListener

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.connector.source.SourceReader

Overrides:

notifyCheckpointComplete in class IncrementalSourceReaderWithCommit

Throws:

Exception

onSplitFinished

protected void onSplitFinished(Map finishedSplitIds)

Overrides:

onSplitFinished in class IncrementalSourceReader