Package io.debezium.connector.mysql

Class MySqlSnapshotChangeEventSource

java.lang.Object

io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<P,O>

io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

io.debezium.connector.mysql.MySqlSnapshotChangeEventSource

All Implemented Interfaces:

io.debezium.pipeline.source.spi.ChangeEventSource, io.debezium.pipeline.source.spi.SnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

public class MySqlSnapshotChangeEventSource
extends io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Copied from Debezium project(1.9.8.Final) to fix MySQL 8.x compatibility.

Line 338: Use probing methods to determine the statement.

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.relational.RelationalSnapshotChangeEventSource

io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<P extends io.debezium.pipeline.spi.Partition,O extends io.debezium.pipeline.spi.OffsetContext>

Nested classes/interfaces inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshotContext<P extends io.debezium.pipeline.spi.Partition,O extends io.debezium.pipeline.spi.OffsetContext>, io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshottingTask

Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource

io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext

Field Summary

Fields inherited from class io.debezium.relational.RelationalSnapshotChangeEventSource

clock, dispatcher, SELECT_ALL_PATTERN

Fields inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

LOG_INTERVAL

Constructor Summary

Constructors

Constructor

Description

MySqlSnapshotChangeEventSource(io.debezium.connector.mysql.MySqlConnectorConfig connectorConfig, MySqlConnection connection, io.debezium.connector.mysql.MySqlDatabaseSchema schema, io.debezium.pipeline.EventDispatcher<io.debezium.connector.mysql.MySqlPartition,io.debezium.relational.TableId> dispatcher, io.debezium.util.Clock clock, io.debezium.connector.mysql.MySqlSnapshotChangeEventSourceMetrics metrics, io.debezium.function.BlockingConsumer<java.util.function.Function<org.apache.kafka.connect.source.SourceRecord,org.apache.kafka.connect.source.SourceRecord>> lastEventProcessor)

Method Summary

Modifier and Type	Method	Description
protected void	complete(io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
protected void	connectionCreated(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
protected void	createSchemaChangeEventsForTables(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext sourceContext, io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext, io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask)
protected void	determineSnapshotOffset(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> ctx, io.debezium.connector.mysql.MySqlOffsetContext previousOffset)
protected Set<io.debezium.relational.TableId>	getAllTableIds(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> ctx)
protected io.debezium.schema.SchemaChangeEvent	getCreateTableEvent(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext, io.debezium.relational.Table table)
protected Optional<String>	getSnapshotSelect(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext, io.debezium.relational.TableId tableId, List<String> columns)	Generate a valid MySQL query string for the specified table and columns
protected io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshottingTask	getSnapshottingTask(io.debezium.connector.mysql.MySqlPartition partition, io.debezium.connector.mysql.MySqlOffsetContext previousOffset)
protected void	lastSnapshotRecord(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
protected void	lockTablesForSchemaSnapshot(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext sourceContext, io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
protected void	postSnapshot()
protected io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>	prepare(io.debezium.connector.mysql.MySqlPartition partition)
protected Statement	readTableStatement(OptionalLong rowCount)
protected void	readTableStructure(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext sourceContext, io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext, io.debezium.connector.mysql.MySqlOffsetContext offsetContext)
protected void	releaseDataSnapshotLocks(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
protected void	releaseSchemaSnapshotLocks(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
protected OptionalLong	rowCountForTable(io.debezium.relational.TableId tableId)

Methods inherited from class io.debezium.relational.RelationalSnapshotChangeEventSource

additionalColumnFilter, createSnapshotConnection, doExecute, enhanceOverriddenSelect, getChangeRecordEmitter, getClock, getColumnValue, getPreparedColumnNames, getSnapshotSourceTimestamp, schema, tryStartingSnapshot

Methods inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

delaySnapshotIfNeeded, determineDataCollectionsToBeSnapshotted, execute

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MySqlSnapshotChangeEventSource

public MySqlSnapshotChangeEventSource(io.debezium.connector.mysql.MySqlConnectorConfig connectorConfig,
                                      MySqlConnection connection,
                                      io.debezium.connector.mysql.MySqlDatabaseSchema schema,
                                      io.debezium.pipeline.EventDispatcher<io.debezium.connector.mysql.MySqlPartition,io.debezium.relational.TableId> dispatcher,
                                      io.debezium.util.Clock clock,
                                      io.debezium.connector.mysql.MySqlSnapshotChangeEventSourceMetrics metrics,
                                      io.debezium.function.BlockingConsumer<java.util.function.Function<org.apache.kafka.connect.source.SourceRecord,org.apache.kafka.connect.source.SourceRecord>> lastEventProcessor)

Method Detail

getSnapshottingTask

protected io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshottingTask getSnapshottingTask(io.debezium.connector.mysql.MySqlPartition partition,
                                                                                                             io.debezium.connector.mysql.MySqlOffsetContext previousOffset)

Specified by:

getSnapshottingTask in class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

prepare

protected io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> prepare(io.debezium.connector.mysql.MySqlPartition partition)
                                                                                                                                                                                          throws Exception

Specified by:

prepare in class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

connectionCreated

protected void connectionCreated(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
                          throws Exception

Overrides:

connectionCreated in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

getAllTableIds

protected Set<io.debezium.relational.TableId> getAllTableIds(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> ctx)
                                                      throws Exception

Specified by:

getAllTableIds in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

lockTablesForSchemaSnapshot

protected void lockTablesForSchemaSnapshot(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext sourceContext,
                                           io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
                                    throws SQLException,
                                           InterruptedException

Specified by:

lockTablesForSchemaSnapshot in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

SQLException

InterruptedException

releaseSchemaSnapshotLocks

protected void releaseSchemaSnapshotLocks(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
                                   throws SQLException

Specified by:

releaseSchemaSnapshotLocks in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

SQLException

releaseDataSnapshotLocks

protected void releaseDataSnapshotLocks(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)
                                 throws Exception

Overrides:

releaseDataSnapshotLocks in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

determineSnapshotOffset

protected void determineSnapshotOffset(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> ctx,
                                       io.debezium.connector.mysql.MySqlOffsetContext previousOffset)
                                throws Exception

Specified by:

determineSnapshotOffset in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

readTableStructure

protected void readTableStructure(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext sourceContext,
                                  io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext,
                                  io.debezium.connector.mysql.MySqlOffsetContext offsetContext)
                           throws Exception

Specified by:

readTableStructure in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

getCreateTableEvent

protected io.debezium.schema.SchemaChangeEvent getCreateTableEvent(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext,
                                                                   io.debezium.relational.Table table)
                                                            throws SQLException

Specified by:

getCreateTableEvent in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

SQLException

complete

protected void complete(io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)

Overrides:

complete in class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

getSnapshotSelect

protected Optional<String> getSnapshotSelect(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext,
                                             io.debezium.relational.TableId tableId,
                                             List<String> columns)

Generate a valid MySQL query string for the specified table and columns

Specified by:

getSnapshotSelect in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Parameters:

tableId - the table to generate a query for

Returns:

a valid query string

rowCountForTable

protected OptionalLong rowCountForTable(io.debezium.relational.TableId tableId)

Overrides:

rowCountForTable in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

readTableStatement

protected Statement readTableStatement(OptionalLong rowCount)
                                throws SQLException

Overrides:

readTableStatement in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

SQLException

createSchemaChangeEventsForTables

protected void createSchemaChangeEventsForTables(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext sourceContext,
                                                 io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext,
                                                 io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask)
                                          throws Exception

Overrides:

createSchemaChangeEventsForTables in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

Exception

lastSnapshotRecord

protected void lastSnapshotRecord(io.debezium.relational.RelationalSnapshotChangeEventSource.RelationalSnapshotContext<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext> snapshotContext)

Overrides:

lastSnapshotRecord in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

postSnapshot

protected void postSnapshot()
                     throws InterruptedException

Overrides:

postSnapshot in class io.debezium.relational.RelationalSnapshotChangeEventSource<io.debezium.connector.mysql.MySqlPartition,io.debezium.connector.mysql.MySqlOffsetContext>

Throws:

InterruptedException