Package io.debezium.connector.oracle.logminer.processor

Class AbstractLogMinerEventProcessor<T extends io.debezium.connector.oracle.logminer.processor.AbstractTransaction>

  • All Implemented Interfaces:
    io.debezium.connector.oracle.logminer.processor.LogMinerEventProcessor, AutoCloseable
    Direct Known Subclasses:
    io.debezium.connector.oracle.logminer.processor.infinispan.AbstractInfinispanLogMinerEventProcessor, io.debezium.connector.oracle.logminer.processor.memory.MemoryLogMinerEventProcessor
    public abstract class AbstractLogMinerEventProcessor<T extends io.debezium.connector.oracle.logminer.processor.AbstractTransaction>
extends Object
implements io.debezium.connector.oracle.logminer.processor.LogMinerEventProcessor
    Copied from Debezium 1.9.8.Final.

    An abstract implementation of LogMinerEventProcessor that all processors should extend.

    Lines 411, 423: pass ROWID to LogMinerChangeRecordEmitter.

    • Constructor Summary

      Constructors 
      Constructor Description
      AbstractLogMinerEventProcessor​(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context, io.debezium.connector.oracle.OracleConnectorConfig connectorConfig, io.debezium.connector.oracle.OracleDatabaseSchema schema, io.debezium.connector.oracle.OraclePartition partition, io.debezium.connector.oracle.OracleOffsetContext offsetContext, io.debezium.pipeline.EventDispatcher<io.debezium.connector.oracle.OraclePartition,​io.debezium.relational.TableId> dispatcher, io.debezium.connector.oracle.OracleStreamingChangeEventSourceMetrics metrics)  

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      protected abstract void addToTransaction​(String transactionId, io.debezium.connector.oracle.logminer.events.LogMinerEventRow row, java.util.function.Supplier<io.debezium.connector.oracle.logminer.events.LogMinerEvent> eventSupplier)
      Add a transaction to the transaction map if allowed.
      protected abstract io.debezium.connector.oracle.Scn calculateNewStartScn​(io.debezium.connector.oracle.Scn endScn, io.debezium.connector.oracle.Scn maxCommittedScn)
      Calculates the new starting system change number based on the current processing range.
      protected abstract PreparedStatement createQueryStatement()
      Create the JDBC query that will be used to fetch the mining result set.
      protected abstract T createTransaction​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Creates a new transaction based on the supplied START event.
      protected abstract void finalizeTransactionCommit​(String transactionId, io.debezium.connector.oracle.Scn commitScn)
      Finalizes the commit of a transaction.
      protected abstract void finalizeTransactionRollback​(String transactionId, io.debezium.connector.oracle.Scn rollbackScn)
      Finalizes the rollback the specified transaction.
      protected abstract T getAndRemoveTransactionFromCache​(String transactionId)
      Gets a transaction instance from the transaction cache while also removing its cache entry.
      protected io.debezium.connector.oracle.OracleConnectorConfig getConfig()  
      protected io.debezium.connector.oracle.Scn getLastProcessedScn()
      Return the last processed system change number handled by the processor.
      protected io.debezium.connector.oracle.OracleDatabaseSchema getSchema()  
      protected abstract Map<String,​T> getTransactionCache()
      Returns the TransactionCache implementation.
      protected abstract io.debezium.connector.oracle.Scn getTransactionCacheMinimumScn()
      Gets the minimum system change number stored in the transaction cache.
      protected abstract int getTransactionEventCount​(T transaction)
      Returns the number of events associated with the specified transaction.
      protected abstract Iterator<io.debezium.connector.oracle.logminer.events.LogMinerEvent> getTransactionEventIterator​(T transaction)
      Get an iterator over the events that are part of the specified transaction.
      protected String getTransactionIdPrefix​(String transactionId)  
      protected void handleCommit​(io.debezium.connector.oracle.OraclePartition partition, io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a COMMIT event.
      protected void handleDataEvent​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a INSERT, UPDATE, or DELETE event.
      protected void handleLobWrite​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a LOB_WRITE event.
      protected void handleMissingScn​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a MISSING_SCN event.
      protected void handleRollback​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a ROLLBACK event.
      protected void handleSchemaChange​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a DDL event.
      protected void handleSelectLobLocator​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.
      protected void handleStart​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a START event.
      protected void handleUnsupportedEvent​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)  
      protected boolean hasSchemaChangeBeenSeen​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Checks whether the LogMinerEvent row for a schema change can be emitted.
      protected boolean isRecentlyProcessed​(String transactionId)
      Check whether a transaction has been recently processed through either a commit or rollback.
      protected boolean isTransactionIdWithNoSequence​(String transactionId)
      Returns whether the transaction id has no sequence number component.
      protected boolean isTransactionUserExcluded​(T transaction)
      Check whether the supplied username associated with the specified transaction is excluded.
      protected boolean isTrxIdRawValue()  
      io.debezium.connector.oracle.Scn process​(io.debezium.connector.oracle.OraclePartition partition, io.debezium.connector.oracle.Scn startScn, io.debezium.connector.oracle.Scn endScn)  
      protected void processResults​(io.debezium.connector.oracle.OraclePartition partition, ResultSet resultSet)
      Processes the LogMiner results.
      protected void processRow​(io.debezium.connector.oracle.OraclePartition partition, io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Processes a single LogMinerEventRow.
      protected abstract void removeEventWithRowId​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
      Removes a specific transaction event by database row identifier.
      protected abstract void removeTransactionAndEventsFromCache​(T transaction)
      Removes the transaction and all its associated event entries from the connector's caches.
      protected void warnPotentiallyStuckScn​(io.debezium.connector.oracle.Scn previousOffsetScn, Map<Integer,​io.debezium.connector.oracle.Scn> previousOffsetCommitScns)
      Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.

      • Methods inherited from interface io.debezium.connector.oracle.logminer.processor.LogMinerEventProcessor

        abandonTransactions

    • Constructor Detail

      • AbstractLogMinerEventProcessor

        public AbstractLogMinerEventProcessor​(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context,
                                      io.debezium.connector.oracle.OracleConnectorConfig connectorConfig,
                                      io.debezium.connector.oracle.OracleDatabaseSchema schema,
                                      io.debezium.connector.oracle.OraclePartition partition,
                                      io.debezium.connector.oracle.OracleOffsetContext offsetContext,
                                      io.debezium.pipeline.EventDispatcher<io.debezium.connector.oracle.OraclePartition,​io.debezium.relational.TableId> dispatcher,
                                      io.debezium.connector.oracle.OracleStreamingChangeEventSourceMetrics metrics)

    • Method Detail

      • getConfig

        protected io.debezium.connector.oracle.OracleConnectorConfig getConfig()

      • getSchema

        protected io.debezium.connector.oracle.OracleDatabaseSchema getSchema()

      • isRecentlyProcessed

        protected boolean isRecentlyProcessed​(String transactionId)
        Check whether a transaction has been recently processed through either a commit or rollback.
        Parameters:
        transactionId - the unique transaction id
        Returns:
        true if the transaction has been recently processed, false otherwise

      • hasSchemaChangeBeenSeen

        protected boolean hasSchemaChangeBeenSeen​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Checks whether the LogMinerEvent row for a schema change can be emitted.
        Parameters:
        row - the result set row
        Returns:
        true if the schema change has been seen, false otherwise.

      • getLastProcessedScn

        protected io.debezium.connector.oracle.Scn getLastProcessedScn()
        Return the last processed system change number handled by the processor.
        Returns:
        the last processed system change number, never null.

      • getTransactionCache

        protected abstract Map<String,​T> getTransactionCache()
        Returns the TransactionCache implementation.
        Returns:
        the transaction cache, never null

      • createTransaction

        protected abstract T createTransaction​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Creates a new transaction based on the supplied START event.
        Parameters:
        row - the event row, must not be null
        Returns:
        the implementation-specific Transaction instance

      • removeEventWithRowId

        protected abstract void removeEventWithRowId​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Removes a specific transaction event by database row identifier.
        Parameters:
        row - the event row that contains the row identifier, must not be null

      • getTransactionEventCount

        protected abstract int getTransactionEventCount​(T transaction)
        Returns the number of events associated with the specified transaction.
        Parameters:
        transaction - the transaction, must not be null
        Returns:
        the number of events in the transaction

      • isTrxIdRawValue

        protected boolean isTrxIdRawValue()

      • process

        public io.debezium.connector.oracle.Scn process​(io.debezium.connector.oracle.OraclePartition partition,
                                                io.debezium.connector.oracle.Scn startScn,
                                                io.debezium.connector.oracle.Scn endScn)
                                         throws SQLException,
                                                InterruptedException
        Specified by:
        process in interface io.debezium.connector.oracle.logminer.processor.LogMinerEventProcessor
        Throws:
        SQLException
        InterruptedException

      • createQueryStatement

        protected abstract PreparedStatement createQueryStatement()
                                                   throws SQLException
        Create the JDBC query that will be used to fetch the mining result set.
        Returns:
        a prepared query statement, never null
        Throws:
        SQLException - if a database exception occurred creating the statement

      • calculateNewStartScn

        protected abstract io.debezium.connector.oracle.Scn calculateNewStartScn​(io.debezium.connector.oracle.Scn endScn,
                                                                         io.debezium.connector.oracle.Scn maxCommittedScn)
                                                                  throws InterruptedException
        Calculates the new starting system change number based on the current processing range.
        Parameters:
        endScn - the end system change number for the previously mined range, never null
        maxCommittedScn - the maximum committed system change number, never null
        Returns:
        the system change number to start then next mining iteration from, never null
        Throws:
        InterruptedException - if the current thread is interrupted

      • processResults

        protected void processResults​(io.debezium.connector.oracle.OraclePartition partition,
                              ResultSet resultSet)
                       throws SQLException,
                              InterruptedException
        Processes the LogMiner results.
        Parameters:
        resultSet - the result set from a LogMiner query
        Throws:
        SQLException - if a database exception occurred
        InterruptedException - if the dispatcher was interrupted sending an event

      • processRow

        protected void processRow​(io.debezium.connector.oracle.OraclePartition partition,
                          io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
                   throws SQLException,
                          InterruptedException
        Processes a single LogMinerEventRow.
        Parameters:
        row - the event row, must not be null
        Throws:
        SQLException - if a database exception occurred
        InterruptedException - if the dispatcher was interrupted sending an event

      • handleMissingScn

        protected void handleMissingScn​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a MISSING_SCN event.
        Parameters:
        row - the result set row

      • handleStart

        protected void handleStart​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a START event.
        Parameters:
        row - the result set row

      • handleCommit

        protected void handleCommit​(io.debezium.connector.oracle.OraclePartition partition,
                            io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
                     throws InterruptedException
        Handle processing a LogMinerEventRow for a COMMIT event.
        Parameters:
        row - the result set row
        Throws:
        InterruptedException - if the event dispatcher was interrupted sending events

      • getAndRemoveTransactionFromCache

        protected abstract T getAndRemoveTransactionFromCache​(String transactionId)
        Gets a transaction instance from the transaction cache while also removing its cache entry.
        Parameters:
        transactionId - the transaction's unique identifier, should not be null
        Returns:
        the transaction instance if found, null if the transaction wasn't found

      • removeTransactionAndEventsFromCache

        protected abstract void removeTransactionAndEventsFromCache​(T transaction)
        Removes the transaction and all its associated event entries from the connector's caches.
        Parameters:
        transaction - the transaction instance, should never be null

      • getTransactionEventIterator

        protected abstract Iterator<io.debezium.connector.oracle.logminer.events.LogMinerEvent> getTransactionEventIterator​(T transaction)
        Get an iterator over the events that are part of the specified transaction.
        Parameters:
        transaction - the transaction instance, should never be null
        Returns:
        an iterator over the transaction's events, never null

      • finalizeTransactionCommit

        protected abstract void finalizeTransactionCommit​(String transactionId,
                                                  io.debezium.connector.oracle.Scn commitScn)
        Finalizes the commit of a transaction.
        Parameters:
        transactionId - the transaction's unique identifier, should not be null
        commitScn - the transaction's system change number, should not be null

      • isTransactionUserExcluded

        protected boolean isTransactionUserExcluded​(T transaction)
        Check whether the supplied username associated with the specified transaction is excluded.
        Parameters:
        transaction - the transaction, never null
        Returns:
        true if the transaction should be skipped; false if transaction should be emitted

      • handleRollback

        protected void handleRollback​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a ROLLBACK event.
        Parameters:
        row - the result set row

      • finalizeTransactionRollback

        protected abstract void finalizeTransactionRollback​(String transactionId,
                                                    io.debezium.connector.oracle.Scn rollbackScn)
        Finalizes the rollback the specified transaction.
        Parameters:
        transactionId - the unique transaction identifier, never null
        rollbackScn - the rollback transaction's system change number, never null

      • handleSchemaChange

        protected void handleSchemaChange​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
                           throws InterruptedException
        Handle processing a LogMinerEventRow for a DDL event.
        Parameters:
        row - the result set row
        Throws:
        InterruptedException - if the event dispatcher is interrupted sending the event

      • handleSelectLobLocator

        protected void handleSelectLobLocator​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.
        Parameters:
        row - the result set row

      • handleLobWrite

        protected void handleLobWrite​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a LOB_WRITE event.
        Parameters:
        row - the result set row

      • handleDataEvent

        protected void handleDataEvent​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)
                        throws SQLException,
                               InterruptedException
        Handle processing a LogMinerEventRow for a INSERT, UPDATE, or DELETE event.
        Parameters:
        row - the result set row
        Throws:
        SQLException - if a database exception occurs
        InterruptedException - if the dispatch of an event is interrupted

      • handleUnsupportedEvent

        protected void handleUnsupportedEvent​(io.debezium.connector.oracle.logminer.events.LogMinerEventRow row)

      • warnPotentiallyStuckScn

        protected void warnPotentiallyStuckScn​(io.debezium.connector.oracle.Scn previousOffsetScn,
                                       Map<Integer,​io.debezium.connector.oracle.Scn> previousOffsetCommitScns)
        Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.
        Parameters:
        previousOffsetScn - the previous offset system change number
        previousOffsetCommitScns - the previous offset commit system change number

      • addToTransaction

        protected abstract void addToTransaction​(String transactionId,
                                         io.debezium.connector.oracle.logminer.events.LogMinerEventRow row,
                                         java.util.function.Supplier<io.debezium.connector.oracle.logminer.events.LogMinerEvent> eventSupplier)
        Add a transaction to the transaction map if allowed.
        Parameters:
        transactionId - the unqiue transaction id
        row - the LogMiner event row
        eventSupplier - the supplier of the event to create if the event is allowed to be added

      • getTransactionCacheMinimumScn

        protected abstract io.debezium.connector.oracle.Scn getTransactionCacheMinimumScn()
        Gets the minimum system change number stored in the transaction cache.
        Returns:
        the minimum system change number, never null but could be Scn.NULL.

      • isTransactionIdWithNoSequence

        protected boolean isTransactionIdWithNoSequence​(String transactionId)
        Returns whether the transaction id has no sequence number component.

        Oracle transaction identifiers are a composite of:

        1. Undo segment number
        2. Slot numbber of the transaction that generated the change
        3. Sequence number of the transaction that generated the change

        When Oracle LogMiner mines records, it is possible that when an undo operation is detected, often the product of a constraint violation, the LogMiner row will have the same explicit XID (transaction id) as the source operation that we should undo; however, if the record to be undone was mined in a prior iteration, Oracle LogMiner won't be able to make a link back to the full transaction's sequence number, therefore the XID value for the undo row will contain only the undo segment number and slot number, setting the sequence number to 4294967295 (aka -1 or 0xFFFFFFFF).

        This method explicitly checks if the provided transaction id has the no sequence sentinel value and if so, returns true; otherwise returns false.

        Parameters:
        transactionId - the transaction identifier to check, should not be null
        Returns:
        true if the transaction has no sequence reference, false if it does

      • getTransactionIdPrefix

        protected String getTransactionIdPrefix​(String transactionId)