Package org.apache.flink.cdc.runtime.operators.sink

Class BatchDataSinkWriterOperator<CommT>

  • Type Parameters:
    CommT - the type of the committable (to send to downstream operators)
    All Implemented Interfaces:
    Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<Event>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<Event,​org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
    @Internal
public class BatchDataSinkWriterOperator<CommT>
extends org.apache.flink.cdc.runtime.operators.AbstractStreamOperatorAdapter<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<Event,​org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>, org.apache.flink.streaming.api.operators.BoundedOneInput
    An operator that processes records to be written into a Sink in batch mode.

    The operator is a proxy of SinkWriterOperator in Flink.

    The operator is always part of a sink pipeline and is the first operator.

    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        chainingStrategy, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

    • Constructor Summary

      Constructors 
      Constructor Description
      BatchDataSinkWriterOperator​(org.apache.flink.api.connector.sink2.Sink<Event> sink, org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeService, org.apache.flink.api.common.operators.MailboxExecutor mailboxExecutor)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      void endInput()  
      void initializeState​(org.apache.flink.runtime.state.StateInitializationContext context)  
      void open()  
      void prepareSnapshotPreBarrier​(long checkpointId)  
      void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<Event> element)  
      void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)  
      void processWatermarkStatus​(org.apache.flink.streaming.runtime.watermarkstatus.WatermarkStatus watermarkStatus)  
      void setup​(org.apache.flink.streaming.runtime.tasks.StreamTask<?,​?> containingTask, org.apache.flink.streaming.api.graph.StreamConfig config, org.apache.flink.streaming.api.operators.Output<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>> output)  
      void snapshotState​(org.apache.flink.runtime.state.StateSnapshotContext context)  

      • Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, snapshotState, useSplittableTimers

      • Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

        notifyCheckpointAborted, notifyCheckpointComplete

      • Methods inherited from interface org.apache.flink.streaming.api.operators.Input

        processLatencyMarker, processRecordAttributes

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

        getCurrentKey, setCurrentKey

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

        hasKeyContext

      • Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

        setKeyContextElement

      • Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

        finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

    • Constructor Detail

      • BatchDataSinkWriterOperator

        public BatchDataSinkWriterOperator​(org.apache.flink.api.connector.sink2.Sink<Event> sink,
                                   org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeService,
                                   org.apache.flink.api.common.operators.MailboxExecutor mailboxExecutor)

    • Method Detail

      • setup

        public void setup​(org.apache.flink.streaming.runtime.tasks.StreamTask<?,​?> containingTask,
                  org.apache.flink.streaming.api.graph.StreamConfig config,
                  org.apache.flink.streaming.api.operators.Output<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>> output)
        Specified by:
        setup in interface org.apache.flink.streaming.api.operators.SetupableStreamOperator<CommT>
        Overrides:
        setup in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>

      • open

        public void open()
          throws Exception
        Specified by:
        open in interface org.apache.flink.streaming.api.operators.StreamOperator<CommT>
        Overrides:
        open in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • initializeState

        public void initializeState​(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception
        Specified by:
        initializeState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
        Overrides:
        initializeState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • snapshotState

        public void snapshotState​(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception
        Specified by:
        snapshotState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
        Overrides:
        snapshotState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • processWatermark

        public void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception
        Specified by:
        processWatermark in interface org.apache.flink.streaming.api.operators.Input<CommT>
        Overrides:
        processWatermark in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • processWatermarkStatus

        public void processWatermarkStatus​(org.apache.flink.streaming.runtime.watermarkstatus.WatermarkStatus watermarkStatus)
                            throws Exception
        Specified by:
        processWatermarkStatus in interface org.apache.flink.streaming.api.operators.Input<CommT>
        Overrides:
        processWatermarkStatus in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • processElement

        public void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<Event> element)
                    throws Exception
        Specified by:
        processElement in interface org.apache.flink.streaming.api.operators.Input<CommT>
        Throws:
        Exception

      • prepareSnapshotPreBarrier

        public void prepareSnapshotPreBarrier​(long checkpointId)
                               throws Exception
        Specified by:
        prepareSnapshotPreBarrier in interface org.apache.flink.streaming.api.operators.StreamOperator<CommT>
        Overrides:
        prepareSnapshotPreBarrier in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • close

        public void close()
           throws Exception
        Specified by:
        close in interface org.apache.flink.streaming.api.operators.StreamOperator<CommT>
        Overrides:
        close in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<CommT>>
        Throws:
        Exception

      • endInput

        public void endInput()
              throws Exception
        Specified by:
        endInput in interface org.apache.flink.streaming.api.operators.BoundedOneInput
        Throws:
        Exception