Interface SplitAssigner
-
- All Superinterfaces:
AutoCloseable,Closeable
- All Known Implementing Classes:
HybridSplitAssigner,SnapshotSplitAssigner,StreamSplitAssigner
@Experimental public interface SplitAssigner extends Closeable
TheSplitAssigneris responsible for deciding what split should be processed. It determines split processing order.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description voidaddSplits(Collection<SourceSplitBase> splits)Adds a set of splits to this assigner.voidclose()Called to close the assigner, in case it holds on to any resources, like threads or network connections.AssignerStatusgetAssignerStatus()Gets the split assigner status, seeAssignerStatus.List<FinishedSnapshotSplitInfo>getFinishedSplitInfos()Gets the finished splits' information.Optional<SourceSplitBase>getNext()Gets the next split.default booleanisSnapshotMetaReleased()Returns whetherreleaseSnapshotMetadata()has dropped the snapshot metadata.booleannoMoreSplits()Indicates there is no more splits available in this assigner.voidnotifyCheckpointComplete(long checkpointId)Notifies the listener that the checkpoint with the givencheckpointIdcompleted and was committed.voidonFinishedSplits(Map<String,Offset> splitFinishedOffsets)Callback to handle the finished splits with finished change log offset.voidonStreamSplitUpdated()Callback to handle the stream split has been updated in the newly added tables process.voidopen()Called to open the assigner to acquire any resources, like threads or network connections.default voidreleaseSnapshotMetadata()Releases the bulk snapshot-split metadata once the reader has assembled it and a checkpoint has covered that assignment.PendingSplitsStatesnapshotState(long checkpointId)Creates a snapshot of the state of this split assigner, to be stored in a checkpoint.voidstartAssignNewlyAddedTables()Starts assign newly added tables.booleanwaitingForFinishedSplits()Whether the split assigner is still waiting for callback of finished splits, i.e.
-
-
-
Method Detail
-
open
void open()
Called to open the assigner to acquire any resources, like threads or network connections.
-
getNext
Optional<SourceSplitBase> getNext()
Gets the next split.When this method returns an empty
Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.
-
waitingForFinishedSplits
boolean waitingForFinishedSplits()
Whether the split assigner is still waiting for callback of finished splits, i.e.onFinishedSplits(Map).
-
noMoreSplits
boolean noMoreSplits()
Indicates there is no more splits available in this assigner.
-
getFinishedSplitInfos
List<FinishedSnapshotSplitInfo> getFinishedSplitInfos()
Gets the finished splits' information. This is useful metadata to generate a stream split that considering finished snapshot splits.
-
onFinishedSplits
void onFinishedSplits(Map<String,Offset> splitFinishedOffsets)
Callback to handle the finished splits with finished change log offset. This is useful for determine when to generate stream split and what stream split to generate.
-
addSplits
void addSplits(Collection<SourceSplitBase> splits)
Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added.
-
snapshotState
PendingSplitsState snapshotState(long checkpointId)
Creates a snapshot of the state of this split assigner, to be stored in a checkpoint.The snapshot should contain the latest state of the assigner: It should assume that all operations that happened before the snapshot have successfully completed. For example all splits assigned to readers via
getNext()don't need to be included in the snapshot anymore.This method takes the ID of the checkpoint for which the state is snapshotted. Most implementations should be able to ignore this parameter, because for the contents of the snapshot, it doesn't matter for which checkpoint it gets created. This parameter can be interesting for source connectors with external systems where those systems are themselves aware of checkpoints; for example in cases where the enumerator notifies that system about a specific checkpoint being triggered.
- Parameters:
checkpointId- The ID of the checkpoint for which the snapshot is created.- Returns:
- an object containing the state of the split enumerator.
-
notifyCheckpointComplete
void notifyCheckpointComplete(long checkpointId)
Notifies the listener that the checkpoint with the givencheckpointIdcompleted and was committed.- See Also:
CheckpointListener.notifyCheckpointComplete(long)
-
getAssignerStatus
AssignerStatus getAssignerStatus()
Gets the split assigner status, seeAssignerStatus.
-
startAssignNewlyAddedTables
void startAssignNewlyAddedTables()
Starts assign newly added tables.
-
onStreamSplitUpdated
void onStreamSplitUpdated()
Callback to handle the stream split has been updated in the newly added tables process. This is useful to check the newly added tables has been finished or not.
-
releaseSnapshotMetadata
default void releaseSnapshotMetadata()
Releases the bulk snapshot-split metadata once the reader has assembled it and a checkpoint has covered that assignment. Implementations holding no snapshot metadata keep the default no-op. Generalizes the MySQL coordinator-memory release (FLINK-39775).
-
isSnapshotMetaReleased
default boolean isSnapshotMetaReleased()
Returns whetherreleaseSnapshotMetadata()has dropped the snapshot metadata.
-
close
void close() throws IOExceptionCalled to close the assigner, in case it holds on to any resources, like threads or network connections.- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Throws:
IOException
-
-