Package org.apache.flink.cdc.connectors.iceberg.sink.v2

Class IcebergSink

java.lang.Object

org.apache.flink.cdc.connectors.iceberg.sink.v2.IcebergSink

All Implemented Interfaces:

Serializable, org.apache.flink.api.connector.sink2.Sink<Event>, org.apache.flink.api.connector.sink2.SupportsCommitter<WriteResultWrapper>, org.apache.flink.api.connector.sink2.SupportsWriterState<Event,IcebergWriterState>, org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink<Event,WriteResultWrapper>, org.apache.flink.streaming.api.connector.sink2.SupportsPostCommitTopology<WriteResultWrapper>, org.apache.flink.streaming.api.connector.sink2.SupportsPreCommitTopology<WriteResultWrapper,WriteResultWrapper>, org.apache.flink.streaming.api.connector.sink2.SupportsPreWriteTopology<Event>, org.apache.flink.streaming.api.connector.sink2.WithPostCommitTopology<Event,WriteResultWrapper>, org.apache.flink.streaming.api.connector.sink2.WithPreCommitTopology<Event,WriteResultWrapper>, org.apache.flink.streaming.api.connector.sink2.WithPreWriteTopology<Event>

public class IcebergSink
extends Object
implements org.apache.flink.api.connector.sink2.Sink<Event>, org.apache.flink.streaming.api.connector.sink2.WithPreWriteTopology<Event>, org.apache.flink.streaming.api.connector.sink2.WithPreCommitTopology<Event,WriteResultWrapper>, org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink<Event,WriteResultWrapper>, org.apache.flink.streaming.api.connector.sink2.WithPostCommitTopology<Event,WriteResultWrapper>, org.apache.flink.api.connector.sink2.SupportsWriterState<Event,IcebergWriterState>

A Sink implementation for Apache Iceberg.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.Sink

org.apache.flink.api.connector.sink2.Sink.InitContext, org.apache.flink.api.connector.sink2.Sink.InitContextWrapper

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.SupportsWriterState

org.apache.flink.api.connector.sink2.SupportsWriterState.WithCompatibleState

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink

org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink.PrecommittingSinkWriter<InputT extends Object,CommT extends Object>

Field Summary

Fields

Modifier and Type	Field	Description
protected Map<String,String>	catalogOptions
protected Map<String,String>	hadoopConfOptions
protected Map<String,String>	tableOptions

Constructor Summary

Constructors

Constructor	Description
IcebergSink(Map<String,String> catalogOptions, Map<String,String> tableOptions, java.time.ZoneId zoneId, CompactionOptions compactionOptions, String jobIdPrefix, Map<String,String> hadoopConfOptions)

Method Summary

Modifier and Type	Method	Description
void	addPostCommitTopology(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<WriteResultWrapper>> committableMessageDataStream)
org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<WriteResultWrapper>>	addPreCommitTopology(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<WriteResultWrapper>> committables)
org.apache.flink.streaming.api.datastream.DataStream<Event>	addPreWriteTopology(org.apache.flink.streaming.api.datastream.DataStream<Event> dataStream)
org.apache.flink.api.connector.sink2.Committer<WriteResultWrapper>	createCommitter()
org.apache.flink.api.connector.sink2.Committer<WriteResultWrapper>	createCommitter(org.apache.flink.api.connector.sink2.CommitterInitContext committerInitContext)
org.apache.flink.api.connector.sink2.SinkWriter<Event>	createWriter(org.apache.flink.api.connector.sink2.Sink.InitContext context)	Deprecated.
org.apache.flink.api.connector.sink2.SinkWriter<Event>	createWriter(org.apache.flink.api.connector.sink2.WriterInitContext context)
org.apache.flink.core.io.SimpleVersionedSerializer<WriteResultWrapper>	getCommittableSerializer()
org.apache.flink.core.io.SimpleVersionedSerializer<WriteResultWrapper>	getWriteResultSerializer()
org.apache.flink.core.io.SimpleVersionedSerializer<IcebergWriterState>	getWriterStateSerializer()
org.apache.flink.api.connector.sink2.StatefulSinkWriter<Event,IcebergWriterState>	restoreWriter(org.apache.flink.api.connector.sink2.WriterInitContext context, Collection<IcebergWriterState> writerStates)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

catalogOptions

protected final Map<String,String> catalogOptions

tableOptions

protected final Map<String,String> tableOptions

hadoopConfOptions

protected final Map<String,String> hadoopConfOptions

Constructor Detail

IcebergSink

public IcebergSink(Map<String,String> catalogOptions,
                   Map<String,String> tableOptions,
                   java.time.ZoneId zoneId,
                   CompactionOptions compactionOptions,
                   String jobIdPrefix,
                   Map<String,String> hadoopConfOptions)

Method Detail

addPreWriteTopology

public org.apache.flink.streaming.api.datastream.DataStream<Event> addPreWriteTopology(org.apache.flink.streaming.api.datastream.DataStream<Event> dataStream)

Specified by:

addPreWriteTopology in interface org.apache.flink.streaming.api.connector.sink2.SupportsPreWriteTopology<Event>

createCommitter

public org.apache.flink.api.connector.sink2.Committer<WriteResultWrapper> createCommitter()

Specified by:

createCommitter in interface org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink<Event,WriteResultWrapper>

createCommitter

public org.apache.flink.api.connector.sink2.Committer<WriteResultWrapper> createCommitter(org.apache.flink.api.connector.sink2.CommitterInitContext committerInitContext)

Specified by:

createCommitter in interface org.apache.flink.api.connector.sink2.SupportsCommitter<WriteResultWrapper>

Specified by:

createCommitter in interface org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink<Event,WriteResultWrapper>

getCommittableSerializer

public org.apache.flink.core.io.SimpleVersionedSerializer<WriteResultWrapper> getCommittableSerializer()

Specified by:

getCommittableSerializer in interface org.apache.flink.api.connector.sink2.SupportsCommitter<WriteResultWrapper>

createWriter

@Deprecated
public org.apache.flink.api.connector.sink2.SinkWriter<Event> createWriter(org.apache.flink.api.connector.sink2.Sink.InitContext context)

Deprecated.

Specified by:

createWriter in interface org.apache.flink.api.connector.sink2.Sink<Event>

createWriter

public org.apache.flink.api.connector.sink2.SinkWriter<Event> createWriter(org.apache.flink.api.connector.sink2.WriterInitContext context)

Specified by:

createWriter in interface org.apache.flink.api.connector.sink2.Sink<Event>

restoreWriter

public org.apache.flink.api.connector.sink2.StatefulSinkWriter<Event,IcebergWriterState> restoreWriter(org.apache.flink.api.connector.sink2.WriterInitContext context,
                                                                                                             Collection<IcebergWriterState> writerStates)

Specified by:

restoreWriter in interface org.apache.flink.api.connector.sink2.SupportsWriterState<Event,IcebergWriterState>

getWriterStateSerializer

public org.apache.flink.core.io.SimpleVersionedSerializer<IcebergWriterState> getWriterStateSerializer()

Specified by:

getWriterStateSerializer in interface org.apache.flink.api.connector.sink2.SupportsWriterState<Event,IcebergWriterState>

addPreCommitTopology

public org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<WriteResultWrapper>> addPreCommitTopology(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<WriteResultWrapper>> committables)

Specified by:

addPreCommitTopology in interface org.apache.flink.streaming.api.connector.sink2.SupportsPreCommitTopology<WriteResultWrapper,WriteResultWrapper>

getWriteResultSerializer

public org.apache.flink.core.io.SimpleVersionedSerializer<WriteResultWrapper> getWriteResultSerializer()

Specified by:

getWriteResultSerializer in interface org.apache.flink.streaming.api.connector.sink2.SupportsPreCommitTopology<WriteResultWrapper,WriteResultWrapper>

Specified by:

getWriteResultSerializer in interface org.apache.flink.streaming.api.connector.sink2.WithPreCommitTopology<Event,WriteResultWrapper>

addPostCommitTopology

public void addPostCommitTopology(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.streaming.api.connector.sink2.CommittableMessage<WriteResultWrapper>> committableMessageDataStream)

Specified by:

addPostCommitTopology in interface org.apache.flink.streaming.api.connector.sink2.SupportsPostCommitTopology<WriteResultWrapper>