Package org.apache.flink.cdc.common.utils

Class SchemaMergingUtils

java.lang.Object

org.apache.flink.cdc.common.utils.SchemaMergingUtils

@PublicEvolving
public class SchemaMergingUtils
extends Object

Utils for merging Schemas and DataTypes. Prefer using this over SchemaUtils to get consistent schema merging behaviors.

Constructor Summary

Constructors

Constructor	Description
SchemaMergingUtils()

Method Summary

Modifier and Type	Method	Description
static Object	coerceObject(String timezone, Object originalField, DataType originalType, DataType destinationType)
static Object[]	coerceRow(String timezone, Schema currentSchema, Schema upcomingSchema, List<Object> upcomingRow)	Coercing upcomingRow with upcomingTypes schema into currentTypes schema.
static Object[]	coerceRow(String timezone, Schema currentSchema, Schema upcomingSchema, List<Object> upcomingRow, boolean toleranceMode)	Coercing upcomingRow with upcomingTypes schema into currentTypes schema.
static Schema	getCommonSchema(List<Schema> schemas)	Merge compatible schemas.
static Schema	getLeastCommonSchema(Schema currentSchema, Schema upcomingSchema)	Try to merge upcomingSchema into currentSchema by performing lenient schema changes.
static int	getNumericPrecision(DataType dataType)
static List<SchemaChangeEvent>	getSchemaDifference(TableId tableId, Schema beforeSchema, Schema afterSchema)	Generating what schema change events we need to do by converting compatible beforeSchema to afterSchema.
static boolean	isDataTypeCompatible(DataType currentType, DataType upcomingType)	Checks whether the upcoming data type can fit into the current data type.
static boolean	isSchemaCompatible(Schema currentSchema, Schema upcomingSchema)	Checking if given upcomingSchema could be fit into currently known currentSchema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SchemaMergingUtils

public SchemaMergingUtils()

Method Detail

isSchemaCompatible

public static boolean isSchemaCompatible(@Nullable
                                         Schema currentSchema,
                                         Schema upcomingSchema)

Checking if given upcomingSchema could be fit into currently known currentSchema. Current schema could be null (as the cold opening state, and in this case it always returns false) but the upcoming schema should never be null.
This method only checks columns' type compatibility, but ignores metadata fields like primaryKeys, partitionKeys, options.

getLeastCommonSchema

public static Schema getLeastCommonSchema(@Nullable
                                          Schema currentSchema,
                                          Schema upcomingSchema)

Try to merge upcomingSchema into currentSchema by performing lenient schema changes. Returns a wider schema that could both of them.

getCommonSchema

public static Schema getCommonSchema(List<Schema> schemas)

Merge compatible schemas.

getSchemaDifference

public static List<SchemaChangeEvent> getSchemaDifference(TableId tableId,
                                                          @Nullable
                                                          Schema beforeSchema,
                                                          Schema afterSchema)

Generating what schema change events we need to do by converting compatible beforeSchema to afterSchema.

coerceRow

public static Object[] coerceRow(String timezone,
                                 Schema currentSchema,
                                 Schema upcomingSchema,
                                 List<Object> upcomingRow)

Coercing upcomingRow with upcomingTypes schema into currentTypes schema. Invoking this method implicitly assumes that isSchemaCompatible(currentSchema, upcomingSchema) returns true. Otherwise, some upstream records might be lost.

coerceRow

public static Object[] coerceRow(String timezone,
                                 Schema currentSchema,
                                 Schema upcomingSchema,
                                 List<Object> upcomingRow,
                                 boolean toleranceMode)

Coercing upcomingRow with upcomingTypes schema into currentTypes schema. Invoking this method implicitly assumes that isSchemaCompatible(currentSchema, upcomingSchema) returns true. Otherwise, some upstream records might be lost.

isDataTypeCompatible

public static boolean isDataTypeCompatible(@Nullable
                                           DataType currentType,
                                           DataType upcomingType)

Checks whether the upcoming data type can fit into the current data type. A missing current type is treated as incompatible.

getNumericPrecision

@VisibleForTesting
public static int getNumericPrecision(DataType dataType)

coerceObject

@VisibleForTesting
public static Object coerceObject(String timezone,
                                  Object originalField,
                                  DataType originalType,
                                  DataType destinationType)