Package io.debezium.connector.mysql
Class MySqlValueConverters
- java.lang.Object
-
- io.debezium.jdbc.JdbcValueConverters
-
- io.debezium.connector.mysql.MySqlValueConverters
-
- All Implemented Interfaces:
io.debezium.relational.ValueConverterProvider
@Immutable public class MySqlValueConverters extends io.debezium.jdbc.JdbcValueConvertersCopied from Debezium project(1.9.8.Final) to fix FLOAT converted to FLOAT64 type issue. See DBZ-3865, DBZ-5843. Remove it when debezium version is upgraded above 2.0.0.Final.Line 240 & 246: add FLOAT type adjustment.
Line 485 & 499: add method to convert FLOAT value.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interfaceMySqlValueConverters.ParsingErrorHandler
-
Constructor Summary
Constructors Constructor Description MySqlValueConverters(io.debezium.jdbc.JdbcValueConverters.DecimalMode decimalMode, io.debezium.jdbc.TemporalPrecisionMode temporalPrecisionMode, io.debezium.jdbc.JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode, io.debezium.config.CommonConnectorConfig.BinaryHandlingMode binaryMode)Create a new instance that always uses UTC for the default time zone when_needed converting values without timezone information to values that require timezones.MySqlValueConverters(io.debezium.jdbc.JdbcValueConverters.DecimalMode decimalMode, io.debezium.jdbc.TemporalPrecisionMode temporalPrecisionMode, io.debezium.jdbc.JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode, io.debezium.config.CommonConnectorConfig.BinaryHandlingMode binaryMode, java.time.temporal.TemporalAdjuster adjuster, MySqlValueConverters.ParsingErrorHandler parsingErrorHandler)Create a new instance that always uses UTC for the default time zone when converting values without timezone information to values that require timezones.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static java.time.temporal.TemporaladjustTemporal(java.time.temporal.Temporal temporal)A utility method that adjusts ambiguous 2-digit year values of DATETIME, DATE, and TIMESTAMP types using these MySQL-specific rules: Year values in the range 00-69 are converted to 2000-2069.protected ByteOrderbyteOrderOfBitType()protected CharsetcharsetFor(io.debezium.relational.Column column)Return theCharsetinstance with the MySQL-specific character set name used by the given column.static booleancontainsZeroValuesInDatePart(String timestampString, io.debezium.relational.Column column, io.debezium.relational.Table table)protected ObjectconvertBigInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)protected ObjectconvertDurationToMicroseconds(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Converts a value object for an expected type ofDurationtoLongvalues that represents the time in microseconds.protected ObjectconvertEnumToString(List<String> options, io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Converts a value object for a MySQLENUM, which is represented in the binlog events as an integer value containing the index of the enum option.io.debezium.relational.ValueConverterconverter(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn)protected ObjectconvertFloat(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)MySql reports FLOAT(p) values as FLOAT and DOUBLE.protected ObjectconvertGeometry(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a GEOMETRYbyte[]value to a Geometry value used in aSourceRecord.protected ObjectconvertInteger(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)protected ObjectconvertJson(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)protected ObjectconvertPoint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a POINTbyte[]value to a Point value used in aSourceRecord.protected ObjectconvertSetToString(List<String> options, io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Converts a value object for a MySQLSET, which is represented in the binlog events contain a long number in which every bit corresponds to a different option.protected StringconvertSetValue(io.debezium.relational.Column column, long indexes, List<String> options)protected ObjectconvertSmallInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)protected ObjectconvertString(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Charset columnCharset, Object data)protected ObjectconvertTimestampToLocalDateTime(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)protected ObjectconvertUnsignedBigint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a Unsigned BIGINT value to the correct Unsigned INT representation.protected ObjectconvertUnsignedInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a Unsigned INT value to the correct Unsigned INT representation.protected ObjectconvertUnsignedMediumint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a Unsigned MEDIUMINT value to the correct Unsigned SMALLINT representation.protected ObjectconvertUnsignedSmallint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a Unsigned SMALLINT value to the correct Unsigned SMALLINT representation.protected ObjectconvertUnsignedTinyint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Convert the a value representing a Unsigned TINYINT value to the correct Unsigned TINYINT representation.protected ObjectconvertYearToInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)Converts a value object for a MySQLYEAR, which appear in the binlog as an integer though returns from the MySQL JDBC driver as either a short or aDate.static voiddefaultParsingErrorHandler(String message, Exception exception)protected List<String>extractEnumAndSetOptions(io.debezium.relational.Column column)protected StringextractEnumAndSetOptionsAsString(io.debezium.relational.Column column)protected booleanisGeometryCollection(String upperCaseTypeName)Determine if the uppercase form of a column's type is geometry collection independent of JDBC driver or server version.protected booleanmatches(String upperCaseTypeName, String upperCaseMatch)Determine if the uppercase form of a column's type exactly matches or begins with the specified prefix.protected byte[]normalizeBinaryData(io.debezium.relational.Column column, byte[] data)org.apache.kafka.connect.data.SchemaBuilderschemaBuilder(io.debezium.relational.Column column)static java.time.DurationstringToDuration(String timeString)static java.time.LocalDatestringToLocalDate(String dateString, io.debezium.relational.Column column, io.debezium.relational.Table table)-
Methods inherited from class io.debezium.jdbc.JdbcValueConverters
convertBinary, convertBinaryToBase64, convertBinaryToBytes, convertBinaryToHex, convertBit, convertBits, convertBits, convertBoolean, convertDateToEpochDays, convertDateToEpochDaysAsDate, convertDecimal, convertDouble, convertNumeric, convertReal, convertRowId, convertString, convertTime, convertTimestampToEpochMicros, convertTimestampToEpochMillis, convertTimestampToEpochMillisAsDate, convertTimestampToEpochNanos, convertTimestampWithZone, convertTimeToMicrosPastMidnight, convertTimeToMillisPastMidnight, convertTimeToMillisPastMidnightAsDate, convertTimeToNanosPastMidnight, convertTimeWithZone, convertTinyInt, convertValue, getTimePrecision, handleUnknownData, padLittleEndian, toBigDecimal, toByteBuffer, unexpectedBinary, withScaleAdjustedIfNeeded
-
-
-
-
Constructor Detail
-
MySqlValueConverters
public MySqlValueConverters(io.debezium.jdbc.JdbcValueConverters.DecimalMode decimalMode, io.debezium.jdbc.TemporalPrecisionMode temporalPrecisionMode, io.debezium.jdbc.JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode, io.debezium.config.CommonConnectorConfig.BinaryHandlingMode binaryMode)Create a new instance that always uses UTC for the default time zone when_needed converting values without timezone information to values that require timezones.- Parameters:
decimalMode- howDECIMALandNUMERICvalues should be treated; may be null ifJdbcValueConverters.DecimalMode.PRECISEis to be usedtemporalPrecisionMode- temporal precision mode based onTemporalPrecisionModebigIntUnsignedMode- howBIGINT UNSIGNEDvalues should be treated; may be null ifJdbcValueConverters.BigIntUnsignedMode.PRECISEis to be usedbinaryMode- how binary columns should be represented
-
MySqlValueConverters
public MySqlValueConverters(io.debezium.jdbc.JdbcValueConverters.DecimalMode decimalMode, io.debezium.jdbc.TemporalPrecisionMode temporalPrecisionMode, io.debezium.jdbc.JdbcValueConverters.BigIntUnsignedMode bigIntUnsignedMode, io.debezium.config.CommonConnectorConfig.BinaryHandlingMode binaryMode, java.time.temporal.TemporalAdjuster adjuster, MySqlValueConverters.ParsingErrorHandler parsingErrorHandler)Create a new instance that always uses UTC for the default time zone when converting values without timezone information to values that require timezones.- Parameters:
decimalMode- howDECIMALandNUMERICvalues should be treated; may be null ifJdbcValueConverters.DecimalMode.PRECISEis to be usedtemporalPrecisionMode- temporal precision mode based onTemporalPrecisionModebigIntUnsignedMode- howBIGINT UNSIGNEDvalues should be treated; may be null ifJdbcValueConverters.BigIntUnsignedMode.PRECISEis to be usedbinaryMode- how binary columns should be representedadjuster- a temporal adjuster to make a database specific time modification before conversionhandler- for errors during postponed binlog parsing
-
-
Method Detail
-
adjustTemporal
public static java.time.temporal.Temporal adjustTemporal(java.time.temporal.Temporal temporal)
A utility method that adjusts ambiguous 2-digit year values of DATETIME, DATE, and TIMESTAMP types using these MySQL-specific rules:- Year values in the range 00-69 are converted to 2000-2069.
- Year values in the range 70-99 are converted to 1970-1999.
- Parameters:
temporal- the temporal instance to adjust; may not be null- Returns:
- the possibly adjusted temporal instance; never null
-
byteOrderOfBitType
protected ByteOrder byteOrderOfBitType()
- Overrides:
byteOrderOfBitTypein classio.debezium.jdbc.JdbcValueConverters
-
schemaBuilder
public org.apache.kafka.connect.data.SchemaBuilder schemaBuilder(io.debezium.relational.Column column)
- Specified by:
schemaBuilderin interfaceio.debezium.relational.ValueConverterProvider- Overrides:
schemaBuilderin classio.debezium.jdbc.JdbcValueConverters
-
converter
public io.debezium.relational.ValueConverter converter(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn)- Specified by:
converterin interfaceio.debezium.relational.ValueConverterProvider- Overrides:
converterin classio.debezium.jdbc.JdbcValueConverters
-
charsetFor
protected Charset charsetFor(io.debezium.relational.Column column)
Return theCharsetinstance with the MySQL-specific character set name used by the given column.- Parameters:
column- the column in which the character set is used; never null- Returns:
- the Java
Charset, or null if there is no mapping
-
convertJson
protected Object convertJson(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertSmallInt
protected Object convertSmallInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
- Overrides:
convertSmallIntin classio.debezium.jdbc.JdbcValueConverters
-
convertInteger
protected Object convertInteger(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
- Overrides:
convertIntegerin classio.debezium.jdbc.JdbcValueConverters
-
convertBigInt
protected Object convertBigInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
- Overrides:
convertBigIntin classio.debezium.jdbc.JdbcValueConverters
-
convertFloat
protected Object convertFloat(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
MySql reports FLOAT(p) values as FLOAT and DOUBLE. A precision from 0 to 23 results in a 4-byte single-precision FLOAT column. A precision from 24 to 53 results in an 8-byte double-precision DOUBLE column. As of MySQL 8.0.17, the nonstandard FLOAT(M,D) and DOUBLE(M,D) syntax is deprecated, and should expect support for it be removed in a future version of MySQL. Based on this future, we didn't handle the case.- Overrides:
convertFloatin classio.debezium.jdbc.JdbcValueConverters
-
convertString
protected Object convertString(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Charset columnCharset, Object data)
- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nullcolumnCharset- the Java character set in which column byte[] values are encoded; may not be nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertYearToInt
protected Object convertYearToInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Converts a value object for a MySQLYEAR, which appear in the binlog as an integer though returns from the MySQL JDBC driver as either a short or aDate.- Parameters:
column- the column definition describing thedatavalue; never nullfieldDefn- the field definition; never nulldata- the data object to be converted into a year literal integer value; never null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertEnumToString
protected Object convertEnumToString(List<String> options, io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Converts a value object for a MySQLENUM, which is represented in the binlog events as an integer value containing the index of the enum option. The MySQL JDBC driver returns a string containing the option, so this method calculates the same.- Parameters:
options- the characters that appear in the same order as defined in the column; may not be nullcolumn- the column definition describing thedatavalue; never nullfieldDefn- the field definition; never nulldata- the data object to be converted into anENUMliteral String value- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertSetToString
protected Object convertSetToString(List<String> options, io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Converts a value object for a MySQLSET, which is represented in the binlog events contain a long number in which every bit corresponds to a different option. The MySQL JDBC driver returns a string containing the comma-separated options, so this method calculates the same.- Parameters:
options- the characters that appear in the same order as defined in the column; may not be nullcolumn- the column definition describing thedatavalue; never nullfieldDefn- the field definition; never nulldata- the data object to be converted into anSETliteral String value; never null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
matches
protected boolean matches(String upperCaseTypeName, String upperCaseMatch)
Determine if the uppercase form of a column's type exactly matches or begins with the specified prefix. Note that this logic works when the column'stypecontains the type name followed by parentheses.- Parameters:
upperCaseTypeName- the upper case form of the column'stype nameupperCaseMatch- the upper case form of the expected type or prefix of the type; may not be null- Returns:
trueif the type matches the specified type, orfalseotherwise
-
isGeometryCollection
protected boolean isGeometryCollection(String upperCaseTypeName)
Determine if the uppercase form of a column's type is geometry collection independent of JDBC driver or server version.- Parameters:
upperCaseTypeName- the upper case form of the column'stype name- Returns:
trueif the type is geometry collection
-
extractEnumAndSetOptions
protected List<String> extractEnumAndSetOptions(io.debezium.relational.Column column)
-
extractEnumAndSetOptionsAsString
protected String extractEnumAndSetOptionsAsString(io.debezium.relational.Column column)
-
convertSetValue
protected String convertSetValue(io.debezium.relational.Column column, long indexes, List<String> options)
-
convertPoint
protected Object convertPoint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a POINTbyte[]value to a Point value used in aSourceRecord.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertGeometry
protected Object convertGeometry(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a GEOMETRYbyte[]value to a Geometry value used in aSourceRecord.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
normalizeBinaryData
protected byte[] normalizeBinaryData(io.debezium.relational.Column column, byte[] data)- Overrides:
normalizeBinaryDatain classio.debezium.jdbc.JdbcValueConverters
-
convertUnsignedTinyint
protected Object convertUnsignedTinyint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a Unsigned TINYINT value to the correct Unsigned TINYINT representation.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertUnsignedSmallint
protected Object convertUnsignedSmallint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a Unsigned SMALLINT value to the correct Unsigned SMALLINT representation.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertUnsignedMediumint
protected Object convertUnsignedMediumint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a Unsigned MEDIUMINT value to the correct Unsigned SMALLINT representation.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertUnsignedInt
protected Object convertUnsignedInt(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a Unsigned INT value to the correct Unsigned INT representation.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertUnsignedBigint
protected Object convertUnsignedBigint(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Convert the a value representing a Unsigned BIGINT value to the correct Unsigned INT representation.- Parameters:
column- the column in which the value appearsfieldDefn- the field definition for theSourceRecord'sSchema; never nulldata- the data; may be null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertDurationToMicroseconds
protected Object convertDurationToMicroseconds(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
Converts a value object for an expected type ofDurationtoLongvalues that represents the time in microseconds.Per the JDBC specification, databases should return
Timeinstances, but that's not working because it can only handle Daytime 00:00:00-23:59:59. We useDurationinstead that can handle the range of -838:59:59.000000 to 838:59:59.000000 of a MySQL TIME type and transfer data as signed INT64 which reflects the DB value converted to microseconds.- Parameters:
column- the column definition describing thedatavalue; never nullfieldDefn- the field definition; never nulldata- the data object to be converted into aDurationtype; never null- Returns:
- the converted value, or null if the conversion could not be made and the column allows nulls
- Throws:
IllegalArgumentException- if the value could not be converted but the column does not allow nulls
-
convertTimestampToLocalDateTime
protected Object convertTimestampToLocalDateTime(io.debezium.relational.Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data)
-
stringToDuration
public static java.time.Duration stringToDuration(String timeString)
-
stringToLocalDate
public static java.time.LocalDate stringToLocalDate(String dateString, io.debezium.relational.Column column, io.debezium.relational.Table table)
-
containsZeroValuesInDatePart
public static boolean containsZeroValuesInDatePart(String timestampString, io.debezium.relational.Column column, io.debezium.relational.Table table)
-
-