Kafka JDBC 连接：根据消息字段中的多个值插入键

Question

我有以下 json 关于 JDBC 连接器发布到

的主题

{"APP_SETTING_ID":9,"USER_ID":10,"APP_SETTING_NAME":"my_name","SETTING_KEY":"my_setting_key"}

这是我的连接器文件

name=data.app_setting
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
poll.interval.ms=500
tasks.max=4
mode=timestamp
query=SELECT APP_SETTING_ID, APP_SETTING_NAME, SETTING_KEY,FROM MY_TABLE with (nolock)
timestamp.column.name=LAST_MOD_DATE
topic.prefix=data.app_setting

key.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter=org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable=false

我现在想通过乘以两个整数字段 - APP_SETTING_ID 和 USER_ID 来为此消息插入一个密钥。所以此消息的密钥变为 9*10 = 90

是否可以通过 Connect 进行这种转换，如果可以，请有人阐明一下

Answer 1

我想看看你能用

走多远

query=SELECT APP_SETTING_ID, APP_SETTING_NAME, SETTING_KEY, (APP_SETTING_ID*USER_ID) as _key FROM MY_TABLE with (nolock)

然后添加一个 ExtractKey 转换

transforms=AddKeys,ExtractKey
# this make a map
transforms.AddKeys.type=org.apache.kafka.connect.transforms.ValueToKey
transforms.AddKeys.fields=_key  
# this gets one field from the map 
transforms.ExtractKey.type=org.apache.kafka.connect.transforms.ExtractField$Key
transforms.ExtractKey.field=_key

Kafka JDBC 连接：根据消息字段中的多个值插入键

Kafka JDBC Connect: Insert key based on multiple values in message fields

jdbc

apache-kafka

apache-kafka-connect

confluent-platform