Using non-Instaclustr Kafka Clusters

In this article we’ll show how to set up an Instaclustr Kafka Connect cluster that targets a non-Instaclustr Kafka cluster via the Instaclustr console. For information on how to provision an Apache Kafka Connect cluster via the Instaclustr Cluster Management API, please refer to our API documentation.

1. Follow the steps in the creation guide for Kafka Connect clusters, except for the following differences when filling out the Kafka Connect Options section.
2. Select the ‘Other Kafka Cluster’ option in the Target Kafka Cluster field, and put in the connection related properties that you will need to insert into the Kafka Connect worker.properties file. Note that in this interface only bootstrap.servers and connection related security options are allowed. Not every valid entry in worker.properties will be allowed here. 
   
   bootstrap.servers=x.x.x.x:9092 security.protocol=SASL_SSL sasl.mechanism=SCRAM-SHA-256 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username = "xxx" password = "xxxxxx"

   1 2 3 4

   bootstrap.servers=x.x.x.x:9092 security.protocol=SASL_SSL sasl.mechanism=SCRAM-SHA-256 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username = "xxx" password = "xxxxxx"

3. If security protocol is implemented on your Kafka cluster, you would need to create a Kafka user, information on Kafka ACL permission can be found here. Here is an example of ACL permissions for the Kafka user:
   

   Domain	Action
   Topic	ALL
   Group	ALL
   Transactional ID	ALL
   Cluster	CREATE
   Cluster	DESCRIBE_CONFIGS
   Cluster	DESCRIBE
   Cluster	IDEMPOTENT_WRITE

4. Additionally, if you are using an SSL/TLS connection to your Kafka cluster, you will need to supply a base64 encoded version of a JKS trust store containing your Kafka server certificate. Also note that we do not support setting up client authentication via SSL/TLS.