Connecting to an Apache Kafka Cluster

Now that you’ve created a cluster, it’s time to connect your application to your new Kafka cluster. Aside from setting your firewall rules, Instaclustr provides a few examples (customised to your cluster) for the wide selection of client libraries Kafka supports to help you through this process.

1. After your Cluster has finished provisioning, navigate to your Cluster Details page. All your nodes should be in a Running state with no errors listed. Some infrastructure providers allocate both public and private (data centre local) IP addresses to cluster nodes. If both are available, they will be listed with their respective nodes on this page.
   
   We suggest that if your application is running within the same data centre as your cluster, you investigate configuring your client to connect to the private addresses. Please refer to your provider’s pricing documentation.
   
   
2. Instaclustr manages the firewall permissions for the nodes in your cluster. Each cluster node only allows connections from one or more trusted IP addresses. You can add one or more trusted IP addresses to the cluster firewall by clicking Firewall Rules from the side bar.
   
   
   
   
3. Under the Firewall Rules section, enter any additional IP addresses you wish to trust to the Kafka Allowed Addresses list. If your cluster is running on AWS and clients are connecting using VPC peering, then you may wish to allow an AWS security group to connect rather than adding individual addresses. If this is the case, please raise a support request and a member of our support team will be in contact. Click the Save button when you are finished.
   
   
   
   
4. Instaclustr provides connection information and examples on the Connection Info page which can be accessed by clicking Connection Info from the side bar.
   
   
   
5. The Connection Info page contains a list of your node addresses, authentication credentials to connect to your cluster and a few connection examples for popular clients Kafka supports.
   
   
   
6. There are a large number of Kafka client libraries that cover a wide range of languages. Instaclustr provides step-by-step examples on producing and consuming messages with Kafka for:
   • Java
   • C#
   • Python
   • Ruby
   • Command Line
7. The Apache Kafka project also maintains a list of clients across a wider set of languages.