I will be using one of my previous blogs as a background for this one. Although the concept remains the same, the execution is different. We will leverage Oracle Event Hub cloud service as the managed Kafka broker. Our producer & consumer microservices run will run on Oracle Application Container Cloud and leverage its Service Binding feature for Oracle Event Hub cloud. CI/CD for both these applications will be handled by Oracle Developer Cloud service. Specific focus areas include


  • Overview of how to get started with Oracle Event Hub cloud including bootstrapping a cluster and topic
  • How to use the Oracle Event Hub Cloud service binding available in Application Container Cloud
  • Configuring Oracle Developer Cloud Service to achieve CI/CD to Application Container Cloud




Let's briefly look at what role do the individual cloud services play


Oracle Event Hub cloud

This is a fully managed Platform-as-a-Service which makes it dead simple to setup and work with an Apache Kafka cluster

  • You can easily setup clusters and scale them elastically
  • Quickly create topics and add/remove partitions
  • It also provides REST interface as well command line clients to work with your Kafka cluster & topics


Oracle Application Container cloud

We continue to use it as the platform to run our producer and consumer microservices. The good thing is that services running on Application Container cloud can easily connect with Oracle Event Hub cloud service using the Service Binding feature. We will this in action


Oracle Developer Cloud

This serves as a central hub for source code repository and DevOps pipeline. All we need to do is configure the build and deployment once and it will take care of seamless CI/CD to Application Container Cloud. Although Developer Cloud is capable of a lot more, this blog will focus on these features. Please refer to the documentation for more details



The logic for our consumer and producer services is very much the same and the details available here. Let's focus on how to use the Oracle Event Hub service binding


Leveraging the Event Hub Service Binding in Application Container Cloud

The service bindings are utilized in the same way in both our services i.e. consumer and producer. The logic uses the Application Container Cloud environment variables (created as a result of the Service Binding) to fetch the location of our Event Hub Kafka cluster as well the topic we want to work with (in this case it’s just a single topic). You do not need to expose ports on the Kafka node(s) for the services on Application Container Cloud to access them. It's all taken care of by the Service Binding internally !


Here is a preview



public class Consumer implements Runnable {

    private static final Logger LOGGER = Logger.getLogger(Consumer.class.getName());
    private static final String CONSUMER_GROUP = "cpu-metrics-group";
    private final AtomicBoolean CONSUMER_STOPPED = new AtomicBoolean(false);
    private KafkaConsumer<String, String> consumer = null;
    private final String topicName;
    public Consumer() {
        Properties kafkaProps = new Properties();
        LOGGER.log(Level.INFO, "Kafka Consumer running in thread {0}", Thread.currentThread().getName());

        this.topicName = System.getenv().get("OEHCS_TOPIC");
        LOGGER.log(Level.INFO, "Kafka topic {0}", topicName);

        String kafkaCluster = System.getenv().get("OEHCS_EXTERNAL_CONNECT_STRING");
        LOGGER.log(Level.INFO, "Kafka cluster {0}", kafkaCluster);

        kafkaProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaCluster);
        kafkaProps.put(ConsumerConfig.GROUP_ID_CONFIG, CONSUMER_GROUP);
        kafkaProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        kafkaProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");

        this.consumer = new KafkaConsumer<>(kafkaProps);



  • The variable OEHCS_EXTERNAL_CONNECT_STRING allows us get the co-ordinates for the Kafka cluster. This is used in the Kafka configuration represented by a java.util.Properties object
  • OEHCS_TOPIC gives us the name of the topic which is then passed on to the subscribe method of the KafkaConsumer




Setting up Oracle Event Hub cloud service

Let’s see how to setup a managed Kafka cluster in Oracle Cloud


Bootstrap the cluster

We first create the Kakfa cluster itself. Use the wizard in the Oracle Event Hub Cloud Service – Platform section to get started



Enter the required details



In this case, we are choosing the following configuration

  • Basic deployment where the Kafka cluster and Zookeeper are co-located on the same node
  • A single Kafka node




You can choose from different options such as

  • changing the deployment type to Recommended
  • opting for 3 nodes with the Basic mode
  • deploying a REST proxy alongside your cluster etc.


Refer to the official product documentation for more details


Click Create to start the provisioning process for your Kafka cluster



Wait for the process to complete



Setup the Kafka topic

Once the Kafka cluster is created, you can now create individual topics. To do so, choose Oracle Event Hub Cloud Service from the Platform Services menu



Click on Create Service and fill in the required details in the subsequent page



Here we create a topic name cpu-metrics in the cluster kafka-cluster (we just created). The number of partitions is 10 and the retention period is one week (168 hours)




Click Create to conclude the process




Within a few seconds, you should see your newly created topic




Configuring Developer Cloud Service


Project & code repository creation

Please refer to the Project & code repository creation section in the Tracking JUnit test results in Developer Cloud service blog or check the product documentation for more details


Configure source code in Git repository

Push the project from your local system to your Developer Cloud Git repo you just created. We will do this via command line and all you need is Git client installed on your local machine. You can use Git or any other tool of your choice


Repeat this process for both your application (producer and consumer)


cd <project_folder> //where you unzipped the source code  
git init  
git remote add origin <developer_cloud_git_repo>  
//e.g. https://john.doe@developer.us.oraclecloud.com/developer007-foodomain/s/developer007-foodomain-project_2009/scm/sample.git//john.doe@developer.us.oraclecloud.com/developer007-foodomain/s/developer007-foodomain-project_2009/scm/sample.git   
git add .  
git commit -m "first commit"  
git push -u origin master  //Please enter the password for your Oracle Developer Cloud account when prompted



Configure build job

Repeat this process for both your application (producer and consumer)


Create a New Job



Basic Configuration


Select JDK



Source Control


Choose Git repository



Build Trigger (Continuous Integration)


Set build trigger - this build job will be triggered in response to updated within the Git repository (e.g. via git push)



Build steps


A Maven Build step – to produce the ZIP file to be deployed to Application Container Cloud



Post-Build actions


Activate a post build action to archive the zip file





Execute Build


Before configuring deployment, we need to trigger the build in order to produce the artifacts which can be referenced by the deployment configuration




After the build is complete, you can check the archived artifacts




Continuous Deployment (CD) to Application Container Cloud


Repeat this process for both your application (producer and consumer)


Create a New Configuration for deployment





  • Enter the required details and configure the Deployment Target
  • Configure the Application Container Cloud instance
  • Configure Automatic deployment option on the final confirmation page
  • Provide content for manifest.json and deployment.json


You’ll end up with the below configuration (the view has been split into two parts)




Application Container Cloud defines two primary configuration descriptors – manifest.json and deployment.json, and each of them fulfill a specific purpose (more details here)




Click Save, initiate your deployment and wait for it to finish


Confirmation screen




Check your application(s) in Application Container Cloud




In the Deployment sub-section of the application details screen, notice that the required Service Bindings have been automatically wired and the environment variables have been populated as well (only a couple of variables have been highlighted below)




Test the application


The details to test the application are the same as described in this section of the previous blog. It’s really simple and here are the high level steps

  • Start your producer application using its REST URL, and
  • Access your consumer application


You should see the real time metrics being sent by the producer component to the Event Hub cloud service instance and consumed by the Server-Sent event (SSE) client via the consumer microservice


Test the CI/CD flow

Make some code changes and push them to the Developer Cloud service Git repo. This should

  • Automatically trigger the build, which once successful will
  • Automatically trigger the deployment process, and
  • Redeploy the new application version to Application Container Cloud



**The views expressed in this post are my own and do not necessarily reflect the views of Oracle.