Forum Stats

  • 3,839,693 Users
  • 2,262,529 Discussions
  • 7,901,042 Comments

Discussions

Building a Scalable, Highly Available Oracle API Gateway 11g Infrastructure in a Cloud Environment

Bob Rhubart-Oracle
Bob Rhubart-Oracle Member Posts: 692 Silver Badge
edited Mar 5, 2015 11:12AM in API Management
by Marcelo Parisi

Introduction

One of the major challenges that companies face in adopting a cloud computing platform is the secure provisioning of services in the cloud. Oracle API Gateway (OAG) 11g can be a very powerful tool in this sense, since it focuses on service protection, with authentication mechanisms, message encryption, and security/policy functionalities.

In this article, we will see how to create a cloud-based OAG infrastructure, with high-availability and scalability support. Both high-availability and scalability operations will be covered here. We’ll be using virtual machines (VMs) and storage concepts, along with OAG and Oracle Traffic Director (OTD). While a physical load balancer will also be necessary, its configuration is beyond the scope of this article.

The service infrastructure—Oracle SOA Suite, Oracle Service Bus or any other kind of service provider environment that needs to be exposed in a secure manner through the environment we’ll be building—will also not be covered in this article.

This article assumes a Network File System (NFS) v4 and Network Information Service/Lightweight Directory Service Protocol (NIS/LDAP) compliant environment. If you don’t support it, the article will indicate the changes so that you can run on a NFSv3 environment without NIS/LDAP.

There is no capacity planning or sizing work done on this article. The number of CPUs, memory and filesystem size are all just for demonstration purposes and should be revisited in a production environment.

OAG and OTD documentation should always be consulted. This document is not intended to replace any of the product’s official documentation.

Finally, please note that OTD is supported only in Exalogic environments.

Infrastructure Architecture

In this article, we’re going to build a brand new infrastructure from scratch to support this environment. We’ll consider two VMs for OTD and, initially, three VMs for OAG, one of them for administration purposes only. The environment infrastructure architecture will resemble the architecture in Figure 1, below:

parisi-api-fig01.png

Figure 1

As you can see, we have high availability on both the OTD layer and the OAG layer. Both layers are scalable either horizontally or vertically. This article discusses scalability only on the OAG layer.

We’re going to create five VMs—three for OAG, with Oracle Linux 5.6; two with Oracle Linux 6.6 for OTD. I suggest using VM Templates or cloning to make this task easier. The VMs’ configuration should resemble the table in Figure 2, below:

parisi-api-fig02.png

Figure 2

On the storage side, to hold this environment we’re going to create six filesystems that will be shared accross the tools. Make sure your VMs have all the packages necessary to run NFS mounts. The storage configuration should resemble the table in Figure 3, below:

parisi-api-fig03.png

Figure 3

On the network side, we’ll be using two virtual network interfaces on each VM, as Figure 2 shows. One of the virtual network interfaces is for the storage network and administrative traffic, and the other one is for the public network. Figure 4, below, depicts network and virtual IP (VIP) configuration:

parisi-api-fig04.png

Figure 4

Some other VIPs must be mapped in order to use the services we are going to provide, as shown in Figure 5, below:

parisi-api-fig05.png

Figure 5

Software Requirements

As mentioned above, the base operating system for this environment will be Oracle Linux. The following downloads must be performed:


Environment Preparation

  1. If you don’t have NIS/LDAP running, create an Oracle user and group on all VMs with the same User ID (UID) and Group ID (GID). For example, on vmoag01 it would look like this:

    [[email protected] ~]# groupadd -g 54321 oinstall

    [[email protected] ~]# useradd -g 54321 -u 54321 -m oracle

  2. Create a directory to handle the environment, and give write permissions to the Oracle user. These steps should be done on all VMs. As an example, on vmotd02, it would look like this:

    [[email protected] ~]# mkdir /u01

    [[email protected] ~]# chown oracle:oinstall /u01

  3. As Oracle user, create the commons directory to handle the installation files. This directory should be created with the Oracle user created before, and should be created the same way all over the environment. For example, on vmoagadm, it should look like this:

    [[email protected] ~]$ mkdir -p /u01/oracle/commons

  4.   Create the directory structure for OAG. On vmoagadm, vmoag01 and vmoag02, perform these steps:

    [[email protected] ~]$ mkdir -p /u01/oracle/logs

    [[email protected] ~]$ mkdir -p /u01/oracle/oag

  5. Map shared storage to those directories. Edit /etc/fstab as root and add the lines below (assuming my storage is on SN01.priv.parisi.spo) on vmoagadm, vmoag01 and vmoag02:

    SN01.priv.parisi.spo:/export/COMMONS      /u01/oracle/commons      nfs
    rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

    SN01.priv.parisi.spo:/export/OAG/oag      /u01/oracle/oag          nfs
    rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

    SN01.priv.parisi.spo:/export/OAG/logs     /u01/oracle/logs         nfs
    rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

  6. Mount the filesystems on all three VMs, with the following command as root:

    [[email protected] ~]# mount –a

    [[email protected] ~]# df -h

  7. To make sure the OAG instances communicate between them through the private network, add the lines below to /etc/hosts on vmoagadm, vmoag01 and vmoag02:

    172.16.14.22 vmoagadm

    172.16.14.23 vmoag01

    172.16.14.24 vmoag02

    This addition is necessary because, when installing, an instance announces itself to the Node Manager (and Node Manager back to the instance) with its host name. Since the environment resolves only fully qualified domain names (FQDNs), we need to add those lines to the VMs’ hosts so they can reach the instances through the private network.

  8. On vmotd01 and vmotd02, perform the steps below with Oracle user to create the directory structure to handle the OTD environment:

    [[email protected] ~]$ mkdir -p /u01/oracle/logs

    [[email protected] ~]$ mkdir -p /u01/oracle/otd

    [[email protected] ~]$ mkdir -p /u01/oracle/middleware

  9. Map the shared storage to those directories. Edit /etc/fstab as root and add the lines below (assuming storage is still on SN01.priv.parisi.spo) on vmotd01 and vmotd02:

    SN01.priv.parisi.spo:/export/COMMONS      /u01/oracle/commons      nfs<br/>      rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

    SN01.priv.parisi.spo:/export/OTD/otd      /u01/oracle/otd      nfs<br/>      rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

    SN01.priv.parisi.spo:/export/OTD/logs      /u01/oracle/logs      nfs<br/>      rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

    SN01.priv.parisi.spo:/export/OTD/middleware      /u01/oracle/middleware      nfs<br/>      rw,bg,hard,nointr,async,noatime,tcp,vers=4 0 0

  10. Mount the filesystems on all three VMs, with the following command as root:

    [[email protected] ~]# mount –a

    [[email protected] ~]# df -h

Note: The commons directory is shared accross all environments. Use the parameters wsize and rsize in fstab to fine-tune your NFS performance. Try to use jumbo frames on your network interface by changing the maximum transmission unit (MTU). If your environment is not ready for NFSv4, on fstab file change vers=4 to vers=3 for all NFS mount points.

Remember: to have a NFSv4 working, your environment must have NIS or LDAP service in place.

  1. To run OTD, install the packages below on vmotd01 and vmotd02:

    compat-libcap1.x86_64

    compat-libstdc++-33.x86_64

    compat-libstdc++-33.i686

    libgcc.i686

    libgcc.x86_64

    glibc.i686 glibc.x86_64

    libstdc++-devel.i686

    libstdc++-devel.i686

    libstdc++.x86_64

    libstdc++-devel.x86_64

    sysstat.x86_64

    gcc.x86_64

    gcc-c++.x86_64

    libaio.i686

    libaio.x86_64

    libaio-devel.i686

    libaio-devel.x86_64

    glibc-devel.i686

    glibc-devel.x86_64

  2. Finally, move the downloaded files below to /u01/oracle/commons:

    ofm_oag_linux_11.1.2.3.0_disk1_1of1.zip

    ofm_otd_linux_11.1.1.7.0_64_disk1_1of1.zip

The infrastructure is now ready to receive the new environment.

Installing OAG: vmoagadm

To install OAG on the admin virtual machine, as Oracle user, first uncompress the file:

[[email protected] ~]$ mkdir -p /u01/oracle/commons/OAG

[[email protected] ~]$ cd /u01/oracle/commons/OAG/

[[email protected] OAG]$ unzip ../ofm_oag_linux_11.1.2.3.0_disk1_1of1.zip

Run the installer file:

[[email protected] OAG]$ cd Linux/64bit/

[[email protected] 64bit]$ chmod +x OAG_11.1.2.3.0-linux-x86-64-installer.run

[[email protected] 64bit]$ ./OAG_11.1.2.3.0-linux-x86-64-installer.run --mode unattended<br/>    --enable-components apigateway,analytics,policystudio,configurationstudio,apitester --prefix<br/>    <span style="color: red;">/u01/oracle/oag/vmoagadm</span> --configureGatewayQuestion 0

After installation is complete, vmoagadm will be on /u01/oracle/oag/vmoagadm. To create a group for our instances in the web admin interface, go to https://vmoagadm.priv.parisi.spo:8090/. The default username is admin, and the default password is changeme. Go to the topology corner, click in Menu, and select Create New Group, as shown in Figure 6, below:

parisi-api-fig06.png

Figure 6

Name the new group oagcluster:

parisi-api-fig07.png

Figure 7

The topology should now resemble Figure 8, below:

parisi-api-fig08.png

Figure 8


Installing OAG: vmoag01

To install OAG on the first node and create an OAG instance inside the oagcluster group, run the installer with the following parameters:

[[email protected]vmoag01 ~]$ cd /u01/oracle/commons/OAG/Linux/64bit/

[[email protected] 64bit]$ ./OAG_11.1.2.3.0-linux-x86-64-installer.run --mode unattended --enable-components apigateway --prefix <span style="color: red;">/u01/oracle/oag/vmoag01</span> --firstInNewDomain <span style="color: red;">0</span> --rnmConnectionUrl <span style="color: red;"><a class="jive-link-external-small" href="https://vmoagadm.priv.parisi.spo:8090/" rel="nofollow">https://vmoagadm.priv.parisi.spo:8090</a></span> --gwName <span style="color: red;">vmoag01inst1</span> --gwGroup <span style="color: red;">oagcluster</span> --gwMgmtPort <span style="color: red;">8085</span> --gwServicesPort <span style="color: red;">8080</span>

The new instance will be called vmoag01inst1. The binary files will be located at /u01/oracle/oag/vmoag01. The instance management port is 8085; the instance traffic port is 8080.

Check the instance status by going to http://vmoag01.pub.parisi.spo:8080/healthcheck:

parisi-api-fig09.png

Figure 9


Installing OAG: vmoag02

To install OAG on the second node and create our OAG instance inside oagcluster in our topology, we need to run the installer with the following parameters:

[[email protected] ~]$ cd /u01/oracle/commons/OAG/Linux/64bit/

[[email protected] 64bit]$ ./OAG_11.1.2.3.0-linux-x86-64-installer.run --mode unattended --enable-components apigateway --prefix <span style="color: red;">/u01/oracle/oag/vmoag0</span>2 --firstInNewDomain <span style="color: red;">0</span> --rnmConnectionUrl <span style="color: red;"><a class="jive-link-external-small" href="https://vmoagadm.priv.parisi.spo:8090/" rel="nofollow">https://vmoagadm.priv.parisi.spo:8090</a></span> --gwName <span style="color: red;">vmoag02inst1</span> --gwGroup <span style="color: red;">oagcluster</span> --gwMgmtPort <span style="color: red;">8085</span> --gwServicesPort <span style="color: red;">8080</span>

The new instance will be called vmoag02inst1. The binary files will be located at /u01/oracle/oag/vmoag02. The instance management port is 8085; the instance traffic port is 8080.

Check the instance status by going to http://vmoag02.pub.parisi.spo:8080/healthcheck:

parisi-api-fig10.png

Figure 10

Checking OAG Infrastructure

To confirm that the installation was fine, connect to the administration interface again and check the topology. The console is at https://vmoagadm.priv.parisi.spo:8090/. The default login is admin; the default password is changeme.

The topology should look like this:

parisi-api-fig12.png

Figure 11

Installing OTD Binaries

We’ll install OTD binaries once only on the shared filesystem that will be used by both VMs. To install OTD binaries, on vmotd01, as Oracle user, first uncompress the file:

[[email protected] ~]$ mkdir -p /u01/oracle/commons/OTD

[[email protected] ~]$ cd /u01/oracle/commons/OTD/

[[email protected] OTD]$ unzip ../ofm_otd_linux_11.1.1.7.0_64_disk1_1of1.zip

Now run the installer file:

[[email protected] OTD]$ cd Disk1/

[[email protected] Disk1]$ chmod ./runInstaller

Only one step needs attention in the installation process.

Installation should be done on /u01/oracle/middleware/11.1.1.7.0/trafficdirector_Home_1, as shown in Figure 12 below:

parisi-api-fig12.png

Figure 12


Creating OTD Administration Nodes

After installing OTD binaries, set up the administration nodes to create the environment. To create the administration node on vmotd01 and start it, perform the commands below (you’ll be asked for an admin password: please remember it as you’ll need to use it again):

[[email protected] ~]$ cd /u01/oracle/middleware/11.1.1.7.0/trafficdirector_Home_1/bin/

[[email protected] bin]$ ./tadm configure-server --user=<span style="color: red;">otd_admin</span> --port=<span style="color: red;">8989</span> --host=<span style="color: red;">vmotd01.priv.parisi.spo</span> --instance-home=<<span style="color:red;">/u01/oracle/otd/vmotd01

[[email protected] bin]$ cd /u01/oracle/otd/vmotd01/admin-server/bin/

[[email protected] bin]$ ./startserv

The administration node will be created at /u01/oracle/ort/vmotd01, with port 8989 and hostname vmotd01.priv.parisi.spo. The administration user will be otd_admin. If you need to access the web admin interface, go to https://vmotd01.priv.parisi.spo:8989/.

To create and start the administration node on vmotd02, perform the following commands (you should use the admin password created in the earler step):

[[email protected] ~]$ cd /u01/oracle/middleware/11.1.1.7.0/trafficdirector_Home_1/bin/

[[email protected] bin]$ ./tadm configure-server --user=<span style="color: red;">otd_admin</span> --port=<span style="color: red;">8989</span> --host=<span style="color: red;">vmotd01.priv.parisi.spo</span> --admin-node --node-port=8989 --instance-home=<span style="color: red;">/u01/oracle/otd/vmotd02</span> --node-host=vmotd02.priv.parisi.spo

[[email protected] bin]$ cd /u01/oracle/otd/vmotd02/admin-server/bin/

[[email protected] bin]$ ./startserv

The administration node will be created at /u01/oracle/ort/vmotd02, with port 8989 and hostname vmotd02.priv.parisi.spo.

Setting Up OTD

Set up OTD using the command line tool. It is possible to perform all these steps on the web admin tool, but the configuration process is faster when the command line tool is used. To set up OTD, first get into its command line tool. To do that, run the following commands on vmotd01:

[[email protected] ~]$ cd /u01/oracle/middleware/11.1.1.7.0/trafficdirector_Home_1/bin/

[[email protected] bin]$ ./tadm --user=<span style="color: red;">otd_admin</span> --host=<span style="color: red;">vmotd01.priv.parisi.spo</span> --port=<span style="color: red;">8989</span>

Inside the command line tool, create a configuration file for our OTD environment. By default, when creating the configuration, OTD also creates an http-listener, an origin-server-pool and a virtual server. They’ll be deleted later. But, for now, to create the default-config, run the following command:

tadm> create-config --listener-port=<span style="color: red;">8090</span> --server-name=<span style="color: red;">services.parisi.spo</span> --origin-server=<span style="color: red;">vmotd01.pub.parisi.spo:8080,vmotd02.pub.parisi.spo:8080 default-config</span>

Now create an origin-server-pool pointing to the OAG instances. Set its load balancing algorithm as “round robin”:

tadm> create-origin-server-pool --config=<span style="color: red;">default-config</span> --type=http --origin-server=vmoag01.pub.parisi.spo:8080,vmoag02.pub.parisi.spo:8080 <span style="color: red;">oag-server-pool</span>

tadm> set-origin-server-pool-prop --config=<span style="color: red;">default-config</span> --origin-server-pool=<span style="color: red;">oag-server-pool</span> load-distribution=<span style="color: red;">round-robin</span>

Now we’re going to adjust some probing parameters used by OTD to check if the Oracle API Gateway is live. As mentioned, OAG provides a healthcheck service; set up OTD to check for that service by changing the HTTP method, the URL of the service and the content expected in the results:

tadm> set-health-check-prop --config=<span style="color: red;">default-config</span> --origin-server-pool=<span style="color: red;">oag-server-pool</span> request-method=<span style="color: red;">GET</span>

tadm> set-health-check-prop --config=<span style="color: red;">default-config</span> --origin-server-pool=<span style="color: red;">oag-server-pool</span> request-uri=<span style="color: red;">/healthcheck</span>

tadm> set-health-check-prop --config=<span style="color: red;">default-config</span> --origin-server-pool=<span style="color: red;">oag-server-pool</span> response-body-match=<span style="color: red;"><status>ok</status></span>

After adjusting the probing parameters, create a new http-listener on the OTD configuration, to listen to the requests on port 8080:

tadm> create-http-listener --config=<span style="color: red;">default-config</span> --listener-port=<span style="color: red;">8080</span> --server-name=<span style="color: red;">services.parisi.spo</span> --default-virtual-server-name=<span style="color: red;">default-config services-listener</span>

Now, create a virtual-server on the OTD configuration, which is responsible for the domain name services.parisi.spo, the domain name that will point to the OAG environment. After that, point http-listeners to the created virtual-server:

tadm> create-virtual-server --config=<span style="color: red;">default-confi</span>g --http-listener=<span style="color: red;">services-listener</span> --host-pattern=<span style="color: red;">services.parisi.spo</span> --log-file=<span style="color: red;">/u01/oracle/logs/services.parisi.spo.log</span> --origin-server-pool=<span style="color: red;">oag-server-pool services.parisi.spo</span>

tadm> set-http-listener-prop --config=<span>default-config</span> --http-listener=<span>services-listener</span> default-virtual-server-name=<span style="color: red;">services.parisi.sp</span>o

tadm> set-virtual-server-prop --config=<span style="color: red;">default-config</span> --vs=<span style="color: red;">default-config</span> http-listener-name=<span style="color: red;">services-listener</span>

tadm> set-http-listener-prop --config=<span style="color: red;">default-config</span> --http-listener=<span style="color: red;">http-listener-1</span> default-virtual-server-name=<span style="color: red;">services.parisi.spo</span>

Next, enable logging in, both for OTD itself, and for the new virtual-server. to help troubleshoot:

tadm> enable-access-log --config=<span style="color: red;">default-config</span> --vs=<span style="color: red;">services.parisi.spo</span> --file=<span style="color: red;">/u01/oracle/logs/services.parisi.spo-access.log</span>

tadm> set-log-prop --config=<span style="color: red;">default-confi</span>g log-file=<span style="color: red;">/u01/oracle/logs/otd-error.log</span>

tadm> enable-access-log --config=<span style="color: red;">default-config</span> --file=<span style="color: red;">/u01/oracle/logs/otd-access.log</span>

Now, delete everything created by the create-config command that won’t be used:

tadm> delete-virtual-server --config=<span style="color: red;">default-config default-config</span>

tadm> delete-origin-server-pool --config=<span style="color: red;">default-config origin-server-pool-1</span>

tadm> delete-http-listener --config=<span style="color: red;">default-config http-listener-1</span>

To accept traffic, create OTD instances. As described in the architecture, there will be two instances:

tadm> create-instance --config=<span style="color: red;">default-config vmotd01.priv.parisi.spo</span>

tadm> create-instance --<span style="color: red;">config=default-config vmotd02.priv.parisi.spo</span>

Finally, start the OTD instances:

tadm> start-instance --config=<span style="color: red;">default-config vmotd01.priv.parisi.spo</span>

tadm> start-instance --config=<span style="color: red;">default-config vmotd02.priv.parisi.spo</span>


Virtualizing Services

To expose services through OAG, virtualize them inside the topology. To demonstrate that, I’ll use a dummy service I have running in my environment, but you can use any service you have.

To virtualize our service, we’ll open OAG Policy Studio. You can launch OAG Policy Studio from vmoagadm. The tool is installed at /u01/oracle/oag/vmoagadm/oagpolicystudio. Run OAG Policy Studio pointing to vmoagadm.priv.parisi.spo on port 8090 using SSL. Default login is admin and default password is changeme. Figure 13 gives an example:

parisi-api-fig13.png

Figure 13

Edit the Active Configuration for oagcluster as shown in Figure 14, below:

parisi-api-fig14.png

Figure 14

Once inside OAG Policy Studio, click on Business Services in the OAG tree on the left side and click on Virtualise a Service, as shown in Figure 15:

parisi-api-fig15.png

Figure 15

The WSDL URL for the WebService should go through the internal Load Balancer/OTD/Oracle HTTP Server, as in Figure 16:

parisi-api-fig16.png

Figure 16

Now, select a user name for the imported WSDL and a Comment, as shown in Figure 17:

parisi-api-fig17.png

Figure 17

Next, select the WSDL Operations that will be virtualized:

parisi-api-fig18.png

Figure 18

In the following screen, WS-Policy Options, just click Next, as shown in Figure 19. (This is related to the service security and policy side of OAG, and won’t be covered here.)

parisi-api-fig19.png

Figure 19

Finally, select where to expose the virtualized service. We’ll select Default Services, as Figure 20 shows:

parisi-api-fig20.png

Figure 20

You should perform these steps for each service that is to be exposed through OAG. In my case, I have two virtualized services for demonstration purposes.


Testing

To test our environment, I suggest you use a load testing tool, pointing to services.parisi.spo, which is the host for our physical load balancer fronting OTD.

I have set up a simple load test tool to send transactions to the load balancer. You can use API Explorer (provided with OAG) to generate some sample request XMLs for testing purposes.

I won’t get into details about performing the tests—just do it the same way you test any other service. The main difference here is that you need to point the test to the balancer. Below, we’ll discuss the results of the tests I’ve run in the environment we’ve built so far.

As shown in Figure 21, in 10 minutes, I could send ~127k transations to my load balancer—and it has hit our OAG environment:

parisi-api-fig21.png

Figure 21

Those transactions were handled by our oagcluster, as shown in Figure 22:

parisi-api-fig22.png

Figure 22

Figure 23 demonstrates that vmoag01inst1, our first OAG instance, has handled ~63k transactions, which is approximately half of our transaction total:

parisi-api-fig23.png

Figure 23

Finally, Figure 24 shows that our second instance, vmoag02inst1, has also handled ~63k transactions, which is aproximately half of the transactions, meaning we have good load balancing within our cluster:

parisi-api-fig24.png

Figure 24

If one of the instances goes down, OTD will detect it, and all the traffic will be redirected to the other instances available. This procedure should be transparent, as shown in Figure 25 below, with vmoag02inst1 turned off, and no failures or exceptions:

parisi-api-fig25.png

Figure 25


Scaling Out

Scaling out the environment means scaling it by adding more computers or VMs to the environment. OAG is highly scalable and, in fact, this architecture we’ve built is very simple to scale.

We’ll create a new VM, based on vmoag01, which we’ll call vmoag03. It needs to have the same configuration so we won’t impact the SLA of our services; that means 2 vCPUs, 2 vNICs, 1536 MB of RAM and 20GB of disk. ThisVM, like the others supporting OAG, should have Oracle Linux 5.6.

For networking, we’ll have vmoag03.priv.parisi.spo as 172.16.14.25 and vmoag03.pub.parisi.spo as 10.0.0.25.

First, we need to review our environment preparation chapter and prepare this VM. After we have everything in place, including groups needed, users needed and shared storage mounted, we must add the following entries to its /etc/hosts:

172.16.14.22    vmoagadm

172.16.14.23    vmoag01

172.16.14.24    vmoag02

172.16.14.25    vmoag03

We’ll also need to add this line to /etc/hosts on vmoagadm, vmoag01 and vmoag02:

172.16.14.25 vmoag03

Now, we’ll execute OAG installer on vmoag03:

[[email protected] ~]$ cd /u01/oracle/commons/OAG/Linux/64bit/

[[email protected] 64bit]$ ./OAG_11.1.2.3.0-linux-x86-64-installer.run --mode unattended --enable-components apigateway --prefix <span style="color: red;">/u01/oracle/oag/vmoag03</span> --firstInNewDomain 0 --rnmConnectionUrl <span style="color: red;"><a class="jive-link-external-small" href="https://vmoagadm.priv.parisi.spo:8090/" rel="nofollow">https://vmoagadm.priv.parisi.spo:8090</a></span> --gwName <span style="color: red;">vmoag03inst1</span> --gwGroup <span style="color: red;">oagcluster</span> --gwMgmtPort <span style="color: red;">8085</span> --gwServicesPort <span style="color: red;">8080</span>

The new instance is called vmoag03inst1 and the binary files are located at /u01/oracle/oag/vmoag03. The management port is 8085 and the instance traffic port is 8080.

After that, we’ll check our topology on the OAG web admin interface by going to https://vmoagadm.priv.parisi.spo:8090/ in the web browser.

parisi-api-fig26.png

Figure 26

As seen in Figure 26, the instance is already there, but there is no traffic coming to it. Now we just need to add the instance to our OTD origin-server-pool. To do that, we’ll first execute the command line tool on vmotd01:

[[email protected] ~]$ cd /u01/oracle/middleware/11.1.1.7.0/trafficdirector_Home_1/bin/

[[email protected] bin]$ ./tadm --user=<span style="color: red;">otd_admin</span> --host=<span style="color: red;">vmotd01.priv.parisi.spo</span> --port=<span style="color: red;">8989</span>

Inside the command line tool, let’s add our new instance to the oag-server-pool:

tadm> create-origin-server --config=<span style="color: red;">default-config</span> --origin-server-pool=<span style="color: red;">oag-server-pool vmoag03.pub.parisi.spo:8080</span>

tadm> deploy-config <span style="color: red;">default-config</span>

We should see traffic already coming to our new instance, as shown in Figure 27, below:

parisi-api-fig27.png

Figure 27

We’re done! We now have a scaled environment. Note that I’ve scaled this environment without stopping the testing tools; in other words, I scaled it with transactions running through, as if it were a live production environment, and it was completely transparent. No failures or exceptions were seen.


Scaling Up

Scaling up the environment means adding more OAG instances to the VMs we already have. This procedure should be even simpler than scaling out.

The table in Figure 28 shows the new instances that will be created:

parisi-api-fig28.png

Figure 28

To achieve this configuration, we need to create the instances through OAG’s web admin interface, available at https://vmoagadm.priv.parisi.spo:8090/. To create vmoag01inst2, click on oagcluster and select New Gateway Server, as in Figure 29, below:

parisi-api-fig29.png

Figure 29

Now we just need to fill in the fields for vmoag01inst2, as in Figure 30:

parisi-api-fig30.png

Figure 30

For vmoag02inst2, it should look like Figure 31:

parisi-api-fig31.png

Figure 31

Finally, for vmoag03inst2 it should resemble Figure 32:

parisi-api-fig32.png

Figure 32

To start the instances, just click in the instance and select Start, as in Figure 33:

parisi-api-fig33.png

Figure 33

After starting all the instances created, our topology should resemble Figure 34:

parisi-api-fig34.png

Figure 34

As a last step, we just need to update OTD to send traffic to those new instances. To do that, we’ll log into vmotd01 and run command line tool:

[[email protected] ~]$ cd /u01/oracle/middleware/11.1.1.7.0/trafficdirector_Home_1/bin/

[[email protected] bin]$ ./tadm --user=otd_admin --host=vmotd01.priv.parisi.spo --port=8989

Now we update our oag-server-pool with the new instances, and we should see traffic coming to them:

tadm> create-origin-server --config=<span style="color: red;">default-config</span> --origin-server-pool=<span style="color: red;">oag-server-pool vmoag01.pub.parisi.spo:9090</span>         
tadm> create-origin-server --config=default-config --origin-server-pool=<span style="color: red;">oag-server-pool vmoag02.pub.parisi.spo:9090</span> tadm> create-origin-server --config=default-config --origin-server-pool=oag-server-pool vmoag03.pub.parisi.spo:9090 tadm> deploy-config default-config



As we can see in Figure 35, the traffic is already coming to one of our new instances:

parisi-api-fig35.png

Figure 35

And again, as with the scale out process, this process was done on the fly, with transactions going through as if it were a production environment. No errors or exceptions were raised.

Conclusion

Companies have been facing huge challenges—notably, security—in the effort to adapt to the new Cloud Computing paradigm. OAG is a robust tool for service provisioning and security in a Cloud environment, but building, creating and managing an OAG environment can turn into a set of really complex activities.

OAG is a highly scalable, high-availability tool. This article has documented a very simple way to build a complete OAG environment from scratch, and has demonstrated that the environment thus built is fault tolerant. Finally, the article has shown that the environment can be scaled right away, both horizontally and vertically, and with transactions going through, without raising any failures or exceptions on the running environment.

Note: This document is not an Oracle official documentation and is not intended to replace any of product’s official documentation. This document is intended to be a simple guideline for building an environment. Offical product documentation should always be reviewed.

About the Author

Marcelo Parisi is an Oracle IT Architecture Certified Specialist working as a Senior Consultant for Oracle Consulting Services in Brazil, where he is a member of the Tech Infrastructure team. He works as an infrastructure architect and consultant for the Oracle Fusion Middleware products family. His main roles includes architecture design, implementation and performance tuning of FMW infrastructure. Marcelo's fields of expertise include Oracle Reference Architecture, Oracle WebLogic Server, Oracle SOA Suite and Oracle WebCenter.

https://www.twitter.com/feitnomore

https://www.linkedin.com/in/marceloparisi

https://www.facebook.com/marcelo.f.pari

«1

Comments

  • Karunanidhi-Oracle
    Karunanidhi-Oracle Member Posts: 13 Employee

    nice work , can otd supplemented with other lb's like big ip /ohs

  • MarceloParisi-Oracle
    MarceloParisi-Oracle Member Posts: 57
    edited Feb 19, 2015 1:38PM

    nice work , can otd supplemented with other lb's like big ip /ohs

    In the past I've done this with OHS and mod_proxy / proxy_balance. You can work with other lbrs.
    BTW, thanks...

  • bornesel
    bornesel Member Posts: 267

    thanks for great article!

    You wrote:

    Finally, please note that OTD is supported only in Exalogic environments
    

    OTD is supported in the ODA VP too

  • thanks for great article!

    You wrote:

    Finally, please note that OTD is supported only in Exalogic environments
    

    OTD is supported in the ODA VP too

    Thank you! Good to know OTD is supported on another platform. Regards!

  • On OHS/Apache, it should be something similar to this:

    --------------------------------------------------------------------------------

    <VirtualHost *:7777>

        ProxyRequests off

        ServerName services.parisi.spo

        <Proxy balancer://apibalancer>

            BalancerMember http://vmoag01.pub.parisi.spo:8080

            BalancerMember http://vmoag02.pub.parisi.spo:8080

            # Security

            Order Deny,Allow

            Deny from none

            Allow from all

            # Balancer Settings

            ProxySet lbmethod=byrequests

        </Proxy>

        <Location /balancer-manager>

            SetHandler balancer-manager

            # Security

            Order Deny,Allow

            Allow from all

        </Location>

        ProxyPass /balancer-manager !

        ProxyPass / balancer://apibalancer/

    </VirtualHost>

    --------------------------------------------------------------------------------

    Thanks!

  • Marc Pérez
    Marc Pérez Member Posts: 11

    Hi Marcelo,

    Good article. I was wondering if it's possible to "environmentalise" Business Services as I couldn't find a way to do so. I'm pretty sure that in most cases when promoting a Policy across environments, the endpoint of the exposed service and it's WSLD will change. How would you manage this? would you have to go into Policy Studio and reload from WSDL URL manually?

    Thanks,

    Marc

  • anishhshah
    anishhshah Member Posts: 7 Blue Ribbon

    Hi Marcelo - qq,

    • We are scaling the integration layer where interface volume will increase.  We also need to do reconfiguration of some of the existing interface policies in order to begin isolating some of the traffic so it can handle the volume.  How do you determine the impact of scalling the OAG layer? Is there any standard template available for such a effort ?? or OAG scale out is pretty much straight forward with minimum efforts in policy migration ??

    • Do we need to build some additional v-servers & reconfigure some of the existing interface policies to isolate the trafic in order to handle the volume?  What additional efforts are needed for the install, config & OAG policy migration?  Is the reconfiguration needed in both source & additional servers?

    Thanks

    Anish

  • Hi Marcelo,

    Good article. I was wondering if it's possible to "environmentalise" Business Services as I couldn't find a way to do so. I'm pretty sure that in most cases when promoting a Policy across environments, the endpoint of the exposed service and it's WSLD will change. How would you manage this? would you have to go into Policy Studio and reload from WSDL URL manually?

    Thanks,

    Marc

    Hello Marc,

    I am not 100% sure about the answer, as I am more the kind of "infrastructure" guy. I don't do too much the functional part (policies, etc...), but have you looked at Configuration Studio? I think this is the tool you are looking for, as it is used to promote policies and circuits across different environments (dev, test, prod...).

    Thanks,

    Parisi

  • Hi Marcelo - qq,

    • We are scaling the integration layer where interface volume will increase.  We also need to do reconfiguration of some of the existing interface policies in order to begin isolating some of the traffic so it can handle the volume.  How do you determine the impact of scalling the OAG layer? Is there any standard template available for such a effort ?? or OAG scale out is pretty much straight forward with minimum efforts in policy migration ??

    • Do we need to build some additional v-servers & reconfigure some of the existing interface policies to isolate the trafic in order to handle the volume?  What additional efforts are needed for the install, config & OAG policy migration?  Is the reconfiguration needed in both source & additional servers?

    Thanks

    Anish

    Hello

    For the 1st question, from my understanding OAG scaling is pretty straight forward. Just make sure you deploy the policies across the nodes so that you achieve consistency on your "group". I don't know about any template available for this.

    In this case, I think the best thing to do in this case is test. Performing tests, you'll be able to determine if your actual environment can handle everything, or if you need to create new v-servers. Also, test both scenarios: isolating the policies, and putting them all together.

    Thanks,

    Marcelo

  • anishhshah
    anishhshah Member Posts: 7 Blue Ribbon

    Marcelo - Thanks for your reply. Expansion is for Production environment. If we need to define the estimates now how could we determine the capacity with increase in number of transaction volume and we can't really do performance testing in production ?? Also are there any benchmark results wrt to OAG capacity expansion ??

    Thanks

    Anish