Automated Application Development and Deployment with DevOps on Oracle Solaris 11

Version 2

    by Glynn Foster

     

    Learn how you can move to a more agile process of development and deployment using Jenkins, Git, Maven, IPS and Puppet.

     

    Introduction

     

    One of the biggest buzzwords of the IT industry today is DevOps. DevOps is a term that was coined to describe a software development process that focuses on collaboration and automation, merging the traditional silos of development, quality assurance, and operations. DevOps covers the entire application delivery pipeline, enabling organizations to rapidly develop applications, continuously deploy them, and incorporate feedback back into the development cycle faster. This often means faster development cycles, reduced risk through incremental changes, and the ability to test these changes with a "fail faster" mentality. Using the DevOps approach, operations are typically programmable and automated on standardized platforms, and driven from an application-centric view.

     

    By far the biggest hurdle in moving to a DevOps model is the organizational changes required: aligning people from different departments and establishing a clear understanding for a new development process—whether that's managing particular tasks through Scrum or Kanban methodologies, instilling an agile set of values, or some other practice that suits an organization. This article focuses on the required tooling and how you can chain different technologies together to provide more-agile application delivery.

     

    There is no single correct way with DevOps, and there are many different technologies that can achieve very similar results. The intent of this article is to show that Oracle Solaris 11 is an equally capable platform for achieving this business agility, without compromising the fundamental enterprise-proven architectures of a more traditional approach. Let's begin our journey by taking a look at Jenkins.

     

    Configuring the Jenkins Continuous Integration Server

     

    Jenkins is a popular open source continuous build and integration server. It allows users to continuously build and test software using automation as the software is developed, and it provides observability into builds and build logs. Jenkins supports the ability for distributed computing and can be extended with the addition of third-party plugins to meet any organization's requirements.

     

    We will install Jenkins on Oracle Solaris 11.3 within an Oracle Solaris Kernel Zone. We can either create a jenkins user that will be responsible for running the Jenkins environment manually, or we can use a System Configuration profile that is used when installing the Oracle Solaris Kernel Zone.

     

    root@solaris:~# useradd -m jenkins
    root@solaris:~# passwd jenkins
    New Password:  Re-enter new Password:  passwd: password successfully changed for jenkins root@solaris:~# su - jenkins

     

    Once we log in as the jenkins user, we can explore our environment:

     

    jenkins@solaris:~$ uname -a
    SunOS solaris 5.11 11.3 sun4v sparc sun4v jenkins@solaris:~$ virtinfo
    NAME            CLASS      kernel-zone     current    logical-domain  parent     non-global-zone supported

     

    Jenkins is not currently included in the Oracle Solaris Image Packaging System (IPS) package repository, so let's first download a copy of Jenkins (in this case, the web application archive [WAR] file) that we will use to set up our build server.

     

    jenkins@solaris:~$ wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war

     

    We need to ensure that the Jenkins build server is always running. We will integrate it with the Oracle Solaris 11 Service Management Facility (SMF) for better reliability; if the service fails for any reason, SMF will automatically restart it. To do this, we need to write an SMF manifest for Jenkins. We can use the svcbundle(1M) utility to quickly create a skeletal manifest. This will create a new service called site/jenkins with a single instance: default. As its start method, the manifest will run our WAR file.

     

    jenkins@solaris:~$ svcbundle -o jenkins.xml -s service-name=site/jenkins \
    > -s model=child -s instance-name=default \
    > -s start-method="java -jar /export/home/jenkins/jenkins.war"

     

    Once we have generated a simple SMF manifest, we'll need do some additional work to ensure that we're running Jenkins using the jenkins user. To do this we'll need to include a method_context clause. This also allows us to be able to define a JENKINS_HOME environmental variable that will be set when running the Jenkins server. For simplicity, let's just point this variable at the jenkins user's home directory.

     

    <method_context>     <method_credential group='nobody'         privileges='basic,net_privaddr' user='jenkins'/>     <method_environment>         <envvar name='JENKINS_HOME' value='/export/home/jenkins'/>     </method_environment> </method_context>

     

    Before we associate the SMF manifest with the running system, it's always a good idea to validate the SMF manifest that we've produced to ensure it's correct. We can use the svccfg validate command to achieve this, as follows:

     

    jenkins@solaris:~$ svccfg validate jenkins.xml
    jenkins@solaris:~$

     

    The command returned without any errors, so we can assume the manifest is good. Now let's copy this manifest to the site-specific directory location for SMF manifests:

     

    jenkins@solaris:~$ sudo cp jenkins.xml /lib/svc/manifest/site/
    Password: jenkins@solaris:~$

     

    Now we can restart the manifest-import:default SMF service instance to associate this manifest with our running system.

     

    jenkins@solaris:~$ sudo svcadm restart manifest-import:default

     

    Let's check that the Jenkins SMF service is online:

     

    jenkins@solaris:~$ svcs jenkins
    STATE          STIME    FMRI online         18:10:24 svc:/site/jenkins:default

     

    Once the SMF service is online, we can open our web browser and navigate to http://host.oracle.com:8080 to see the following screen:

     

    f1.png

    Figure 1. The initial Jenkins dashboard screen.

     

    We have a running Jenkins instance. The next step is to create an application that we will use as a base for our Jenkins executions.

     

    Developing a Simple Java Application with Git and Maven

     

    All good software development begins by installing a distributed revision control management system for source code. There are a number of options available in the IPS repository, but let's choose Git. Git is a popular open source solution used in many high-profile open source projects. We can install it using the Oracle Solaris packaging tools:

     

    root@solaris:~# pkg install git
               Packages to install:  1             Services to change:  1        Create boot environment: No Create backup boot environment: No  DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED Completed                                1/1       334/334      7.8/7.8  3.6M/s  PHASE                                          ITEMS Installing new actions                       503/503 Updating package state database                 Done  Updating package cache                           0/0  Updating image state                            Done  Creating fast lookup database                   Done  Updating package cache                           1/1

     

    Apache Maven is an open source build automation tool used by many Java-based applications. Maven uses an XML file called the Project Object Model (POM) to describe how software should be built and what the dependencies are. Apache Maven is not currently included in the IPS repository, so we will need to download it separately.

     

    Once Maven has been downloaded, we will unpack the tarball in /opt.

     

    root@solaris:~# wget http://mirror.nexcess.net/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz
    root@solaris:~# cd /opt
    root@solaris:/opt# tar -zxf /root/apache-maven-3.3.3-bin.tar.gz

     

    Because we are developing a Java-based application, we also need to install the Java Development Kit (JDK) from the IPS repository:

     

    root@solaris:~# pkg install jdk-8
               Packages to install:  2        Create boot environment: No Create backup boot environment: No  DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED Completed                                2/2       625/625    46.3/46.3  9.0M/s  PHASE                                          ITEMS Installing new actions                       736/736 Updating package state database                 Done  Updating package cache                           0/0  Updating image state                            Done  Creating fast lookup database                   Done  Updating package cache                           1/1

     

    Let's adjust our $PATH environmental variable and check that Maven works:

     

    demo@solaris:~$ export PATH=/opt/apache-maven-3.3.3/bin/:$PATH
    demo@solaris:~$ mvn --version
    Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T04:57:37-07:00) Maven home: /opt/apache-maven-3.3.3 Java version: 1.8.0_60, vendor: Oracle Corporation Java home: /usr/jdk/instances/jdk1.8.0/jre Default locale: en, platform encoding: ISO646-US OS name: "sunos", version: "5.11", arch: "sparcv9", family: "unix"

     

    As a very simple Java application, let's use the canonical "Hello World" Maven example with the following command:

     

    demo@solaris:~$ mvn archetype:generate -DgroupId=com.myproj.app -DartifactId=myproj \
    -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
    [INFO] Scanning for projects... Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom (4 KB at 4.5 KB/sec) Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/22/maven-plugins-22.pom ... [INFO] ---------------------------------------------------------------------------- [INFO] Using following parameters for creating project from Old (1.x) Archetype: maven-archetype-quickstart:1.0 [INFO] ---------------------------------------------------------------------------- [INFO] Parameter: basedir, Value: /export/home/demo [INFO] Parameter: package, Value: com.myproj.app [INFO] Parameter: groupId, Value: com.myproj.app [INFO] Parameter: artifactId, Value: myproj [INFO] Parameter: packageName, Value: com.myproj.app [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] project created from Old (1.x) Archetype in dir: /export/home/demo/myproj [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 28.062 s [INFO] Finished at: 2015-09-09T18:59:55-07:00 [INFO] Final Memory: 16M/146M [INFO] ------------------------------------------------------------------------  demo@solaris:~$ cd myproj/
    demo@solaris:~/myproj$ find .
    . ./pom.xml ./src ./src/main ./src/main/java ./src/main/java/com ./src/main/java/com/myproj ./src/main/java/com/myproj/app ./src/main/java/com/myproj/app/App.java ./src/test ./src/test/java ./src/test/java/com ./src/test/java/com/myproj ./src/test/java/com/myproj/app ./src/test/java/com/myproj/app/AppTest.java

     

    As we can see in the output above, the tool has created the default directory structure used in all Maven projects that includes the application itself, along with unit tests that will take advantage of the JUnit test framework.

     

    Let's take a look at the application source code:

     

    demo@solaris:~/myproj$ cat src/main/java/com/myproj/app/App.java
    package com.myproj.app;  /**  * Hello world!  *  */ public class App  {     public static void main( String[] args )     {         System.out.println( "Hello World!" );     } }

     

    The utility has also generated a POM file. Maven uses this file to describe the project and its dependencies.

     

    demo@solaris:~/myproj$ cat pom.xml
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">   <modelVersion>4.0.0</modelVersion>   <groupId>com.myproj.app</groupId>   <artifactId>myproj</artifactId>   <packaging>jar</packaging>   <version>1.0-SNAPSHOT</version>   <name>myproj</name>   <url>http://maven.apache.org</url>   <dependencies>     <dependency>       <groupId>junit</groupId>       <artifactId>junit</artifactId>       <version>3.8.1</version>       <scope>test</scope>     </dependency>   </dependencies> </project>

     

    Before we go any further and try to build this project, we'll check it into our Git repository for some version control. We start by initializing our Git repository:

     

    demo@solaris:~/myproj$ git init
    Initialized empty Git repository in /export/home/demo/myproj/.Git/

     

    We also need to set an identity that will be used to check in our code:

     

    demo@solaris:~/myproj$ git config --global user.email "demo@oracle.com"
    demo@solaris:~/myproj$ git config --global user.name "Demo User"

     

    Now let's commit these files to our repository:

     

    demo@solaris:~/myproj$ git commit -m "Initial project import"
    [master (root-commit) ca2afba] Initial project import  3 files changed, 69 insertions(+)  create mode 100644 pom.xml  create mode 100644 src/main/java/com/myproj/app/App.java  create mode 100644 src/test/java/com/myproj/app/AppTest.java

     

    To build our project, we will call mvn compile, as follows:

     

    demo@solaris:~/myproj$ mvn compile
    [INFO] Scanning for projects... [INFO]                                                                          [INFO] ------------------------------------------------------------------------ [INFO] Building myproj 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom (8 KB at 14.8 KB/sec) ... [INFO] Changes detected - recompiling the module! [WARNING] File encoding has not been set, using platform encoding ISO646-US, i.e. build is platform dependent! [INFO] Compiling 1 source file to /export/home/demo/myproj/target/classes [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 6.751 s [INFO] Finished at: 2015-09-09T20:28:59-07:00 [INFO] Final Memory: 15M/139M [INFO] ------------------------------------------------------------------------

     

    You will notice from the output above that Maven will automatically download a series of different components from the Maven repository that it requires in order to compile this project. Most of this will be plugins to Maven to allow different functionality to be exposed. Our Java application is so simple that it won't really have many dependencies.

     

    Once we have successfully compiled the code, we can also do a test run using the test target. Maven will download the related Maven plugins it needs in order to run the tests.

     

    demo@solaris:~/myproj$ mvn test
    [INFO] Scanning for projects... [INFO]                                                                          [INFO] ------------------------------------------------------------------------ [INFO] Building myproj 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom (11 KB at 19.0 KB/sec) ... -------------------------------------------------------  T E S T S ------------------------------------------------------- Running com.myproj.app.AppTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec  Results :  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0  [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.331 s [INFO] Finished at: 2015-09-09T22:35:55-07:00 [INFO] Final Memory: 16M/146M [INFO] ------------------------------------------------------------------------

     

    Let's now take the next step and build a JAR file for this project using the package target.

     

    demo@solaris:~/myproj$ mvn package
    [INFO] Scanning for projects... [INFO]                                                                          [INFO] ------------------------------------------------------------------------ [INFO] Building myproj 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.pom Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.pom (6 KB at 10.2 KB/sec) ... [INFO]  [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ myproj --- [WARNING] Using platform encoding (ISO646-US actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /export/home/demo/myproj/src/main/resources [INFO]  [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ myproj --- [INFO] Nothing to compile - all classes are up to date [INFO]  [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ myproj --- [WARNING] Using platform encoding (ISO646-US actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /export/home/demo/myproj/src/test/resources [INFO]  [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ myproj --- [INFO] Nothing to compile - all classes are up to date [INFO]  [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ myproj --- [INFO] Surefire report directory: /export/home/demo/myproj/target/surefire-reports  -------------------------------------------------------  T E S T S ------------------------------------------------------- Running com.myproj.app.AppTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec  Results :  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0  [INFO]  [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ myproj --- Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/2.5/maven-archiver-2.5.pom Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/2.5/maven-archiver-2.5.pom (5 KB at 86.9 KB/sec) ... [INFO] Building jar: /export/home/demo/myproj/target/myproj-1.0-SNAPSHOT.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.653 s [INFO] Finished at: 2015-09-09T22:41:07-07:00 [INFO] Final Memory: 13M/141M [INFO] ------------------------------------------------------------------------

     

    If we take a look at the target directory, we will now see a new JAR file:

     

    demo@solaris:~/myproj$ ls target/
    classes                  myproj-1.0-SNAPSHOT.jar maven-archiver           surefire-reports maven-status             test-classes

     

    We can go ahead and run our program from this JAR file, as follows:

     

    demo@solaris:~/myproj$ java -cp target/myproj-1.0-SNAPSHOT.jar com.myproj.app.App
    Hello World!

     

    To clean up our workspace to the original state, we run the clean lifecycle:

     

    demo@solaris:~/myproj$ mvn clean

     

    Integrating the Maven Project into Jenkins

     

    We will need to configure Jenkins to know about our installed versions of the JDK and Maven (JAVA_HOME and MAVEN_HOME). To do this, navigate to the Manage Jenkins screen and find the sections for JDK and Maven.

     

    For the JDK configuration, we will use our JDK 8 location, /usr/jdk/jdk1.8.0_60, as shown in the Figure 2.

     

    f2.png

    Figure 2. JDK settings in Jenkins.

     

    For the Maven configuration, we will use /opt/apache-maven-3.3.3, as shown in the Figure 3.

     

    f3.png

    Figure 3. Maven settings in Jenkins.

     

    Installing the Git Plugin for Jenkins

     

    Now that we have configured JDK and Maven in Jenkins, we need to integrate our Maven project into Jenkins so that whenever a change is pushed back to our Maven project, we will do a Jenkins run. To achieve this, we need to install the Git plugin for Jenkins. This can be done in the Manage Jenkins > Manage Plugins page on the Jenkins dashboard. For convenience, you can search for this plugin by clicking the Available tab and using the filter. Jenkins will automatically install any dependent plugins that it requires, and you'll typically see the screen shown in Figure 4 once it has finished:

     

    f4.png

    Figure 4. Installing the Git plugin for Jenkins.

     

    To restart Jenkins, we'll issue the following command:

     

    jenkins@solaris:~$ sudo svcadm restart jenkins

     

    Building the Maven Project Using Jenkins

     

    To get Jenkins to build our project, we need to add a new entry for it in the Jenkins dashboard. To do this, we will use the New Item menu item. From there, we can choose a Maven project and give it a name, as shown in Figure 5.

     

    f5.png

    Figure 5. Creating a new Maven project in Jenkins.

     

    In the screen shown in Figure 6, we get to configure further details about the project. The most important part is to provide a location to our Git source code repository that Jenkins can use to check out the source into a workspace and build it. For simplicity, we are using a file-based repository in this example, but this could instead be a repository on another system or a repository hosted on a site such as GitHub.

    f6.png

    Figure 6. Configuring the location of the Git repository in Jenkins.

     

    Now that we have configured our Maven project, we can save this configuration and kick off a build using the Build Now menu item. This will trigger Jenkins to clone the Git repository and run through a build of our project. We can see status of this build in the left navigation pane shown in Figure 7:

     

    f7.png

    Figure 7. Build status of our Maven project.

     

    We can click through to see the overall status of this build, including the console output, by choosing the build number from the Build History. In Figure 8, we can see that the build succeeded. Jenkins compiled the project and also created a JAR file.

     

    f8.png

    Figure 8. Build status screen in Jenkins.

     

    We can easily kick off any number of builds on a manual basis using this process. In all cases, Jenkins will pull down the latest code from the Git repository into a local workspace and build it.

     

    Performing Automatic Builds with Git and Jenkins

     

    What we really want to achieve is to automatically kick off new Jenkins builds upon each commit to the Git repository. For this, we need to add a post-commit hook in Git. This is a simple process of adding a post-commit script within the .git directory of our Git workspace, as follows:

     

    demo@solaris:~/myproj/.git/hooks$ cat post-commit 
    #!/usr/bin/bash  curl http://myhost:8080/Git/notifyCommit?url=file:///export/home/demo/myproj demo@solaris:~/myproj/.git/hooks$ chmod +x post-commit

     

    You will notice the script contains a simple curl command to ping the Jenkins server and provide the server the location of our Git repository. Jenkins will search its list of projects based on this Git repository and will run a build if it finds a match.

     

    One important change that we need to configure in Jenkins is to accommodate the post-commit hook. We need to configure our project to accept Source Code Management (SCM) polling by selecting the Poll SCM option, as shown in Figure 9. We don't really want Jenkins to actually poll, so we will not provide it with a schedule; however, Jenkins still requires the Poll SCM option to be selected in order to process our notification from the Git repository.

     

    f9.png

    Figure 9. Configuring Jenkins to accept a post-commit notification from Git.

     

    Once we have made this change, we can easily test it by making a change to our Java file and committing the file. Let's change our "Hello World" string to something different:

     

    demo@solaris:~/myproj/src/main/java/com/myproj/app$ cat App.java
    package com.myproj.app;  /**  * Hello world!  *  */ public class App  {     public static void main( String[] args )     {         System.out.println( "Hello Brave New World" );     } }

     

    Let's check the Git status to see if there are changes to be made, and then push these changes:

     

    demo@solaris:~/myproj/src/main/java/com/myproj/app$ git status
    # On branch master # Changes not staged for commit: #   (use "git add <file>..." to update what will be committed) #   (use "git checkout -- <file>..." to discard changes in working directory) # #       modified:   App.java # no changes added to commit (use "git add" and/or "git commit -a") demo@solaris:~/myproj/src/main/java/com/myproj/app$ git commit -a -m "New Hello"
    Scheduled polling of MyProject [master a659653] New change  1 file changed, 1 insertion(+), 1 deletion(-)

     

    We can quickly return to our Jenkins dashboard to see that a new build has been scheduled.

     

    f10.png

    Figure 10. A new build (build #3) is scheduled in Jenkins.

     

    Creating IPS Packages with Maven

     

    As we go through the process of checking new code into the Git repository and building it using Jenkins, it would be nice to also be able to build a new IPS package each time. To create a new IPS package from a Maven project, we can use a Maven IPS plugin. This plugin allows us to create an IPS package manifest based on the code we have compiled and publish the package to either a local or remote IPS repository.

     

    The Maven IPS plugin is currently hosted in a Mercurial repository on java.net. We first need to install the Mercurial package, which is another distributed source code manager:

     

    root@solaris:~# pkg install mercurial
               Packages to install:  3        Create boot environment: No Create backup boot environment: No  DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED Completed                                3/3     1139/1139      6.7/6.7  887k/s  PHASE                                          ITEMS Installing new actions                     1274/1274 Updating package state database                 Done  Updating package cache                           0/0  Updating image state                            Done  Creating fast lookup database                   Done  Updating package cache                           1/1

     

    Once we have done this, we can clone the Maven IPS plugin workspace:

     

    demo@solaris:~$ hg clone https://hg.java.net/hg/ips~mvn-plugin-repo
    destination directory: ips~mvn-plugin-repo requesting all changes adding changesets adding manifests adding file changes added 2 changesets with 33 changes to 27 files updating to branch default 27 files updated, 0 files merged, 0 files removed, 0 files unresolved

     

    We need to install this plugin so we can use it in our project. We use mvn install to achieve this. We first need to navigate to where the pom.xml file is located:

     

    demo@solaris:~$ cd ips~mvn-plugin-repo/src/util/ips-maven-plugin/
    demo@solaris:~/ips~mvn-plugin-repo/src/util/ips-maven-plugin$ mvn install

     

    Next, we need to create a new IPS repository that will be the location where we want to publish the packages that we'll create using the IPS Maven plugin. We can do this using the pkgrepo utility.

     

    demo@solaris:~$ pkgrepo create myproj-repository
    demo@solaris:~$ pkgrepo -s myproj-repository set publisher/prefix=myproj

     

    We need to make some edits to our pom.xml file to include details about how to create our package. Fortunately we can automate this by calling the com.oracle.ips:ips-maven-plugin:autoconf goal, as follows:

     

    demo@solaris:~/myproj$ mvn com.oracle.ips:ips-maven-plugin:autoconf
    [INFO] Scanning for projects... Downloading: https://repo.maven.apache.org/maven2/com/oracle/ips/ips-maven-plugin /maven-metadata.xml [INFO]                                                                          [INFO] ------------------------------------------------------------------------ [INFO] Building myproj 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO]  [INFO] --- ips-maven-plugin:1.0-SNAPSHOT:autoconf (default-cli) @ myproj --- [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.071 s [INFO] Finished at: 2015-09-10T18:51:11-07:00 [INFO] Final Memory: 10M/137M [INFO] ------------------------------------------------------------------------

     

    This goal will automatically add some content to our pom.xml file. Let's take a look at the file, which is shown in Listing 1. We can see that a new section has been added to define a build target:

     

    <build>     <plugins>       <plugin>         <groupId> com.oracle.ips</groupId>         <artifactId>ips-maven-plugin</artifactId>         <version>1.0-SNAPSHOT</version> <configuration>           <projectRoot>${basedir}</projectRoot>           <pkgName>PKG_NAME</pkgName>           <publisher>PUBLISHER_NAME</publisher>           <version>1.0</version>           <projectSummary>Project summary text goes here!</projectSummary>           <projectDescription>Project description text goes here!</projectDescription>         </configuration>         <executions>           <execution>             <id>ips-packager</id>             <configuration>               <mappings>                 <mapping>                   <directory>/usr/local/java_apps/bin</directory>                   <filemode>0755</filemode>                   <username>root</username>                   <groupname>bin</groupname>                   <sources>                     <source>                       <location>${project.build.directory}/myproj-1.0-SNAPSHOT.jar</location>                     </source>                   </sources>                 </mapping>                 <mapping>                   <directory>/usr/local/java_apps/lib</directory>                   <filemode>0755</filemode>                   <username>root</username>                   <groupname>bin</groupname>                   <dep>                     <includeProjectDep>true</includeProjectDep>                   </dep>                 </mapping>               </mappings>             </configuration>             <phase>package</phase>             <goals>               <goal>packager</goal>             </goals>           </execution>         </executions>       </plugin>     </plugins>   </build>

     

    Listing 1. Updated pom.xml file

     

    Change the pom.xml file to change the package name and publisher. To do this, locate the <pkgName> and <publisher> declarations and change them as follows:

     

    <pkgName>myproj</pkgName> <publisher>myproj</publisher>

     

    Now we can run mvn package and see what happens:

     

    demo@solaris:~/myproj$ mvn package
    [INFO] Scanning for projects... [INFO]                                                                          [INFO] ------------------------------------------------------------------------ [INFO] Building myproj 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO]  [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ myproj --- [WARNING] Using platform encoding (ISO646-US actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /export/home/demo/myproj/src/main/resources [INFO]  [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ myproj --- [INFO] Nothing to compile - all classes are up to date [INFO]  [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ myproj --- [WARNING] Using platform encoding (ISO646-US actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /export/home/demo/myproj/src/test/resources [INFO]  [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ myproj --- [INFO] Nothing to compile - all classes are up to date [INFO]  [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ myproj --- [INFO] Surefire report directory: /export/home/demo/myproj/target/surefire-reports  -------------------------------------------------------  T E S T S ------------------------------------------------------- Running com.myproj.app.AppTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec  Results :  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0  [INFO]  [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ myproj --- [INFO] Building jar: /export/home/demo/myproj/target/myproj-1.0-SNAPSHOT.jar [INFO]  [INFO] --- ips-maven-plugin:1.0-SNAPSHOT:packager (ips-packager) @ myproj --- [INFO] IPS Maven Plugin Packager [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.652 s [INFO] Finished at: 2015-09-10T20:33:45-07:00 [INFO] Final Memory: 12M/179M [INFO] ------------------------------------------------------------------------

     

    If we check our workspace, we'll see that two things have been created: an IPS package manifest and an IPS proto area that is used to determine what will be installed into the package. Let's look at the contents:

     

    demo@solaris:~/myproj$ cat ips_manifest.p5m
    set name=pkg.fmri value=pkg://myproj/myproj@1.0,5.11-0 set name=pkg.summary value="myproj" set name=pkg.description value="My Project" set name=variant.arch value=sparc file usr/local/java_apps/lib/junit-3.8.1.jar path=usr/local/java_apps/lib/junit-3.8.1.jar owner=root group=bin mode=0755 file usr/local/java_apps/bin/myproj-1.0-SNAPSHOT.jar path=usr/local/java_apps/bin/myproj-1.0-SNAPSHOT.jar owner=root group=bin mode=0755

     

    At some point, we might want to ensure that our Git commit SHA-1 hash sum (used for each commit we make of our source code) is integrated into our package. This will mean that we can reliably trace a package back to an individual commit. We can use the Maven Build Number plugin to achieve this.

     

    We could take this IPS package manifest and our proto area and manually publish this to our IPS repository ourselves. However, we can also include another part in our pom.xml file to automatically publish our package when we have finished creating the manifest. To do that, we can add the bold code shown in Listing 2:

     

    <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">   <modelVersion>4.0.0</modelVersion>   <groupId>com.myproj.app</groupId>   <artifactId>myproj</artifactId>   <packaging>jar</packaging>   <version>1.0-SNAPSHOT</version>   <name>myproj</name>   <description>My Project</description>   <url>http://maven.apache.org</url>   <dependencies>     <dependency>       <groupId>junit</groupId>       <artifactId>junit</artifactId>       <version>3.8.1</version>       <scope>test</scope>     </dependency>   </dependencies>   <build>     <plugins>       <plugin>         <groupId>com.oracle.ips</groupId>         <artifactId>ips-maven-plugin</artifactId>         <version>1.0-SNAPSHOT</version>         <configuration>           <projectRoot>${basedir}</projectRoot>           <pkgName>myproj</pkgName>           <publisher>myproj</publisher>           <version>1.0</version>           <projectSummary>${project.name}</projectSummary>           <projectDescription>${project.description}</projectDescription>           <localRepoPath>/export/home/demo/myproj-repository/</localRepoPath>
            </configuration>         <executions>           <execution>             <id>ips-packager</id>             <configuration>               <mappings>                 <mapping>                   <directory>/usr/local/java_apps/bin</directory>                   <filemode>0755</filemode>                   <username>root</username>                   <groupname>bin</groupname>                   <sources>                     <source>                       <<location>${project.build.directory}/myproj-1.0-SNAPSHOT.jar</location>                     </source>                   </sources>                 </mapping>                 <mapping>                   <directory>/usr/local/java_apps/lib</directory>                   <filemode>0755</filemode>                   <username>root</username>                   <groupname>bin</groupname>                   <dep>                     <includeProjectDep>true</includeProjectDep>                   </dep>                 </mapping>               </mappings>             </configuration>             <phase>package</phase>             <goals>               <goal>packager</goal>             </goals>           </execution>           <execution>
                <id>ips-install</id>
                <phase>install</phase>
                <goals>
                  <goal>installer</goal>
                </goals>
              </execution>
            </executions>       </plugin>     </plugins>   </build> </project>

     

    Listing 2. Updating pom.xml so it will automatically publish our package.

     

    To get this code to execute, we will need to call the ips_system:ips-maven-plugin:installer goal of the Maven IPS plugin. The reason we have this extra goal after we package our application is that we might not always be running on an Oracle Solaris platform. We can generate the IPS manifest and proto area on any platform, but we must publish on an Oracle Solaris platform.

     

    demo@solaris:~/myproj$ mvn com.oracle.ips:ips-maven-plugin:installer
    [INFO] Scanning for projects... [INFO]                                                                          [INFO] ------------------------------------------------------------------------ [INFO] Building myproj 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO]  [INFO] --- ips-maven-plugin:1.0-SNAPSHOT:installer (default-cli) @ myproj --- [INFO] IPS Maven Plugin Installer [INFO] Running as demo. [INFO] pkg://myproj/myproj@1.0,5.11-0:20150929T020609Z PUBLISHED  [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.289 s [INFO] Finished at: 2015-09-10T21:52:18-07:00 [INFO] Final Memory: 8M/109M [INFO] ------------------------------------------------------------------------

     

    If we check our repository, we can see that our package has been newly published:

     

    demo@solaris:~$ pkgrepo -s /export/home/demo/myproj-repository/ info
    PUBLISHER PACKAGES STATUS           UPDATED myproj    1        online           2015-09-29T02:06:09.624351Z

     

    If we ran this goal several times we would see that the repository contains multiple packages of the same version, but with a different time stamp, as follows:

     

    demo@solaris:~$ pkgrepo -s /export/home/demo/myproj-repository/ list
    PUBLISHER NAME                                  O VERSION myproj    myproj                                  1.0,5.11-0:20150929T020723Z myproj    myproj                                  1.0,5.11-0:20150929T020718Z myproj    myproj                                  1.0,5.11-0:20150929T020609Z

     

    And this is the basis for integration into Jenkins.

     

    Performing Automatic IPS Publication Using Jenkins

     

    We first need to ensure that the Maven IPS plugin is installed in the Maven repository that is local to Jenkins. As before, we'll clone the Mercurial workspace and install it. This will install it to $HOME/.m2/repository.

     

    jenkins@solaris:~/ips~mvn-plugin-repo/src/util/ips-maven-plugin$ mvn install

     

    We will also set up an IPS repository to allow Jenkins to publish to it (as opposed to the one we created with the demo user). In this case, let's create a repository stored in a separate ZFS dataset at /repository, and make it available over HTTP.

     

    root@solaris:~# zfs create rpool/repository
    root@solaris:~# zfs set mountpoint=/repository rpool/repository
    root@solaris:~# pkgrepo create /repository
    root@solaris:~# pkgrepo -s /repository set publisher/prefix=myproj
    root@solaris:~# svccfg -s pkg/server setprop pkg/inst_root=/repository
    root@solaris:~# svccfg -s pkg/server setprop pkg/port=9001
    root@solaris:~# svccfg -s pkg/server setprop pkg/readonly=false
    root@solaris:~# svcadm enable pkg/server

     

    We can now open our web browser and navigate to http://solaris.oracle.com:9001 on this host to see that the IPS repository has been set up correctly.

     

    f11.png

    Figure 11. An IPS repository served up over HTTP.

     

    Having brought this IPS repository online, we can now change our pom.xml file in our myproj Git repository to publish to this new repository instead. We will modify the configuration to replace the existing install goal that publishes to a local repository with a new goal that publishes to a remote repository.

     

    <remoteRepoPath>http://solaris.oracle.com:9001</remoteRepoPath> ... <execution>   <id>ips-deployer</id>   <phase>deploy</phase>   <goals>     <goal>deployer</goal>   </goals> </execution>

     

    As you can see from the XML code above, we have a new goal called deployer. We can commit this change to our Git repository so it is available upon the next checkout by Jenkins.

     

    Let's now link this as a post-build goal to invoke when a build succeeds, as shown in Figure 12.

     

    f12.png

    Figure 12. Invoking the deployer goal with a post-build Maven goal.

     

    When we run a new build of our project, we can see that a package has been published to this new repository, as shown in Figure 13.

     

    f13.png

    Figure 13. A new package has been published to the IPS repository served over HTTP.

     

    Performing Automatic Installation of Packages Using Puppet

     

    After a new build has been run by Jenkins and a new IPS package has been published to our package repository, the next step is to ensure that the package is automatically installed. For this task, we will use Puppet. Puppet is a popular open source configuration management solution that not only applies configuration to a system, but also enforces it, helping to improve the overall automation of provisioning and reduce the unexpected cost of human error.

     

    Let's start by installing the Puppet IPS package. Puppet can typically use a master/agent architecture, where one or more master nodes are responsible for managing any number of agent nodes. A single IPS package includes both the master and agent services.

     

    root@solaris:~# pkg install puppet
               Packages to install:  7            Mediators to change:  1             Services to change:  2        Create boot environment: No Create backup boot environment: No  DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED Completed                                7/7   16576/16576    13.1/13.1 87.8k/s  PHASE                                          ITEMS Installing new actions                   16752/16752 Updating package state database                 Done  Updating package cache                           0/0  Updating image state                            Done  Creating fast lookup database                   Done  Updating package cache                           1/1   root@solaris:~# svccfg -s puppet:master setprop config/server=solaris.oracle.com
    root@solaris:~# svccfg -s puppet:master refresh
    root@solaris:~# svcadm enable puppet:master
    STATE          STIME    FMRI online         13:42:56 svc:/application/puppet:master

     

    Once we have our Puppet master online, we can write a simple Puppet manifest to ensure that our IPS package repository is configured and the latest myproj package is installed. We do this through the site manifest, /etc/puppet/manifest/site.pp, on the master node:

     

    pkg_publisher { 'myproj':   ensure      => 'present',   enable      => 'true',   origin      => ['http://host.oracle.com:9001'], }  package { 'myproj':   ensure     => 'latest',   require    => Pkg_publisher['myproj'], }

     

    Once we have this in place, we can check to see that it's a valid manifest by applying it locally using --noop, which ensures that we do not make any changes to the system.

     

    root@solaris:~ # puppet apply -v --noop /etc/puppet/manifest/site.pp
    Notice: Compiled catalog for solaris.us.oracle.com in environment production in 0.36 seconds Info: Applying configuration version '1442004420' Notice: /Stage[main]/Main/Package[myproj]/ensure: current_value absent, should be latest (noop) Notice: /Stage[main]/Main/Pkg_publisher[myproj]/ensure: current_value absent, should be present (noop) Notice: Class[Main]: Would have triggered 'refresh' from 2 events Notice: Stage[main]: Would have triggered 'refresh' from 1 events Info: Creating state file /var/lib/puppet/state/state.yaml Notice: Finished catalog run in 2.36 seconds

     

    From the output above, we can see that the Puppet run successfully detected that the IPS publisher for our project isn't currently configured, nor is the myproj IPS package installed.

     

    Let's set up our Puppet agent on another system, as follows:

     

    root@node:~# svccfg -s puppet:agent setprop config/server=solaris.oracle.com
    root@node:~# svccfg -s puppet:agent refresh
    root@node:~# svcadm enable puppet:agent
    root@node:~# svcs puppet:agent
    STATE          STIME    FMRI online         15:18:18 svc:/application/puppet:agent

     

    We need to test the connection from the Puppet agent to the master. The following command creates a certificate on the agent and registers it with the master:

     

    root@node:~# puppet agent --test
    Info: csr_attributes file loading from /etc/puppet/csr_attributes.yaml Info: Creating a new SSL certificate request for node.oracle.com Info: Certificate Request fingerprint (SHA256): 82:CC:07:A8:4B:31:26:23:C1:F2:25:E6:C1:96:A4:AA:18:C0:27:3C:C0:08:D3:CF:F3:6A:BB:F2:AB:4F:01:D5 Exiting; no certificate found and waitforcert is disabled

     

    On the Puppet master, we need to accept this agent to our network. First, let's list the pending certificate requests:

     

    root@solaris:~# puppet cert list
      "node.oracle.com" (SHA256) 82:CC:07:A8:4B:31:26:23:C1:F2:25:E6:C1:96:A4:AA:18:C0:27:3C:C0:08:D3:CF:F3:6A:BB:F2:AB:4F:01:D5

     

    Now we can now accept this request:

     

    root@solaris:~# puppet cert sign node.oracle.com
    Notice: Signed certificate request for node.oracle.com Notice: Removing file Puppet::SSL::CertificateRequest node.oracle.com at '/etc/puppet/ssl/ca/requests/node.oracle.com.pem'

     

    Puppet has a default time interval between calls between the Puppet agent and the master. After approximately 30 minutes, we'll see that the system node.oracle.com has been updated:

     

    root@node:~# pkg publisher
    PUBLISHER                   TYPE     STATUS P LOCATION solaris                     origin   online F http://pkg.oracle.com/solaris/release/ myproj                      origin   online F http://solaris.oracle.com:9001/ root@node:~# pkg list myproj
    myproj (myproj)                                   5.11-0                     i-- root@node:~# pkg info myproj
              Name: myproj        Summary: Project summary text goes here!    Description: Project description text goes here!          State: Installed      Publisher: myproj        Version: 1.0  Build Release: 5.11         Branch: 0 Packaging Date: Tue Sep 29 02:32:43 2015           Size: 120.93 kB           FMRI: pkg://myproj/myproj@1.0,5.11-0:20150929T023243Z

     

    Note: At the time of writing, there was a bug in the IPS Puppet provider for the package resource type. See the Appendix of this article for a patch that needs to be applied in order for Puppet to successfully update the package. A fix for this is planned to go into a future Support Repository Update (SRU).

     

    We have finally reached our end goal. Whenever a change is checked in to our Git repository, the repository will notify our Jenkins build server. Jenkins will start a build using Maven, and a package will be created using the IPS Maven plugin and published to an IPS repository available over HTTP. Our node server will contact the Puppet master every 30 minutes and check to see whether any updates are available. If updates are available, it will apply them locally.

     

    If we check our repository for a list of the packages it contains, we can see two packages:

     

    root@solaris:~# pkgrepo -s http://solaris.oracle.com:9001 list
    PUBLISHER NAME                                          O VERSION myproj    myproj                                          1.0,5.11-0:20150929T025628Z myproj    myproj                                          1.0,5.11-0:20150929T023243Z

     

    The top package has a newer date stamp. We can check our node server and verify that the latest package version has been installed.

     

    root@node:~# pkg list -nv myproj
    FMRI                                                       INFO pkg://myproj/myproj@1.0,5.11-0:20150929T025628Z            i--

     

    One additional step we can take is to integrate this Puppet manifest fragment into our Git repository so that we can dynamically change the final application environment as part of our commits.

     

    Summary

     

    As organizations look for faster go-to-market strategies for their businesses, many are turning toward a new trend that is merging the traditional development, quality assurance and operations roles to provide a more agile approach with incremental continuous delivery, commonly known as DevOps.

     

    In this article, we looked at a very simple application deployment pipeline using a combination of Git, Maven, Jenkins, and Puppet on Oracle Solaris 11. Through minimal integration with the Oracle Solaris Service Management Facility (SMF) and Image Packaging System (IPS), we ensured highly reliable application delivery from development through production.

     

    See Also

     

     

    Also see the additional resources:

     

     

    Appendix: IPS Puppet Patch

     

    The following patch that needs to be applied in order for Puppet to successfully update a package:

     

    --- /usr/ruby/1.9/lib/ruby/vendor_ruby/1.9.1/puppet/provider/package/ pkg.rb     2015-09-29 16:40:45.000000000 +1300 +++ /usr/ruby/1.9/lib/ruby/vendor_ruby/1.9.1/puppet/provider/package/ pkg-new.rb     2015-09-29 14:22:19.000000000 +1300 @@ -126,15 +126,11 @@    # http://defect.opensolaris.org/bz/show_bug.cgi?id=19159%    # notes that we can't use -Ha for the same even though the manual page reads that way.    def latest -    lst = pkg(:list, "-Hn", @resource[:name]).split("\n").map{|l|self.class.parse_line(l)}   -    # Now we know there is a newer version. But is that installable? (i.e are there any constraints?) -    # return the first known we find. The only way that is currently available is to do a dry run of -    # pkg update and see if could get installed (`pkg update -n res`). -    known = lst.find {|p| p[:status] == 'known' } -    return known[:ensure] if known and exec_cmd(command(:pkg), 'update', '-n', @resource[:name])[:exit].zero? +    return exec_cmd(command(:pkg), 'update', '-n', @resource[:name])[:exit].zero?        # If not, then return the installed, else nil +    lst = pkg(:list, "-Hi", @resource[:name]).split("\n").map{|l|self.class.parse_line(l)}      (lst.find {|p| p[:status] == 'installed' } || {})[:ensure]    end

     

    About the Author

     

    Glynn Foster is a principal product manager for Oracle Solaris. He is responsible for a number of technology areas including OpenStack, the Oracle Solaris Image Packaging System, installation, and configuration management.

     

     

    Revision 1.0, 10/01/2015

     

    Follow us:
    Blog | Facebook | Twitter | YouTube