How to Apply Read-Only Protection with Oracle Solaris Immutable Zones

Version 3

    by Glynn Foster

     

    Learn how to configure an Oracle Solaris 11.3 system to lock down the host and virtual environments with read-only protection using Oracle Solaris Immutable Zones.

     

    Introduction

     

    Security and compliance are, undoubtedly, the top concerns of organizations today. Data breaches are commonplace, with sensitive data getting into the hands of unknown groups causing unrecoverable damage, such as the loss of customer confidence, the high costs of remediation, and credit rating downgrades. Attacks can come through a variety of channels—from denial of service to SQL injection, stolen user credentials, social engineering, and more. As organizations look to move towards increased business agility through cloud computing, these concerns increase as boundaries shift towards a multitenant environment using private, public, or hybrid clouds.

     

    Oracle Solaris 11 offers a variety of proven security features, which when used in a "defense-in-depth" architecture, provide a sophisticated network-wide security system that controls the way users access files, protect system databases, and use system resources. Oracle Solaris 11 addresses security requirements at every layer. It is installed "secure by default" as a minimal-protection profile upon which you can add additional protection. This helps to reduce the chance of intrusion by disabling all network services other than Secure Shell (SSH). It also records login attempts and minimizes system access to services using a "least privilege" model. You can read more about Oracle Solaris security features in the Oracle Solaris 11 Security and Hardening Guidelines.

     

    Enforcing change control across systems is an important strategy used by organizations to reduce the cost of human error that might cause a significant security breach or unplanned system downtime. Oracle Solaris provides integrated compliance management and reporting tools to meet compliance obligations but also, more importantly, to help maintain change control through simple instructions for mitigating any compliance failures.

     

    However, there's another option that can be used—Oracle Solaris Immutable Zones—that can be a forcing function for secure change control that ultimately also benefits compliance requirements.

     

    Oracle Solaris Immutable Zones

     

    Using immutable zones is one technique that can protect applications by applying read-only protection to the host global zone and guest non-global zones. Oracle Solaris Zones technology is the recommended approach for deploying application workloads in an isolated environment—no processes in one zone can monitor or affect processes running in another zone. Immutable zones extend this level of isolation and protection by enabling a read-only root file system, preventing any modification to the system or system configuration. The Oracle Solaris kernel uses Mandatory Write Access Control (MWAC) to enforce file system write privileges in a flexible, policy-driven way.

     

    A simple example that employs immutable zones might involve locking down a zone running a web server so it is read-only. A private database network connects this zone to another zone running a database with full read-write access to the database itself. Figure 1 shows this example architecture.

     

    f1.png

    Figure 1: A typical architecture that leverages immutable zones

     

    We can apply immutability to either the global zone or non-global zone using similar mechanisms. Both approaches take advantage of a zone configuration property, file-mac-profile, which provides exemptions for what parts of the file system can be written to.  There are currently five supported values that can be used:

     

                

    Value of file-mac-profile PropertyDescription
    noneAllows full read-write access. This is the default behavior for newly created zones if you didn't use the file-mac-profile property.
    flexible-configurationSame as fixed-configuration (or dynamic-zones, in the global zone case) but also allows read-write access on files located in /etc.
    fixed-configurationAllows read-write access to files located in /var except for the following locations:

     

    /var/ld
    /var/lib/postrun
    /var/pkg
    /var/spool/cron
    /var/spool/postrun
    /var/svc/manifest
    /var/svc/profiles
    dynamic-zonesSame as fixed-configuration but allows for the creation of non-global or kernel zones. Applies only to the global zone (or the global zone of a kernel zone).
    strictProvides a full read-only file system. System logs and audit trails need to be sent off to another centralized system.

     

    Creating an Immutable Non-Global Zone

     

    Creating an immutable non-global zone is easy. For example, let's create a new zone called myzone and set file-mac-profile to fixed-configuration, as follows:

     

    # zonecfg -z myzone

    Use 'create' to begin configuring a new zone.

    zonecfg:myzone> create

    create: Using system default template 'SYSdefault'

    zonecfg:myzone> set file-mac-profile=fixed-configuration

    zonecfg:myzone> verify

    zonecfg:myzone> commit

    zonecfg:myzone> exit

     

    We can now go ahead and install this zone, as usual:

     

    # zoneadm -z myzone install

     

    Once this has finished, let's boot the zone and log in to the console:

     

    # zoneadm -z myzone boot

    # zlogin -C myzone

     

    Because we did not provide a system configuration profile during the zone installation, we will use the System Configuration Tool to enter into a set of interactive screens where we can specify the configuration for the non-global zone host name, networking, the locale, and the initial users for this zone. This profile will then be applied to the zone. However, compared to a regular zone installation, we will see something like the following:

     

    Exiting System Configuration Tool. Log is available at:

    /system/volatile/sysconfig/sysconfig.log.6477

    Hostname: myzone

     

    [NOTICE: This read-only system transiently booted read/write]

    [NOTICE: Now that self assembly has been completed, the system is rebooting]

     

    [NOTICE: Zone rebooting]

     

    SunOS Release 5.11 Version 11.3 64-bit

    Copyright (c) 1983, 2015, Oracle and/or its affiliates. All rights reserved.

     

    Hostname: myzone

     

    myzone console login:

     

    If we try to install a package, the operation will also fail, as follows:

     

    root@myzone:~# pkg install wireshark

    pkg install: Could not complete the operation on /var/pkg/lock: read-only filesystem.

     

    And we can see that various other parts of the file system are also read-only:

     

    root@myzone:~# touch /etc/foobar

    touch: cannot create /etc/foobar: Read-only file system

    root@myzone:~# touch /var/foobar

    touch: cannot create /var/foobar: Read-only file system

    root@myzone:~# touch /usr/bin/foobar

    touch: cannot create /usr/bin/foobar: Read-only file system

     

    When we use the fixed-configuration profile, we allow the system to be able to store logs relating to the system's services, which means that the system can collect and store service logs in /var.

     

    We can see that this zone is using the fixed-configuration profile from the global zone by using the -p argument to zoneadm list:

     

    # zoneadm list -p

    0:global:running:/::solaris:shared:-:none:

    2:myzone:running:/system/zones/myzone:dfd65825-ed4b-46bb-a8f6-f2043f5282cd:solaris:excl:R:fixed-configuration:

     

    But what happens when we want to make a change in the configuration or apply a critical security patch? There are two approaches that we can take. We can simply reboot the zone into read-write mode using the following:

     

    # zoneadm -z myzone reboot -w

     

    The -w argument causes the zone to boot into read-write mode, overriding the file-mac-profile property. However, this is in effect only until the next zone reboot. However, this does open up a window of opportunity for security issues if you make the entire zone read-write for a given amount of time while you make your changes.

     

    The better, alternative way is to use a mechanism that is called the Trusted Path domain. The Trusted Path provides a way to modify protected files without having to open up the entire environment—but it provides this only to privileged processes.

     

    To log in as the Trusted Path, we use the -T argument to zlogin:

     

    # zlogin -T myzone

    [Connected to zone 'myzone' pts/2]

    Oracle Corporation      SunOS 5.11      11.3    June 2015

    root@myzone:~# touch /foobar

    root@myzone:~# ls /foobar

    /foobar

    root@myzone:~# pkg install wireshark

               Packages to install:  1

                Services to change:  1

           Create boot environment: No

    Create backup boot environment: No

     

    DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED

    Completed                                1/1           8/8      1.3/1.3  1.1M/s

     

    PHASE                                          ITEMS

    Installing new actions                         28/28

    Updating package state database                 Done

    Updating package cache                           0/0

    Updating image state                            Done

    Creating fast lookup database                   Done

    Updating package cache                           1/1

     

    And to confirm that read-write mode is not set outside the Trusted Path, let's exit out of the zone and log in to the console as we would normally:

     

    root@myzone:~# ~.

    [Connection to zone 'myzone' pts/2 closed]

    # zlogin -C myzone

    [Connected to zone 'myzone' console]

     

    myzone console login: root

    Password:

    Last login: Thu Aug  6 07:02:32 2015 on pts/2

    Aug  6 07:04:49 myzone login: ROOT LOGIN /dev/console

    Oracle Corporation      SunOS 5.11      11.3    June 2015

    root@myzone:~# touch /foobar

    touch: cannot change times on /foobar: Read-only file system

     

    We will talk more about the Trusted Path later in this article.

     

    Creating an Immutable Global Zone

     

    Creating an immutable global zone uses a similar process.

     

    First, we create the zone, as follows:

     

    # zonecfg -z global

     

    You will notice in the command above that we used a zone name of global. Once we have entered the interactive zone configuration interface, we can set the file-mac-profile property:

     

    zonecfg:global> set file-mac-profile=dynamic-zones

    zonecfg:global> commit

    updating /platform/i86pc/amd64/boot_archive

    zonecfg:global> exit

     

    In this case, we have chosen to set it to dynamic-zones to allow us to be able to create and destroy non-global and kernel zones, as required, but keep the rest of the system configuration as read-only.

     

    This profile is particularly important in the context of a cloud environment, because we can lock down management of the system, yet still allow tenants of the system to be able to create virtual machines (VMs) in a self-service manner. This profile was added explicitly for this use case. We can use this profile in conjunction with OpenStack, the popular open source infrastructure as a service (IaaS) cloud solution. Within OpenStack, the Nova service is responsible for creating VMs across physical compute hosts. By using dynamic-zones, we can maintain a strict change control strategy on our cloud infrastructure (commonly referred to as "under-cloud") that is read-only, but still allow the creation of VM environments.

     

    Once we have set the policy, we need to reboot the system to apply the read-only protection profile.

     

    The Trusted Path

     

    Performing a pkg update for a global zone must be done from the Trusted Path.  When updating the global zone, updates of non-global zones are performed on a writeable clone boot environment (for the zone) that is not mounted inside the non-global zone.

     

    For immutable global zones, we need to access the system console using Oracle Integrated Lights Out Manager (Oracle ILOM). Once we log in through the Oracle ILOM console, we can use the break sequence to enter the Trusted Path:

     

    trusted path console login:

     

    The Trusted Path is specifically designed to provide a protected way to access the system for maintenance. The tpdlogin(1M) utility is automatically invoked by the kernel when a break sequence is used, and it uses a separate PAM service: tpdlogin.

     

    In Oracle Solaris 11, certain processes can be marked as part of the Trusted Path Domain (TPD). From the perspective of these processes, they are allowed to perform all restricted options—essentially, they see the environment as a standard read/write environment. In order to prevent non-TPD processes from interfering with TPD processes, TPD processes can not be trussed by non-TPD processes. TPD-processes terminals and FIFOs are marked specifically, and they cannot be opened by non-TPD processes.

     

    Note that executing the reboot -- -w command while the system is in read-only mode will not work. The command must be executed through the Trusted Path. To apply updates to a system, simply executing pkg update through the Trusted Path will work. The first boot after an update of an immutable global zone will cause the system to transiently enter a read-write mode. This is required to perform any self-assembly of configuration based on the update (essentially to build a working configuration from a collection of installed software). Once the system has finished this, it will enter into read-only mode once again.

     

    Additional ZFS Datasets and File Systems

     

    ZFS datasets can be manually whitelisted using the add dataset resource. These ZFS datasets have full read-write permissions and are not constrained by the MWAC policy.

     

    For example, to ensure that rpool/export has read-write permissions, we would do the following:

     

    # zonecfg -z myzone

    zonecfg:myzone> add dataset

    zonecfg:myzone:dataset> set name=rpool/export

    zonecfg:myzone:dataset> end

    zonecfg:myzone> commit

    zonecfg:myzone> exit

     

    We do not need to use this mechanism to add other datasets that might reside in a separate ZFS pool, because they are not constrained by the MWAC policy for the root pool. Any file systems added using the add fs resource are also not constrained by the MWAC policy, but they can be mounted as read-only.

     

    Summary

     

    Oracle Solaris Immutable Zones are a powerful way to apply a read-only protection policy and take further steps to minimize potential attacks. When used with other Oracle Solaris security features, such as ZFS encryption, a defense-in-depth strategy can be taken for identity management, data management, networking, and virtualization.

     

    Oracle Solaris Immutable Zones also provide the benefit of a more locked-down system, enabling further change control in the data center and protecting against unexpected events caused by human error. Combined with the integrated compliance and archiving features of Oracle Solaris, they provide an application lifecycle from development to production ensuring that what you've developed and tested has been delivered unmodified to production for greater reliability.

     

    See Also

     

     

    Also see the following resources:

     

     

    About the Author

     

    Glynn Foster is a principal product manager for Oracle Solaris. He is responsible for a number of technology areas including OpenStack, the Oracle Solaris Image Packaging System, installation, and configuration management.

     

     

    Revision 1.0, 08/20/2015

     

    Join the Conversation

     

    Join the Java community conversation on Facebook, Twitter, and the Oracle Java Blog!