by Kevin Klapak
Learn how to use the RESTful API for Oracle ZFS Storage Appliance systems to streamline your deployment of LUNs.
|
Introduction
As projects scale in size, creating a large number of LUNs using the browser-based user interface (BUI) of the products in the Oracle ZFS Storage Appliance family can be a slow and monotonous process. By using the RESTful API for Oracle ZFS Storage Appliance systems, you can streamline this process and remove the need to use the BUI entirely.
Oracle Compute Cloud Service – Dedicated Compute Capacity – SPARC Model 300 (hereafter called SPARC Model 300) uses Oracle ZFS Storage Appliance systems for its storage and was architected to use only iSCSI LUNs. Although the following practices can also apply to Fibre Channel and NFS, this article focuses on iSCSI to give examples that can be leveraged on SPARC Model 300 today.
| |
|
Oracle Optimized Solutions provide tested and proven best practices for how to run software products on Oracle systems. Learn more.
|
|
Table of Contents
Working with the RESTful API for Oracle ZFS Storage Appliance Systems
The Oracle white paper "Working with the RESTful API for the Oracle ZFS Storage Appliance" goes into great detail on the RESTful API architecture for Oracle ZFS Storage Appliance systems, as well as examples using shell and Python scripts. This paper is a great resource that I recommend you read if you plan to use the API.
The white paper has two Python modules, restclient.py and restmulty.py, that provide an example of a REST client and multithreaded requests.
Note: The examples in the white paper mislabeled restclient.py as restclientlib in the imports, so just be aware that when the paper references restclientlib it is talking about the restclient.py module.
The Oracle Optimized Solutions team used these modules to create our own Python command-line interface (CLI) script and library, which was then used to create an interactive CLI for interacting with Oracle ZFS Storage Appliance systems. There are no client libraries provided by Oracle, so you will need to create this framework and can use this article and the white paper as a reference.
Tool for Streamlining iSCSI Deployment
As mentioned before, Oracle leveraged the scripts restclient.py and restmulty.py to help build out its own Python library/CLI tool for Oracle ZFS Storage Appliance systems, zfssa.py. This library allows you to execute commands, directly from the command line, to manage the appliances. As an example, you can execute the following command:
python zfssa.py -u <username> -p <password> 10.0.1.0 project create-luns opc test 10 500G
Note: If you try to use zfssa.py, you will need to have restclient.py, restmulty.py, and interactiveCLI.py within the same directory as well Python 2.7 in the path.
First, the command accepts an optional username (-u <username>) and password (-p <password>) to connect to the one of the Oracle ZFS Storage Appliance heads at 10.0.1.0. Next, it uses the project API service and runs the create-luns command. The create-luns command will create ten 500 gigabyte LUNs within the opc project with names test1 through test10. If the command is successful, it returns a JSON object for each LUN created.
To do the same thing within the Oracle ZFS Storage Appliance BUI, you would need to manually create each LUN by entering the same parameters individually within the BUI ten times. Using the API can save a tremendous amount of time.
Below is the usage message showing all of the other functionality built into the tool.
usage: zfssa.py [options] <zfssa_host> <api-service> <command> [command-args]
options:
-h or --help Usage message
-i Load interactive CLI
-u <user> Login user. (default is opc)
-p <pass> Login password.
<api-service>:
pool:
<command>:
get-pools
get-pools-info
project:
<command>:
get-projects
get-project-details <project_name>
create-project [-f properties_file] <project_name>
modify-project -f properties_file <project_name>
get-luns <project_name>
create-luns <project-name> <lun-prefix> <amount> <volsize> [lun-props]
[lun-props]:
--amount: <int>
--volsize: <int>{K,M,G,T}
--volblocksize: <int>K default= '8K'
--sparse: {true/false} default= true
--initiatorgroup: <group-name> default= default
--targetgroup: <group-name> default= default
--status: {online/offline} default= online
delete-luns <project_name> <space-separated list lun names>
san:
<command>:
get-initiators <protocol>
create-initiator <protocol> <alias> <iqn> [optional pool_name]
get-initiator-groups <protocol>
create-initiator-group <protocol> [iqns of created initiators]
get-targets <protocol>
create-target <protocol> <alias>
get-target-groups <protocol>
create-target-group <protocol> <alias> [iqns of created targets]
Examples Using the RESTful API for Oracle ZFS Storage Appliance Systems
The remainder of this article shows a few examples of functions developed using the API with Python, as well as the two modules that can be references for your own development. These modules are unsupported examples and are for non-commercial or non-production use only.
Example 1: Create a RestClient object to Interact with an Appliance Head
This first piece of code creates a RestClient object, which is used to send API requests to the Oracle ZFS Storage Appliance head.
from restclient import *
from restmulty import *
def login(host=None, user=None, password=None, interactive=False):
"""Create and Login to ZFSSA RestClient
:param host: IP Address or Hostname of ZFSSA Head
:param user: ZFSSA user account
:param password: ZFSSA user password
:param interactive: True/False whether this is being used in interactive mode
Returns:
Logged in RestClient Object
"""
if interactive:
os.system(CLEAR)
if not host:
host = raw_input("Enter host name or IP address of ZFSSA Head: ")
if not user:
user = raw_input("Username: ")
if not password:
password = getpass.getpass(prompt="Password: ", stream=None)
client = RestClient(host)
response = client.login(user, password)
if response.status != restclient.Status.CREATED:
print "Login failed:"
print json.dumps(response.getdata(), sort_keys=True, indent=4)
sys.exit(1)
else:
return client
#Execution Example:
client = login("10.0.1.0","opc","solaris123")
Example 2: GET a List of Storage Pools
This example is a function that returns a JSON object of storage pools associated with the Oracle ZFS Storage Appliance systems. It takes a RestClient object, like the one created previously, as an argument
from restclient import *
from restmulty import *
def get_pools(client):
"""Get List of Pools on ZFSSA
:param client: RestClient object
Returns:
JSON Object of Pools in Alphabetical Order
"""
result = client.get("/api/storage/v1/pools")
if result.status != restclient.Status.OK:
print "Failed to get Pools"
sys.exit(1)
else:
pools = result.getdata()
return pools
#Execution Example:
pools = get_pools(client)
Example 3: GET a List of LUNs
This example is a function that returns a JSON object of LUNs associated with a project on an Oracle ZFS Storage Appliance system. It takes a RestClient, a storage pool name, and a project name as arguments.
from restclient import *
from restmulty import *
def get_luns(client, pool_name, project_name):
"""Get a list of LUNs associated with project
:param client: RestClient object
:param pool_name: String of a pool name
:param project_name: String of a project name
Returns:
JSON Object of a Project
"""
result = client.get("/api/storage/v1/pools/%s/projects/%s/luns" % (pool_name, project_name))
if result.status != restclientlib.Status.OK:
print "Failed to get LUNs for Project: %s" % project_name
print json.dumps(result.getdata(), sort_keys=True, indent=4)
sys.exit(1)
else:
return sorted(result.getdata()["luns"], key=lambda k: k["name"])
#Execution Example:
luns = get_luns(client, "spool1", "opc")
Example 4: CREATE Multiple LUNs
This is an example of how you can create multiple LUNs synchronously using a multithreaded approach. It takes a RestClient object, a storage pool name, a project name, a LUN name prefix, the number of LUNs you want to create, the size of the LUNs, and an optional dictionary of arguments (see comments for more details on arguments).
from restclient import *
from restmulty import *
def create_luns(client, pool_name, project_name, prefix, amount, volsize, **kwargs):
"""Create new Project
:param client: RestClient object
:param pool_name: String of a pool name
:param project_name: String of a project name
:param prefix: String Prefix for LUN name
:param amount: Amount of LUNs you want to create
:param volsize: Size of LUN in format " <int>[K,M,G,T]" ie "500G"
:param kwargs: (Dictionary of Optional parameters)
volblocksize: String size of Blocksize in format "<int>K" default="8K"
sparse: String if sparse LUN "true/false" default="true"
initiatorgroup: String initiatorgroup name default="default"
targetgroup: String targetgroup name default="default"
status: String of Status "online/offline " default="online"
Returns:
JSON Object of a list of new LUNs created
"""
luns = get_luns(client, pool_name, project_name)
current_lun_names = [lun['name'] for lun in luns]
volsize = size_to_bytes(volsize)
volblocksize = size_to_bytes(kwargs.get("volblocksize")) or size_to_bytes("8K")
sparse = kwargs.get("sparse") or "true"
initiatorgroup = kwargs.get("initiatorgroup") or "default"
targetgroup = kwargs.get("targetgroup") or "default"
status = kwargs.get("status") or "online"
start = starting_index(current_lun_names, prefix)
names = [prefix + str(i) for i in range(start, start + int(amount))]
lun_details = RestMultiRequest()
for name in names:
lun = dict(name=name, volsize=volsize, volblocksize=volblocksize, sparse=sparse, initiatorgroup=initiatorgroup,
targetgroup=targetgroup, status=status)
path = "/api/storage/v1/pools/%s/projects/%s/luns" % (pool_name, project_name)
request = RestRequest("POST", path, lun)
lun_details.add_request(client, request)
if kwargs.get("execute") is None:
lun_details.run()
lun_details.wait()
luns = []
for run in lun_details.runs:
result = run.result()
luns.append(result.getdata())
return dict(luns=luns)
else:
return lun_details
#Execution Example:
luns = create_luns(client, "spool1", "opc", "db-lun",10,"500G")
Python Scripts
A zip file named ZFSSA_API_Sample_Scripts.zip, which contains the following Python scripts, can be downloaded here:
zfssa.py: This is a library of functions that use the Oracle ZFS Storage Appliance API and can be executed directly from the command line.interactiveCLI.py: This script builds an interactive CLI tool, leveraging the zfssa.py library functions that we previously created.restclient.py: This is the RestClient library from the white paper.restmulty.py: This is the REST multithreaded library from the white paper.
Conclusion
As more workloads move to the cloud, organizations are looking to streamline their IT processes through automation and by codifying their IT infrastructure. Leveraging the RESTful API for Oracle ZFS Storage Appliance systems can aid in this process, making the deployment of LUNs more efficient and removing the need to use the Oracle ZFS Storage Appliance BUI.
About the Author
Kevin Klapak is a product manager on the Oracle Optimized Solutions team. He has a background in computer science and over five years of experience in various IT roles. Since joining Oracle, he has been working on projects related to database migration, Apache Spark and big data analytics, systems security, and cloud computing.