5 Replies Latest reply on Feb 28, 2018 6:48 PM by user10211083

Terraform Remote State and OCI Object Storage

user10211083 Jan 17, 2018 5:58 PM

We are planning to utilize terraform for our new project in OCI and looking for ways to store remote data state securely with versioning. AWS S3 with dynamo DB offers excellent solution, however we would like to keep everything in Oracle Cloud.

Any idea what will be the good Oracle Cloud service for storing terraform remote state? To summarize, remote state store should have the following:

1. Shared storage - Ability for multiple team members to share the same terraform state.

2. Locking State Files - For obvious reasons, the solution should have the ability to lock the state file. This is very important otherwise we end up with corrupted state, conflicts and data loss.

3. Isolation - Would like to maintain different terraform state for different environments. For example separate one for Dev, Stage and PROD environment.

Any word from OCI dev if they are planning to enable OCI Object Storage for OCI provider? Or what is there recommendation for immediate needs? As stated above we would like to stay away from AWS S3 and if possible terraform enterprise aka atlas....

1. Re: Terraform Remote State and OCI Object Storage

3581875 Jan 29, 2018 6:47 PM (in response to user10211083)
Hi user10211083

You can store remote state in Oracle Object Storage by creating pre-authenticated requests (PAR). You can create a PAR on an object or bucket and then use the URL returned by this operation in your Terraform configuration file. This will upload the state on Oracle Cloud Infrastructure.

You can find detailed steps published on this wiki- https://github.com/oracle/terraform-provider-oci/wiki/Using-Oracle-Object-Storage-to-store-Terraform-state-files

To your above points

Creating PAR requests and uploading the Terraform state on object storage will provide the ability for multiple teams to share the Terraform state.
OCI Object storage is strongly consistent (https://docs.us-phoenix-1.oraclecloud.com/Content/Object/Concepts/overview.htm#resources). Hence any read-after-write will return the latest written data. This should guarantee that the state file is securely stored in remote backend. However, currently there are some limitations on the ability to lock state files and we are working on a solution to enhance this feature set.
This can be easily achieved by keeping separate state files for different environment. We are also working on a better solution to achieve isolation.
Like Show 0 Likes(0)

Actions
2. Re: Terraform Remote State and OCI Object Storage

user10211083 Jan 29, 2018 7:45 PM (in response to 3581875)

Awesome! Thanks.
Like Show 0 Likes(0)

Actions
3. Re: Terraform Remote State and OCI Object Storage

3148515 Feb 8, 2018 11:38 PM (in response to user10211083)

Hello user10211083, can you please mark this question as answered?
Like Show 0 Likes(0)

Actions
4. Re: Terraform Remote State and OCI Object Storage

user10211083 Feb 28, 2018 2:53 AM (in response to user10211083)

Ok, there seems to be an issue with using Object Storage PAR as http backend.
I setup the backend using the PAR for the bucket and ran terraform apply. It created the state file remotely in the bucket and I can see it. However I cannot do terraform state operations on this state file.
For example I want to pull the state file locally and run the terraform refresh or run some other operations. I know "http" backend is standard backend and I cannot run many operations, however as per the documentation, I should be able to run terraform state pull to see the contents of the remote state file.

$ cat backend.tf

#Remote state goes to iam_bucket under root compartment.
terraform {
backend "http" {
    address = "https://objectstorage.us-ashburn-1.oraclecloud.com/p/VNa20KTF23noJv3kPgG3nVmazesxhX0U0Ldlg-Vv44Y/n/hidbkofc/b/iam_statefiles/o/GROUP_ASSO_iti_admins:svc_iti_admin"

    update_method = "PUT"
}
}

$ cat .terraform/terraform.tfstate
{
    "version": 3,
    "serial": 4,
    "lineage": "a9a712bf-9d0a-4d93-a0b4-6639592c4582",
    "backend": {
        "type": "http",
        "config": {
            "address": "https://objectstorage.us-ashburn-1.oraclecloud.com/p/VNa20KTF23noJv3kPgG3nVmazesxhX0U0Ldlg-Vv44Y/n/hidbkofc/b/iam_statefiles/o/GROUP_ASSO_iti_admins:svc_iti_admin",
            "update_method": "PUT"
        },
        "hash": 5640762538862597652
    },
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

$ terraform state pull
2018/02/28 02:43:46 [INFO] Terraform version: 0.11.3 3802b14260603f90c7a1faf55994dcc8933e2069
2018/02/28 02:43:46 [INFO] Go runtime version: go1.9.1
2018/02/28 02:43:46 [INFO] CLI args: []string{"/usr/local/bin/terraform", "state", "pull"}
2018/02/28 02:43:46 [DEBUG] Attempting to open CLI config file: /home/vagrant/.terraformrc
2018/02/28 02:43:46 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2018/02/28 02:43:46 [INFO] CLI command args: []string{"state", "pull"}
2018/02/28 02:43:46 [DEBUG] command: loading backend config file: /home/vagrant/hid-oci-terraform/hid/iam/user_group_membership/iti_admins_svc_iti_admin
2018/02/28 02:43:46 [WARN] BackendOpts.Config not set, but config found
2018/02/28 02:43:46 [TRACE] Preserving existing state lineage "a9a712bf-9d0a-4d93-a0b4-6639592c4582"
2018/02/28 02:43:46 [TRACE] Preserving existing state lineage "a9a712bf-9d0a-4d93-a0b4-6639592c4582"
2018/02/28 02:43:46 [INFO] command: backend initialized: *legacy.Backend
2018/02/28 02:43:46 [DEBUG] checking for provider in "."
2018/02/28 02:43:46 [DEBUG] checking for provider in "/usr/local/bin"
2018/02/28 02:43:46 [DEBUG] checking for provider in ".terraform/plugins/linux_amd64"
2018/02/28 02:43:46 [DEBUG] checking for provider in "/home/vagrant/.terraform.d/plugins"
2018/02/28 02:43:46 [DEBUG] found provider "terraform-provider-oci_v2.0.6"
2018/02/28 02:43:46 [DEBUG] found valid plugin: "oci", "2.0.6", "/home/vagrant/.terraform.d/plugins/terraform-provider-oci_v2.0.6"
2018/02/28 02:43:46 [DEBUG] checking for provisioner in "."
2018/02/28 02:43:46 [DEBUG] checking for provisioner in "/usr/local/bin"
2018/02/28 02:43:46 [DEBUG] checking for provisioner in ".terraform/plugins/linux_amd64"
2018/02/28 02:43:46 [DEBUG] checking for provisioner in "/home/vagrant/.terraform.d/plugins"
2018/02/28 02:43:46 [INFO] command: backend *legacy.Backend is not enhanced, wrapping in local
2018/02/28 02:43:47 [DEBUG] plugin: waiting for all plugin processes to complete...
Empty state (no state)
Like Show 0 Likes(0)

Actions
5. Re: Terraform Remote State and OCI Object Storage

user10211083 Feb 28, 2018 6:48 PM (in response to user10211083)

Ignore this. I figured out what I was doing wrong. I created PAR at the bucket level and I was able to write statefiles to it. However terraform could not read with this PAR.
I created PAR at the object level and was able to write and read from Object Storage.

On the side note, this is really tedious as I have to create PAR for each statefile that I need to place in remote store on object storage. Not very efficient.
Like Show 2 Likes(2)

Actions

Go to original post