5 Replies Latest reply on Feb 28, 2018 6:48 PM by user10211083

    Terraform Remote State and OCI Object Storage

    user10211083

      We are planning to utilize terraform for our new project in OCI and looking for ways to store remote data state securely with versioning. AWS S3 with dynamo DB offers excellent solution, however we would like to keep everything in Oracle Cloud.

      Any idea what will be the good Oracle Cloud service for storing terraform remote state? To summarize, remote state store should have the following:

      1. Shared storage - Ability for multiple team members to share the same terraform state.

      2. Locking State Files - For obvious reasons, the solution should have the ability to lock the state file. This is very important otherwise we end up with corrupted state, conflicts and data loss.

      3. Isolation - Would like to maintain different terraform state for different environments. For example separate one for Dev, Stage and PROD environment.

       

      Any word from OCI dev if they are planning to enable OCI Object Storage for OCI provider? Or what is there recommendation for immediate needs? As stated above we would like to stay away from AWS S3 and if possible terraform enterprise aka atlas....

        • 1. Re: Terraform Remote State and OCI Object Storage
          3581875

          Hi user10211083

           

          You can store remote state in Oracle Object Storage by creating pre-authenticated requests (PAR). You can create a PAR on an object or bucket and then use the URL returned by this operation in your Terraform configuration file. This will upload the state on Oracle Cloud Infrastructure.

           

          You can find detailed steps published on this wiki- https://github.com/oracle/terraform-provider-oci/wiki/Using-Oracle-Object-Storage-to-store-Terraform-state-files

           

           

          To your above points

           

          1. Creating PAR requests and uploading the Terraform state on object storage will provide the ability for multiple teams to share the Terraform state.
          2. OCI Object storage is strongly consistent (https://docs.us-phoenix-1.oraclecloud.com/Content/Object/Concepts/overview.htm#resources). Hence any read-after-write will return the latest written data. This should guarantee that the state file is securely stored in remote backend. However, currently there are some limitations on the ability to lock state files and we are working on a solution to enhance this feature set.
          3. This can be easily achieved by keeping separate state files for different environment. We are also working on a better solution to achieve isolation.
          • 3. Re: Terraform Remote State and OCI Object Storage
            3148515

            Hello user10211083, can you please mark this question as answered?

            • 4. Re: Terraform Remote State and OCI Object Storage
              user10211083

              Ok, there seems to be an issue with using Object Storage PAR as http backend.

              I setup the backend using the PAR for the bucket and ran terraform apply. It created the state file remotely in the bucket and I can see it. However I cannot do terraform state operations on this state file.

              For example I want to pull the state file locally and run the terraform refresh or run some other operations. I know "http" backend is standard backend and I cannot run many operations, however as per the documentation, I should be able to run terraform state pull to see the contents of the remote state file.

               

               

              $ cat backend.tf

               

              #Remote state goes to iam_bucket under root compartment.

              terraform {

                backend "http" {

                  address = "https://objectstorage.us-ashburn-1.oraclecloud.com/p/VNa20KTF23noJv3kPgG3nVmazesxhX0U0Ldlg-Vv44Y/n/hidbkofc/b/iam_statefiles/o/GROUP_ASSO_iti_admins:svc_iti_admin"

               

                  update_method = "PUT"

                }

              }

               

               

               

              $ cat .terraform/terraform.tfstate

              {

                  "version": 3,

                  "serial": 4,

                  "lineage": "a9a712bf-9d0a-4d93-a0b4-6639592c4582",

                  "backend": {

                      "type": "http",

                      "config": {

                          "address": "https://objectstorage.us-ashburn-1.oraclecloud.com/p/VNa20KTF23noJv3kPgG3nVmazesxhX0U0Ldlg-Vv44Y/n/hidbkofc/b/iam_statefiles/o/GROUP_ASSO_iti_admins:svc_iti_admin",

                          "update_method": "PUT"

                      },

                      "hash": 5640762538862597652

                  },

                  "modules": [

                      {

                          "path": [

                              "root"

                          ],

                          "outputs": {},

                          "resources": {},

                          "depends_on": []

                      }

                  ]

              }

               

               

              $ terraform state pull

              2018/02/28 02:43:46 [INFO] Terraform version: 0.11.3  3802b14260603f90c7a1faf55994dcc8933e2069

              2018/02/28 02:43:46 [INFO] Go runtime version: go1.9.1

              2018/02/28 02:43:46 [INFO] CLI args: []string{"/usr/local/bin/terraform", "state", "pull"}

              2018/02/28 02:43:46 [DEBUG] Attempting to open CLI config file: /home/vagrant/.terraformrc

              2018/02/28 02:43:46 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.

              2018/02/28 02:43:46 [INFO] CLI command args: []string{"state", "pull"}

              2018/02/28 02:43:46 [DEBUG] command: loading backend config file: /home/vagrant/hid-oci-terraform/hid/iam/user_group_membership/iti_admins_svc_iti_admin

              2018/02/28 02:43:46 [WARN] BackendOpts.Config not set, but config found

              2018/02/28 02:43:46 [TRACE] Preserving existing state lineage "a9a712bf-9d0a-4d93-a0b4-6639592c4582"

              2018/02/28 02:43:46 [TRACE] Preserving existing state lineage "a9a712bf-9d0a-4d93-a0b4-6639592c4582"

              2018/02/28 02:43:46 [INFO] command: backend initialized: *legacy.Backend

              2018/02/28 02:43:46 [DEBUG] checking for provider in "."

              2018/02/28 02:43:46 [DEBUG] checking for provider in "/usr/local/bin"

              2018/02/28 02:43:46 [DEBUG] checking for provider in ".terraform/plugins/linux_amd64"

              2018/02/28 02:43:46 [DEBUG] checking for provider in "/home/vagrant/.terraform.d/plugins"

              2018/02/28 02:43:46 [DEBUG] found provider "terraform-provider-oci_v2.0.6"

              2018/02/28 02:43:46 [DEBUG] found valid plugin: "oci", "2.0.6", "/home/vagrant/.terraform.d/plugins/terraform-provider-oci_v2.0.6"

              2018/02/28 02:43:46 [DEBUG] checking for provisioner in "."

              2018/02/28 02:43:46 [DEBUG] checking for provisioner in "/usr/local/bin"

              2018/02/28 02:43:46 [DEBUG] checking for provisioner in ".terraform/plugins/linux_amd64"

              2018/02/28 02:43:46 [DEBUG] checking for provisioner in "/home/vagrant/.terraform.d/plugins"

              2018/02/28 02:43:46 [INFO] command: backend *legacy.Backend is not enhanced, wrapping in local

              2018/02/28 02:43:47 [DEBUG] plugin: waiting for all plugin processes to complete...

              Empty state (no state)

              • 5. Re: Terraform Remote State and OCI Object Storage
                user10211083

                Ignore this. I figured out what I was doing wrong. I created PAR at the bucket level and I was able to write statefiles to it. However terraform could not read with this PAR.

                I created PAR at the object level and was able to write and read from Object Storage.

                 

                On the side note, this is really tedious as I have to create PAR for each statefile that I need to place in remote store on object storage. Not very efficient.