Leveraging Terraform Cloud Audit Trails API for Enhanced Governance

Leveraging Terraform Cloud Audit Trails API for Enhanced Governance 

Introduction:

In the realm of infrastructure management and deployment, Terraform Cloud stands as a powerful tool, offering automation and control over cloud resources. A crucial aspect of managing infrastructure is maintaining visibility and accountability, which brings us to the Terraform Cloud Audit Trails API. This Proof of Concept (PoC) serves as an exploration into harnessing the capabilities of this API to access and monitor audit logs, empowering us with insights into changes made to our Terraform Cloud organization's entities.

Objectives:

• Access and retrieve audit logs using the Terraform Cloud Audit Trails API.

• Monitor changes to Terraform Cloud resources, specifically in the context of a Google Cloud Platform (GCP) project.

• Strengthen security, compliance, and governance practices by tracking infrastructure changes and actions.

Prerequisites:

To embark on this PoC journey, make sure you have the following prerequisites in place:

• A Terraform Cloud Plus Edition subscription with access to the Audit Trails API.

• A provisioned GCP project managed through Terraform Cloud.

• An organization token enabling API requests to the Terraform Cloud Audit Trails API.

Implementation Steps:

Step1: Obtain Organization Token

1. Log in to your Terraform Cloud account.

2. Click on your user avatar or initials in the top right corner.

3. Choose "User Settings" from the dropdown menu.

4. In the left navigation pane, select "Tokens."

5. Click on "Create organization token."

6. Provide a name and set token permissions (at least read access for Audit Trails API).

7. Click "Create organization token."

8. Copy the generated token; you'll need it for making API requests.

Step2: API Request Using curl

Execute the following curl command to make the API request and access audit logs for your GCP project. Be sure to replace YOUR_ORGANIZATION_TOKEN with your actual token value:

curl --header "Authorization: Bearer YOUR_ORGANIZATION_TOKEN" \
     --request GET \
     "https://app.terraform.io/api/v2/organization/audit-trail?page[number]=1&since=2020-05-30T17:52:46.000Z"

If you want to run the above curl command in Windows machine, here is the PowerShell command:

$token = "YOUR_ORGANIZATION_TOKEN"
$apiUrl = "https://app.terraform.io/api/v2/organization/audit-trail?page[number]=1&since=2020-05-30T17:52:46.000Z"

$headers = @{
    "Authorization" = "Bearer $token"
}

$response = Invoke-RestMethod -Uri $apiUrl -Headers $headers -Method Get=-
+
$response | ConvertTo-Json

Step3: API Response

Review the API response to verify that audit logs are being successfully retrieved. Confirm that the response includes crucial details such as ID, timestamp, actor information, resource details, and performed actions.

Sample Response:

{
  "data": [
    {
      "id": "ae66e491-db59-457c-8445-9c908ee726ae",
      "version": "0",
      "type": "Resource",
      "timestamp": "2020-06-30T17:52:46.000Z",
      "auth": {
        "accessor_id": "user-MaPuLxAXvtq2PWTH",
        "description": "pveverka",
        "type": "Client",
        "impersonator_id": null,
        "organization_id": "org-AGLwRmx1snv34Yts"
      },
      "request": {
        "id": "4df584d4-7e2a-01e6-6cc0-4adbefa020e6"
      },
      "resource": {
        "id": "at-sjt83qTw3GZatuPm",
        "type": "authentication_token",
        "action": "create",
        "meta": null
      }
    }
  ],
  "pagination": {
    "current_page": 1,
    "prev_page": null,
    "next_page": 2,
    "total_pages": 8,
    "total_count": 778
  }
}

Step4: Interpretation and Analysis

Take a closer look at the audit log data to identify and understand changes made to your GCP project. This could include activities like workspace creations, run actions, and other relevant events. Leverage this data for security, compliance, and change tracking purposes.

Step5: Automating Data Retrieval (Optional)

For added efficiency, consider automating the process of data retrieval using scripting or programming languages. This could involve creating scripts to periodically fetch and process audit logs, facilitating ongoing monitoring of changes over time.

An example Python script to periodically retrieve and store audit logs locally:

# Python script to fetch and store Terraform Cloud audit logs
import requests
import time

# Terraform Cloud organization token
TFC_ORG_TOKEN = "YOUR_ORGANIZATION_TOKEN"

def fetch_audit_logs(since=None):
    api_url = "https://app.terraform.io/api/v2/organization/audit-trail"
    headers = {"Authorization": f"Bearer {TFC_ORG_TOKEN}"}
    params = {"page[number]": 1}
    if since:
        params["since"] = since

    response = requests.get(api_url, headers=headers, params=params)
    print(response.json())  # Add this line to print the response
    if response.status_code == 200:
        return response.json()
    else:
        print(f"Failed to fetch audit logs. Status code: {response.status_code}")
        return None


def process_audit_logs(logs):
    # Process the logs here as needed
    with open("audit_logs.txt", "a") as log_file:
        for log in logs.get("data", []):
            log_file.write(str(log) + "\n")  # Write logs to a local file

def main():
    since = None
    while True:
        logs = fetch_audit_logs(since)
        if logs and logs.get("data"):
            process_audit_logs(logs)
            last_log_timestamp = logs["data"][-1]["timestamp"]
            since = last_log_timestamp
        time.sleep(60)  # Fetch logs every 60 seconds

if __name__ == "__main__":
    main()

Be sure to replace YOUR_ORGANIZATION_TOKEN with your actual token value.

Next Steps

Based on the outcomes of this PoC, you may consider the following next steps:

• Implement automation for periodic data retrieval and analysis of audit logs.

• Explore options to set up monitoring and alerts for critical audit events.

• Evaluate the potential for broader adoption of the Audit Trails API across other projects and teams within your organization.

Conclusion

The Proof of Concept for Terraform Cloud Audit Trails API has demonstrated how we can leverage this powerful tool to access and monitor audit logs. By harnessing the API's capabilities, we gain valuable insights into changes made to our Terraform Cloud resources. This newfound visibility enhances our security, compliance, and governance practices, providing a robust foundation for tracking infrastructure changes and actions.

References: https://developer.hashicorp.com/terraform/cloud-docs/api-docs/audit-trails