Automating Data Audit Process with Instaclustr User Actions API

In today’s fast-paced digital world, managing and auditing data efficiently is critical for maintaining security, compliance, and operational excellence. Automating these processes can save time, reduce errors, and provide timely insights into system activities.

Since the release of the Users Actions feature on the Instaclustr platform, we have added API support for programmatic access to the logs. This blog will demonstrate how to use this API to streamline your data audit workflows.

The User Actions feature provides detailed logging and tracking of user actions across organizations, accounts, and clusters. It is designed for Instaclustr users, especially administrators and auditors, to gain insights into the actions taken on their clusters.

User Actions are recorded regardless of who triggered them, whether it was account users or the Instaclustr Support team. Utilizing this feature enhances security, audit trails, and operational transparency. With customized filters such as action types, user, and account, the User Actions API allows comprehensive monitoring and analysis. This capability is particularly valuable for maintaining compliance and diagnosing issues.

What is the User Actions API?

The User Action API provides a way to track and audit user actions within the Instaclustr platform. This API allows you to retrieve detailed information about user activities, such as login attempts, cluster modifications, and other critical operations. By utilizing this API, you can automate the collection and analysis of audit logs, ensuring comprehensive monitoring of user activities.

Key Features of the User Actions API

Comprehensive Tracking: The API captures a comprehensive set of user actions, ensuring you have a complete audit trail.
Detailed Logs: More than 100+ recorded user actions are logged with detailed information, including timestamps, usernames, and specific actions performed.
Integration Capabilities: The API can be integrated with various data analysis and monitoring tools, enhancing your ability to analyze and respond to user activities.

Getting Started with the User Actions API

To begin automating your data audit process, you’ll need to access the User Actions API. The Instaclustr API documentation provides information on how to access the user actions endpoints and which query parameters to use for better filtering. Below are some examples of such parameters:

Query Parameter	Description
crudOperations: array of unique strings	Used in both account and organization level endpoints. The response returns the operation type in the CRUD (create, read, update, delete) form. Please refer to the Instaclustr API documentation for more information on the accepted values for this query parameter.
getActionCounts: boolean	Default: true Used in both account and organization level endpoints When set to true, the response includes counts of each action type returned.
organisationLogsOnly: boolean	Default: false Used only in the organization level endpoint When set to true, the response will include only the logs related to the organization
accountLogsOnly: boolean	Default: false Used only in the organization level endpoint When set to true, the response will include only the logs related to the organization’s accounts

Below are listed the key endpoints you will use:

1. List Account User Actions

Endpoint	GET /cluster-management/v2/operations/user-actions/v2
Description	This endpoint retrieves a list of user actions, providing details about each action performed within the Instaclustr account.
Documentation	User Actions API – Account level
API Key	Provisioning API Key

2. List Organization User Actions

Endpoint	GET /v1/organisations/{organisationId}/user-actions
Description	This endpoint lists user actions within the context of organization management (including all the actions from the child accounts of the organization), allowing you to track organizational activities.
Documentation	Organization User Actions API
API Key	Organization Management API Key

Use Cases

Here are some scenarios in which Instaclustr User Actions API would be used:

1. Monitoring User Activity

Security teams can investigate unauthorized changes performed by a suspected user by filtering the user actions for that user. This request will generate a detailed report of all activities performed up until that point. The report would include cluster or API key modifications, resource deletions, and many more. For more user action filters, refer to the API docs.

2. Auditing Actions within a Specific Organization

Compliance teams can audit actions within My Org (member of) to ensure adherence to regulatory requirements. By filtering events by organization name, they can verify that all activities comply with internal and external policies and regulations.

3. Identifying Anomalous Behavior

The security team can identify potential security threats by monitoring for unusual user behaviour. By filtering for actions like multiple API key creations/deletions (ACCOUNT_GENERATE_API_KEY, ACCOUNT_REVOKE_API_KEY), or sudden mass deletions (DELETE_CLUSTER, REQUEST_CLUSTER_DELETE), they can alert and investigate users or organizations exhibiting malicious activity, potentially preventing security breaches. To further mitigate such threats, we have implemented measures like two-factor authentication, ensuring an additional layer of security. This helps verify the legitimacy of actions and reduces the risk of unauthorized or accidental operations.

4. Managing Resource Utilization and Efficiency

The operations team can monitor resource utilization with My Org to optimize efficiency. By filtering user actions related to resource (i.e. clusters, data centres, firewall rules etc.) creation and deletion, they can track which resources are being created or removed, ensuring efficient use and minimizing unnecessary expenses.

Automating Data Audits via User Action API

Let’s consider a scenario where you want to automate the collection of audit logs for user actions in your organization. You can set up a Python script to call the User Action Events API at regular intervals (Note: In line with Instaclustr monitoring metrics, user action data is refreshed every 20 seconds), which would retrieve the latest user actions and store them in a database or a log management system.

Benefits of Automating Data Audits:

Improved Security: Automated audits help identify unauthorized or suspicious activities promptly.
Regulatory Compliance: Automated logging ensures you meet compliance requirements by maintaining detailed records of user actions.
Operational Efficiency: API helps reduce the manual effort required for auditing, allowing your team to focus on more strategic tasks. Manual effort implies that the users access the Instaclustr console and filter it via the provided UI (user interface). By utilizing the API, users can “set and forget” – they can build detailed reports, set alerts and monitor their resources without any additional manual steps.

How-to Steps

The Python script below is a simple example of fetching the latest user actions from the API.

1. Prepare your script

The script needs to send a GET request to the Instaclustr API to retrieve the latest user actions and store the response in a JSON file. For this, it is required that the requests and json libraries are imported.

import requests 
import json

1 2	import requests import json

Sending a GET request to the Instaclustr API requires an endpoint and an API Key. The endpoint for listing an organisation’s user actions is found on the Instaclustr API documentation and it requires Basic Authentication via Organization Management API key.

# Replace with your Instaclustr API key and endpoint 
API_KEY = 'your_api_key' 
ENDPOINT = 'https://api.instaclustr.com/v1/organisations/{organisationId}/user-actions'

1

2

3

# Replace with your Instaclustr API key and endpoint

API_KEY = 'your_api_key'

ENDPOINT = 'https://api.instaclustr.com/v1/organisations/{organisationId}/user-actions'

2. Get user actions

The function defined below will send a GET request to the Instaclustr API and store the response in a variable. The function returns the response variable in JSON format. If the request is not successful, it will raise an HTTPError containing the appropriate Instaclustr API’s error message.

def get_user_actions(api_key, endpoint):
headers = {
'Authorization': f'Bearer {api_key}',
'Content-Type': 'application/json'
}
response = requests.get(endpoint, headers=headers)
if response.status_code == 200:
return response.json()
else:
response.raise_for_status()

1

2

3

4

5

6

7

8

9

10

def get_user_actions(api_key, endpoint):

headers = {

'Authorization': f'Bearer {api_key}',

'Content-Type': 'application/json'

}

response = requests.get(endpoint, headers=headers)

if response.status_code == 200:

return response.json()

else:

response.raise_for_status()

3. Store the user actions

Another function is defined below. This function will dump the content of the data argument into a JSON file called audit_log.json.

Note: You can modify the save_audit_log() function to store the data in a database or any other preferred storage.

def save_audit_log(data): 
# Save the audit log data to a file or a database 
with open('audit_log.json', 'a') as f: 
json.dump(data, f, indent=4) 
f.write('\n')

1

2

3

4

5

def save_audit_log(data):

# Save the audit log data to a file or a database

with open('audit_log.json', 'a') as f:

json.dump(data, f, indent=4)

f.write('\n')

4. Run the script

Using the two functions defined above, the script will get the user actions, store them in a variable and save the audit logs in a JSON file:

def main(): 
# Fetch user actions 
user_actions = get_user_actions(API_KEY, ENDPOINT) 

# Save the user actions to audit log 
save_audit_log(user_actions) 

if __name__ == '__main__': 
main()

1

2

3

4

5

6

7

8

9

def main():

# Fetch user actions

user_actions = get_user_actions(API_KEY, ENDPOINT)

# Save the user actions to audit log

save_audit_log(user_actions)

if __name__ == '__main__':

main()

save_audit_log function to store the data in a database or any other preferred storage.

Conclusion

Automating your data audit process with Instaclustr’s User Action API can significantly enhance your organization’s ability to monitor and manage user activities. By leveraging this API, you can ensure comprehensive tracking, detailed logging, and seamless integration with your existing tools and workflows. Start using the User Action API today to streamline your data audit processes and boost your operational efficiency.

For more information, check out our blog  about the User Actions feature and explore the possibilities of automating your audit workflows.

If you have any questions about this feature, please contact Instaclustr Support at any time. If you are not a current Instaclustr customer and you’re interested in learning more, register for a free trial and spin up your first cluster for free!

Speak To An Expert Today.

Contact us