In Chapter 11, in the Cloud Logging section, we explored various types of Google Cloud logs and learned how to view, filter, and store them. The concept of Cloud Audit Logs was briefly introduced as well. This section aims to provide more visibility into the Cloud Audit Logs topic.
Cloud Audit Logs are a part of Cloud Logging. Still, contrary to logs generated by workloads, they record user actions providing details on who did what activity (for example, created or deleted a resource) from where (such as from a local computer, browser etc.)and when this happened. This type of log is mainly collected for auditing, troubleshooting, and compliance.
There are the following audit log types:
- Admin Activity logs that log users’ creation and deletion of Google Cloud resources. The log collection is enabled by default and can’t be disabled. We also can’t delete such logs. Google will store them for 400 days in the _Required log bucket for free. You can retain them longer by creating a parallel log sink to another destination. Refer to the Configuring log routers section in Chapter 11 to review how this can be achieved.
A simplified view of Admin Activity logs can can be found (at the time of writing this chapter) on the Google Cloud console home page in the ACTIVITY tab.
The following screenshot presents example log entries generated when the app-owner user created and deleted buckets:
Figure 12.42 – The ACTIVITY tab presenting a simplified version of Admin Activity logs
Logs Explorer in the Cloud Logging section provides detailed insight into Cloud Audit Logs. If you select CLOUD AUDIT: activity for Log name, you will see all the Admin Activity entries, as presented in the following screenshot:
Figure 12.43 – Admin activity audit log in Log Explorer generated during a Google Cloud Storage bucket creation
- Data Access logs are generated when users read or write to Google Cloud resources or read metadata information. For example, when a user creates a Google Cloud Storage bucket, an Admin Activity log is generated. When users upload or access objects in this bucket, this activity generates Data Access logs. Multiple users or applications could access those objects, generating a high volume of logs, which is why Data Access logs are disabled by default. You can enable log generation per service in the Audit log view of the IAM & admin section. The following figure presents how Data Access logs can be enabled for a selected service.
Three types of Data Access logs can be enabled separately (see Figure 12.44): Admin Reads (for reading metadata and configuration information), Data Read (for reading user data), and Data Write (for writing user data). By default, those logs are chargeable, have 30 days of retention, and are stored in the _Default log bucket.
To limit the number of generated logs, you can “exempt principals,” meaning logs will not be generated for selected user or service accounts, even if the log collection is enabled. For example, suppose you have an application that constantly reads from a Google Cloud Storage bucket using a service account. This service account could be added as an exempted principal, so its actions won’t generate Data Access logs. On the other hand, you can set up an organization policy to disallow Audit Logging exemptions to enforce logging for all accounts.
Figure 12.44 – Enabling audit logs for data access to Google Cloud Storage
- System Event logs are automatically generated by Google systems and stored for free in the _Required log bucket. You can’t disable or delete them. They record actions driven by the Google Cloud platform and can be viewed in the Logs Explorer. For instance, the following screenshot shows system_event logs generated during a server maintenance action when a Compute Engine VM was migrated to another server. System Event logs could be a good source of information for troubleshooting. For example, if you want to understand why a VM was rebooted, you can check whether a host error caused this by verifying System Event log entries:
Figure 12.45 – System Event log entry generated during a host maintenance event
- Policy Denied logs are generated when a user or service account is denied access to a Google Cloud service because of a security policy violation. The logs are generated and stored in the _Default log bucket and can’t be disabled. Policy Denied logs are chargeable, but you can exclude them from ingestion.
For security reasons, only certain users should have access to audit logs, which may contain sensitive data. The Logs Viewer role grants access to Admin Activity and System Event logs, but to view Data Access audit logs, a separate Private Log Viewer role is required.
This chapter explored setting up an organization and building a resource hierarchy to match a company’s structure. We learned how to control access to Google Cloud resources on a project, folder, and organizational level. One of the most important topics we investigated was working with service accounts used by Google Cloud services and external applications. Lastly, we looked into Cloud Audit Logs that need to be collected to meet an enterprise’s security and compliance needs.