Google Cloud Storage offers an automated command line tool, gsutil, to manage bucket and object level operations such as create, list, delete, move, and copy on a larger scale.
You can start by using gsutil from the Cloud Shell and practice the following commands:
• To create a unique-name-of-the-bucket bucket with a standard storage class in the europe-central2 location, use the following command:
gsutil mb -c standard -l europe-central2 gs://unique-name-of-the-bucket
• The following command is how you can list objects in a bucket:
gsutil ls -l gs://unique-name-of-the-bucket
You can find an example output of those two actions in the following screenshot:
Figure 8.9 – Creating a bucket and listing its content using gsutil
• To download a file from a bucket, use gsutil cp:
gsutil cp gs://unique-name-of-the-bucket/folder/file_name /folder/destination_folder
Next, you can find an example of downloading a file to a Cloud Shell virtual machine directly and listing the folder’s content to make sure the photo1.jpg file was downloaded successfully:
Figure 8.10 – A file copy from a bucket using gsutil
• If you are transferring or deleting many files, you can improve the performance of such an operation by using multi-threading with the gsutil -m option. For example, with the following command, you can delete all the content of the tmp folder on the unique-name-of-the-bucket bucket:
gsutil rm gs://unique-name-of-the-bucket /tmp/*
If you add the -m option, this operation (especially when the number of manipulated objects is significant) will be faster. Conversely, if you forget to use the -m option, you will be notified that the operation will be done sequentially, and that will take more time. Take a look at the following figure, which shows a notification about using the -m option for sequence operations:
Figure 8.11 – Deleting content of a folder in a bucket where we got a note that the -m option could be used
If you want to practice more, you can find other examples of using gsutil here: https://cloud.google.com/storage/docs/gsutil/commands/help.
Client libraries and REST API access for developers
Google Cloud Storage offers client libraries for languages such as C++, C#, Go, Java, Node.js, PHP, Python, and Ruby so that you can interact with buckets directly from your code.
Furthermore, JSON and XML APIs are also available, so you can, for example, call APIs to upload data from your folder onto a Google Cloud Storage bucket.
Managing access to objects
There are two ways to control access to your objects in a bucket. You can select one of the following options when creating a bucket:
• Uniform, where you use Identity and Access Management (IAM) to define access permissions on a bucket level, so all objects inherit them. To grant access to a bucket, you need to select Edit access, as shown in the following screenshot:
Figure 8.12 – Editing permissions for a bucket with uniform access control
In the next step, you provide a user and a role that you want to assign to this user – for example, a predefined role, Storage Object Viewer, that allows only viewing and listing an object and its metadata:
Figure 8.13 – Granting access to a bucket with uniform access control
Note that, in Figure 8.13, the Resource field shows a bucket, not an individual object.
• Fine-grained access control allows you to assign permissions to individual objects in conjunction with bucket-level permissions. Once the bucket is created, you can select individual objects and assign permissions to access it by selecting Edit access, as shown in the following screenshot:
Figure 8.14 – Editing access permissions per object for a bucket with fine-grained access
Note that, in the following screenshot, the resource that we provide access to is an individual object:
Figure 8.15 – Creating a fine-grained access level for a bucket with permissions applied on an object level
The Fine-grained access option also allows creating time-limited signed URLs for accessing an object through a link. You can give users a signed URL for temporary access to Cloud Storage objects without needing a Google account.
Alternatively, you can edit access to an object and set Entity as Public, Name as all Users, and Access as Reader to allow public access.
As shown in the following example, a public URL can be generated for an object so that everyone can download it:
Figure 8.16 – An object from a fine-grained access bucket that can be accessed by anyone
Let’s put all this information from all the sections of this chapter together and use an example for summarizing what we have discussed in this chapter so far.