How to Create Google Cloud Storage Signed URL In Java
Google cloud storage is a powerful and simple cloud based object storage API available as part of the Google cloud platform. Following are some of the key benefits of Google cloud storage,
- Multi-regional storage enables geo-redundant storage of objects with highest availability. This ensures availability of data even in the case of large scale natural disasters at the primary region.
- Infinitely scalable in terms of storage capacity and user access. Just pay for the use.
- Highly durable (99.999999999%) and available(99.95%) object storage.
Some of the common uses cases of Google cloud storage are,
- Host static web pages or web resources
- Primary storage of user uploads in a web application (images, videos or documents)
If you are hosting your Web application in Google cloud platform, Google cloud storage is a good option for storing user uploads. Your application may need to deliver the uploaded documents to an authenticated user of your application through a secure link. Google cloud storage signed URLs enables anyone to securely access a protected file for short period of time. The main advantage of signed URLs is that the user doesn't need a Google account to access the protected file.
Using Google Cloud Storage Signed URLs
Step 1: Create a Google cloud storage bucket and upload a file
From Google cloud console, click on storage and create a bucket. For this example, I used the bucket name - "qptbucket". Note that a multi-regional bucket offers highest availability option at a higher price. Upload a file to the bucket. In the following example, I have uploaded a file named "earth.jpg".
Step 2: Create a service account with access to Google cloud storage
Now we need a Google account which we will use as a proxy to sign and access the protected urls. This type of account is known as a service account in Google cloud platform. From Google cloud console, click on IAM and then click on service accounts option. Create a service account by entering a name, selecting the option "Furnish a new private key" and the key type as JSON. Select the role as Storage => Storage Object Viewer. This gives the necessary storage permissions to the service account.
We will use the downloaded private key for creating the authenticated signed URL. Keep this secure as it cannot be recovered if lost (you can always recreate service accounts or create new keys).
Now you have all the info required for creating signed urls,
- Bucket URL: https://storage.googleapis.com/qptbucket/earth.jpg
- Private key : Value of private_key in downloaded JSON
- Client Account: Value of client_email in downloaded JSON
Step 3: Write a Java program to generate signed URLs
import java.net.URLEncoder; import java.security.KeyFactory; import java.security.PrivateKey; import java.security.Signature; import java.security.spec.PKCS8EncodedKeySpec; import java.util.Base64; // A stand-alone program to generate Google cloud storage signed url for a file. public class GCSSignUrl { // Google Service Account Client ID. Replace with your account. static final String CLIENT_ACCOUNT = "u@demo-00000.iam.gserviceaccount.com"; // Private key from the private_key field in the download JSON file. Replace // this! static final String PRIVATE_KEY = "-----BEGIN PRIVATE KEY -----\nMIIEvgIBADANBgk sd\n-----END PRIVATE KEY-----\n"; // base url of google cloud storage objects static final String BASE_GCS_URL = "https://storage.googleapis.com"; // bucket and object we are trying to access. Replace this! static final String OBJECT_PATH = "/qptbucket/earth.jpg"; // full url to the object. static final String FULL_OBJECT_URL = BASE_GCS_URL + OBJECT_PATH; // expiry time of the url in Linux epoch form (seconds since january 1970) static String expiryTime; public static void main(String[] args) throws Exception { // Set Url expiry to one minute from now! setExpiryTimeInEpoch(); String stringToSign = getSignInput(); PrivateKey pk = getPrivateKey(); String signedString = getSignedString(stringToSign, pk); // URL encode the signed string so that we can add this URL signedString = URLEncoder.encode(signedString, "UTF-8"); String signedUrl = getSignedUrl(signedString); System.out.println(signedUrl); } // Set an expiry date for the signed url. Sets it at one minute ahead of // current time. // Represented as the epoch time (seconds since 1st January 1970) private static void setExpiryTimeInEpoch() { long now = System.currentTimeMillis(); // expire in a minute! // note the conversion to seconds as needed by GCS. long expiredTimeInSeconds = (now + 60 * 1000L) / 1000; expiryTime = expiredTimeInSeconds + ""; } // The signed URL format as required by Google. private static String getSignedUrl(String signedString) { String signedUrl = FULL_OBJECT_URL + "?GoogleAccessId=" + CLIENT_ACCOUNT + "&Expires=" + expiryTime + "&Signature=" + signedString; return signedUrl; } // We sign the expiry time and bucket object path private static String getSignInput() { return "GET" + "\n" + "" + "\n" + "" + "\n" + expiryTime + "\n" + OBJECT_PATH; } // Use SHA256withRSA to sign the request private static String getSignedString(String input, PrivateKey pk) throws Exception { Signature privateSignature = Signature.getInstance("SHA256withRSA"); privateSignature.initSign(pk); privateSignature.update(input.getBytes("UTF-8")); byte[] s = privateSignature.sign(); return Base64.getEncoder().encodeToString(s); } // Get private key object from unencrypted PKCS#8 file content private static PrivateKey getPrivateKey() throws Exception { // Remove extra characters in private key. String realPK = PRIVATE_KEY.replaceAll("-----END PRIVATE KEY-----", "") .replaceAll("-----BEGIN PRIVATE KEY-----", "").replaceAll("\n", ""); byte[] b1 = Base64.getDecoder().decode(realPK); PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(b1); KeyFactory kf = KeyFactory.getInstance("RSA"); return kf.generatePrivate(spec); } }
Note that the above program only compiles in Java 8 or above. For lower Java versions, replace Base64 class with a third party base64 encoder.
Step 4: Use the signed URL to access protected resource
Copy and paste the signed URL generated by the Java program above in a browser window. Voila! you now have access to protected resources through signed URLs. Since the above program sets an expiry period of one minute, you will get the following error when you try to access the signed URL after one minute.
<Error> <Code>ExpiredToken</Code> <Message>The provided token has expired.</Message> <Details>Request has expired: 1487555508</Details> </Error>
Interestingly IAM has no bucket level permission settings (this feature is currently in alpha stage). Hence if you want to restrict a service account to have access only to a specific bucket, you should remove the service account from IAM and then add its permission explicitly in the ACL section of the bucket. Otherwise the service account is authorized to create signed URLs for objects in any bucket.