Connecting to MongoDB from Java

To connect to MongoDB from Java, you need MongoDB Java driver. Depending on the type of project you are working on there are 3 ways to use this driver. If you are using Gradle or Maven, you can add it as a dependency. If you are not using a build system, you can use the legacy fat jar of the MongoDB Java driver.

Use the Gradle dependency,

dependencies {
    compile 'org.mongodb:mongodb-driver:3.4.2'
}

Or use the Maven dependency,

<dependencies>
    <dependency>
        <groupId>org.mongodb</groupId>
        <artifactId>mongodb-driver</artifactId>
        <version>3.4.2</version>
    </dependency>
</dependencies>

Or use the MongoDB library jar file directly. This jar contains all the dependencies including the BSON library.

Note that the above example use version 3.4.2 of the MongoDB driver. You may want to use a later version if available.

Connecting to MongoDB from Java

The following program shows how to connect to MongoDB from Java. Note the use of a singleton pattern for MongoClient where the instance is loaded lazily. MongoClient is designed for multi-threaded calls and has a connection pool internally (default pool size in Java is 100). Hence for most use cases you will need just one instance of MongoClient. Using multiple MongoClient instances may lead to too many open connections in MongoDB.

The following example inserts and queries records in a collection named "user" in the "test" database. We also assume that the MongoDB is running on the local machine with default port.

import org.bson.Document;

import com.mongodb.Block;
import com.mongodb.MongoClient;
import com.mongodb.MongoClientURI;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;

// Demo program for connecting to MongoDB.
// Uses singleton pattern for MongoClient.
public class MongoDemo {

    private static MongoClient mClient;

    // Singleton for MongoClient
    // Creates a single connection pool internally
    private MongoClient getMongoClient() {
        if (mClient == null) {
            mClient = new MongoClient(new MongoClientURI("mongodb://localhost:27017"));
        }
        return mClient;
    }

    // Utility method to get database instance
    private MongoDatabase getDB() {
        return getMongoClient().getDatabase("test");
    }

    // Utility method to get user collection
    private MongoCollection<Document> getUserCollection() {
        return getDB().getCollection("user");
    }

    public static void main(String[] args) {
        MongoDemo demo = new MongoDemo();
        demo.insertUser();
        demo.queryUsers();
    }

    // Read all documents from user collection
    private void queryUsers() {
        getUserCollection().find().forEach(new Block<Document>() {
            @Override
            public void apply(Document t) {
                System.out.println(t);
            }
        });
    }

    // Insert a document in user collection
    private void insertUser() {
        Document document = new Document("username","qpt")
                            .append("email", "testemail@example.com");
        getUserCollection().insertOne(document);
    }
}

We have used an iterator with a lambda function for reading Mongo documents. Alternatively you can use the DBCursor class to iterate through the document collection.

MongoDB Java References

How to Create Google Cloud Storage Signed URL In Java

Google cloud storage is a powerful and simple cloud based object storage API available as part of the Google cloud platform. Following are some of the key benefits of Google cloud storage,

  • Multi-regional storage enables geo-redundant storage of objects with highest availability. This ensures availability of data even in the case of large scale natural disasters at the primary region.
  • Infinitely scalable in terms of storage capacity and user access. Just pay for the use.
  • Highly durable (99.999999999%) and available(99.95%) object storage.

Some of the common uses cases of Google cloud storage are,

  • Host static web pages or web resources
  • Primary storage of user uploads in a web application (images, videos or documents)

If you are hosting your Web application in Google cloud platform, Google cloud storage is a good option for storing user uploads. Your application may need to deliver the uploaded documents to an authenticated user of your application through a secure link. Google cloud storage signed URLs enables anyone to securely access a protected file for short period of time. The main advantage of signed URLs is that the user doesn’t need a Google account to access the protected file.

Using Google Cloud Storage Signed URLs

Step 1: Create a Google cloud storage bucket and upload a file

From Google cloud console, click on storage and create a bucket. For this example, I used the bucket name – "qptbucket". Note that a multi-regional bucket offers highest availability option at a higher price. Upload a file to the bucket. In the following example, I have uploaded a file named "earth.jpg".

Step 2: Create a service account with access to Google cloud storage

Now we need a Google account which we will use as a proxy to sign and access the protected urls. This type of account is known as a service account in Google cloud platform. From Google cloud console, click on IAM and then click on service accounts option. Create a service account by entering a name, selecting the option "Furnish a new private key" and the key type as JSON. Select the role as Storage => Storage Object Viewer. This gives the necessary storage permissions to the service account.

We will use the downloaded private key for creating the authenticated signed URL. Keep this secure as it cannot be recovered if lost (you can always recreate service accounts or create new keys).

Now you have all the info required for creating signed urls,

Step 3: Write a Java program to generate signed URLs

import java.net.URLEncoder;
import java.security.KeyFactory;
import java.security.PrivateKey;
import java.security.Signature;
import java.security.spec.PKCS8EncodedKeySpec;
import java.util.Base64;

// A stand-alone program to generate Google cloud storage signed url for a file.
public class GCSSignUrl {

    // Google Service Account Client ID. Replace with your account.
    static final String CLIENT_ACCOUNT = "u@demo-00000.iam.gserviceaccount.com";

    // Private key from the private_key field in the download JSON file. Replace
    // this!
    static final String PRIVATE_KEY = "-----BEGIN PRIVATE KEY -----\nMIIEvgIBADANBgk sd\n-----END PRIVATE KEY-----\n";

    // base url of google cloud storage objects
    static final String BASE_GCS_URL = "https://storage.googleapis.com";

    // bucket and object we are trying to access. Replace this!
    static final String OBJECT_PATH = "/qptbucket/earth.jpg";

    // full url to the object.
    static final String FULL_OBJECT_URL = BASE_GCS_URL + OBJECT_PATH;

    // expiry time of the url in Linux epoch form (seconds since january 1970)
    static String expiryTime;

    public static void main(String[] args) throws Exception {

        // Set Url expiry to one minute from now!
        setExpiryTimeInEpoch();

        String stringToSign = getSignInput();
        PrivateKey pk = getPrivateKey();
        String signedString = getSignedString(stringToSign, pk);

        // URL encode the signed string so that we can add this URL
        signedString = URLEncoder.encode(signedString, "UTF-8");

        String signedUrl = getSignedUrl(signedString);
        System.out.println(signedUrl);
    }

    // Set an expiry date for the signed url. Sets it at one minute ahead of
    // current time.
    // Represented as the epoch time (seconds since 1st January 1970)
    private static void setExpiryTimeInEpoch() {
        long now = System.currentTimeMillis();
        // expire in a minute!
        // note the conversion to seconds as needed by GCS.
        long expiredTimeInSeconds = (now + 60 * 1000L) / 1000;
        expiryTime = expiredTimeInSeconds + "";
    }

    // The signed URL format as required by Google.
    private static String getSignedUrl(String signedString) {
        String signedUrl = FULL_OBJECT_URL 
                           + "?GoogleAccessId=" + CLIENT_ACCOUNT 
                           + "&Expires=" + expiryTime
                           + "&Signature=" + signedString;
        return signedUrl;
    }

    // We sign the expiry time and bucket object path
    private static String getSignInput() {
        return "GET" + "\n" 
                    + "" + "\n" 
                    + "" + "\n" 
                    + expiryTime + "\n" 
                    + OBJECT_PATH;
    }

    // Use SHA256withRSA to sign the request
    private static String getSignedString(String input, PrivateKey pk) throws Exception {
        Signature privateSignature = Signature.getInstance("SHA256withRSA");
        privateSignature.initSign(pk);
        privateSignature.update(input.getBytes("UTF-8"));
        byte[] s = privateSignature.sign();
        return Base64.getEncoder().encodeToString(s);
    }

    // Get private key object from unencrypted PKCS#8 file content
    private static PrivateKey getPrivateKey() throws Exception {
        // Remove extra characters in private key.
        String realPK = PRIVATE_KEY.replaceAll("-----END PRIVATE KEY-----", "")
                .replaceAll("-----BEGIN PRIVATE KEY-----", "").replaceAll("\n", "");
        byte[] b1 = Base64.getDecoder().decode(realPK);
        PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(b1);
        KeyFactory kf = KeyFactory.getInstance("RSA");
        return kf.generatePrivate(spec);
    }

}

Note that the above program only compiles in Java 8 or above. For lower Java versions, replace Base64 class with a third party base64 encoder.

Step 4: Use the signed URL to access protected resource

Copy and paste the signed URL generated by the Java program above in a browser window. Voila! you now have access to protected resources through signed URLs. Since the above program sets an expiry period of one minute,  you will get the following error when you try to access the signed URL after one minute.

<Error>
  <Code>ExpiredToken</Code>
  <Message>The provided token has expired.</Message>
  <Details>Request has expired: 1487555508</Details>
</Error>

Interestingly IAM has no bucket level permission settings (this feature is currently in alpha stage). Hence if you want to restrict a service account to have access only to a specific bucket, you should remove the service account from IAM and then add its permission explicitly in the ACL section of the bucket. Otherwise the service account is authorized to create signed URLs for objects in any bucket.

How to Create SHA256 RSA Signature Using Java

SHA256 with RSA signature is an efficient asymmetric encryption method used in many secure APIs. This algorithm first calculates a unique hash of the input data using SHA256 algorithm. The hash is then encrypted with a private key using the RSA algorithm. This is useful in scenarios where we only need to verify that the data is not tampered and is authentic. The original data itself is sent without encryption (usually protected by a TLS secure link). Encrypting just the hash is much faster due to its small size. For example, SHA256 with RSA is used to generate the signature part of the Google cloud storage signed URLs.

The following Java program generates a signature from an input string and a primary key in the unencrypted PKCS#8 format (If you are using Google cloud storage signed URLs, this value is in the private_key field of the downloaded JSON file)

import java.security.KeyFactory;
import java.security.Signature;
import java.security.spec.PKCS8EncodedKeySpec;
import java.util.Base64;

public class SHA256RSA {
    
    public static void main(String[] args) throws Exception {
        String input = "sample input";
        
        // Not a real private key! Replace with your private key!
        String strPk = "-----BEGIN PRIVATE KEY-----\nMIIEvwIBADANBgkqhkiG9"
                + "w0BAQEFAASCBKkwggSlAgEAAoIBAQDJUGqaRB11KjxQ\nKHDeG"
                + "........................................................"
                + "Ldt0hAPNl4QKYWCfJm\nNf7Afqaa/RZq0+y/36v83NGENQ==\n" 
                + "-----END PRIVATE KEY-----\n";
        
        String base64Signature = signSHA256RSA(input,strPk);
        System.out.println("Signature="+base64Signature);
    }

    // Create base64 encoded signature using SHA256/RSA.
    private static String signSHA256RSA(String input, String strPk) throws Exception {
        // Remove markers and new line characters in private key
        String realPK = strPk.replaceAll("-----END PRIVATE KEY-----", "")
                             .replaceAll("-----BEGIN PRIVATE KEY-----", "")
                             .replaceAll("\n", "");

        byte[] b1 = Base64.getDecoder().decode(realPK);
        PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(b1);
        KeyFactory kf = KeyFactory.getInstance("RSA");

        Signature privateSignature = Signature.getInstance("SHA256withRSA");
        privateSignature.initSign(kf.generatePrivate(spec));
        privateSignature.update(input.getBytes("UTF-8"));
        byte[] s = privateSignature.sign();
        return Base64.getEncoder().encodeToString(s);
    }
}

Note that the signed string is returned in base64 encoded form. Also note that for Base64 class, you need to use Java 8 or above. If you are using lower Java version, get a third party Base64 encoder.

How to Generate Expires Value for Amazon S3 or Google Cloud Storage Using Java

Google Cloud Storage and Amazon S3 supports signed URLs for protected objects. Signed URLs can be used to enable temporary public access to protected content. Anyone with the signed URL can access the protected resource. However the validity of the signed URL is specified by a query parameter named Expires. This parameter specifies the validity of the signed URL in terms of seconds elapsed since 1st January 1970 (also know as epoch).

Following is a sample signed URL to a protected Google cloud storage object (signature is truncated for brevity),

https://storage.googleapis.com/samplebucket/samplefile.jpg? GoogleAccessId=serviceuser@demoaccount-12345.iam.gserviceaccount.com& Expires=1487561198&Signature=nVmsboiEMOBZONx…

In the above URL, the Expires value specifies the validity of the URL as the number of seconds since 1st January 1970 (UTC).

Generating Expires Value in Java

The following simple method in Java can be used to generate Expires query parameter in an Amazon S3 or Google cloud storage signed URL. It will calculate an expiry time since the execution of the method itself. The parameter to the method is the number of seconds of validity of the signed URL since the execution of the method.

    // Return an Expires value calculated as number of seconds of validity from now!
    private long getExpiresForSecondsSinceNow(long seconds) {
        long now = System.currentTimeMillis();
        long expiryTime = (now + seconds*1000L)/1000; // convert millis to seconds
        return expiryTime;
    }

Note that the Expires field in the signed URL query parameter must be the same one specified in the original signed text used for generation of signed URL. If you are using signed URLs for delivering protected media content such as video or images, it is important to keep a short expiry time such as 60 seconds.

Buildship 2.0 and Spring STS 3.8.3

Spring STS 3.8.3 doesn’t work well with Buildship 2.0 Gradle plugin. When you try to create a new Spring starter project with Buildship Gradle plugin selected, you will get the following error,

can not import using Gradle (Buildship) because Buildship Gradle Tooling is not installed. You can install it from Eclipse Marketplace.

Note that you will get this error even after installing Buildship 2.0 from the Eclipse marketplace. This is actually an STS bug since it had a Buildship version dependency built into it. Luckily this bug is fixed in the upcoming STS 3.8.4 release.

There are two solutions to this problem. You can either get a nightly build of the upcoming STS 3.8.4 release or you can install an older version of the Buildship plug-in.

Option 1: Installing Nightly Build of STS 3.8.4

The quickest solution to the above bug is to install the latest nightly build of the upcoming STS 3.8.4 release. The only downside of this approach is that 3.8.4 itself may have other bugs! To install the nightly build in Spring STS, click on Help => Install New Software. In the work with field, add the URL to the Spring STS nightly build for Eclipse Neon. Install the updates listed.

Option 2: Installing Older Version of Buildship

If you have Buildship 2.0 already installed, uninstall it from Help => Installation Details. Then click on Help => Install New Software. In the work with field, add the URL to the Buildship 1.0.21 plug-in for Eclipse Neon. Install the Buildship 1.0.21 version listed. Voila! Now you can create Spring starter projects using Buildship.

References

Bug report in GitHub