If you're ready to step into MongoDB version 4.x, or looking to move to the next generation of Compose for MongoDB, look no further than IBM Cloud Databases for MongoDB.

Since the GA launch of Databases for MongoDB back in February, we've had a lot of interest in migrating over to Databases for MongoDB from Compose. So, our engineering team got together and designed an easy tool that will allow you to export your existing Compose for MongoDB databases over to Databases for MongoDB right from the Compose console.

If you've seen the import tool, it's pretty similar. All you need to provide is a Databases for MongoDB URI, click a button, and it will export your data. At the same time, your existing Compose for MongoDB database will still be connected to your application and accepting new writes. This means you'll be able to avoid unnecessary interruptions while you're getting your new deployment on IBM Cloud set up.

In this article, we'll walk you through the export process by setting up a Databases for MongoDB deployment, exploring the export tool for Compose for MongoDB, and exporting a database. Let's get started ...

Provisioning Databases for MongoDB on IBM Cloud

To export your Compose for MongoDB data to Databases for MongoDB, you'll first need to create the deployment on IBM Cloud. If you haven't signed up on IBM Cloud, you can click here. After signing up for the service, or signing into your IBM Cloud account, select the Create resource button at the top of your cloud dashboard.

From there, type MongoDB in the search bar and you'll see the Databases for MongoDB box appear. Click that box.

Provisioning Options

This will take you the Databases for MongoDB provisioning page. From here, you can make some changes to the default options before provisioning the database. You'll see that MongoDB version 4.x is set as the preferred version.

Notice the RAM and Disk options that are available. You'll want your RAM and Disk sizes to be the same, or a little larger than what you currently have for your Compose deployment.

Unlike Compose, all IBM Cloud Databases are manually scaled. This means that you'll have to watch what you use for RAM and disk, and scale up if you're anticipating a heavy workload. For disk space, all IBM Cloud Databases use block storage. Block storage can only be scaled up, and not scaled down once disk space is allocated.

Other options that are available are database public and private endpoints, selecting shared CPU or dedicated cores, and IBM Cloud Key Protect.

You can read more about these options in the Databases for MongoDB documentation.

Creating the Deployment

Once you're done with the configuration, select Create at the bottom of the panel to kick off the provisioning process.

You'll be taken to the IBM Cloud Resource list panel where you'll see your database with the status "Provision in progress".

When it's done provisioning, you'll see the database in your IBM Cloud Resource list panel with the status "provisioned".

Click on the database to get to its management settings.

Changing the Default Admin Password

After provisioning the database, you'll first want to reset the administrator password of the deployment. You'll do that by clicking on the Settings tab at the top of your Databases for MongoDB management panel.

From there, scroll down until you see the Change Password panel. Where it says New Password, create a new password for the deployment's admin user. Make sure to use at least 15 characters.

Connecting to Databases for MongoDB

With a new Databases for MongoDB deployment, you'll be provided with a replica set of two endpoints to connect to your database. Databases for MongoDB also uses a TLS certificate, so you'll need to configure your MongoDB application driver to accept two hosts and a TLS certificate.

In the Connections panel, you'll have two tabs. The MongoDB tab contains the mongodb:// protocol to connect your application to MongoDB.

mongodb://$USERNAME:$PASSWORD@aaa.databases.appdomain.cloud:99999,aaa.databases.appdomain.cloud:88888/ibmclouddb?authSource=admin&replicaSet=replset  

The CLI tab gives you the mongo command that you can use in the terminal to connect directly to your database.

With your Databases for MongoDB deployment set up, let's move on to export the databases from Compose for MongoDB to Databases for MongoDB.

Exporting Compose for MongoDB Databases

In your Compose for MongoDB deployment, you'll need to select the database that you want to export. The export tool doesn't export the entire deployment. So, you'll have to export each database individually by using the export tool from each database. While all the data from the Compose database will be imported to Databases for MongoDB, users will not be imported. So make sure to recreate any users you had in your Compose deployment.

Once you've selected a database from your Compose for MongoDB console, you will see the Exports button on the left menu bar.

Click the button, and you'll be taken to the Exports view for that database. From here, click on the New Export button on the top right of the view.

In this view, you can see that there are Source and Destination sections.

Let's show you what your options are here.

Tailing and the Oplog

In the Source section, you will see a checkbox to Enable Tailing for your Compose for MongoDB deployment.

Tailing is particularly useful if you are anticipating a lot of writes to your Compose database while you're migrating. What this does is after the initial bulk export of your Compose for MongoDB data to your Databases for MongoDB deployment, the new database will continue to read the oplog of your Compose database for up to three days, or until you cancel the process. After tailing has ended, your new database will have the same data as your Compose database.

Before checking Enable Tailing, you'll need to add the oplog addon to your Compose deployment. You can select that by clicking on the link hyperlink under About Oplog Access description on the right, or by going to your Compose for MongoDB's Overview page, and clicking the Add-ons button which takes you to the Add-ons view.

You'll see a list of supported add-ons and click on the Add button next to Oplog Access.

This will take you to the Oplog Access view where you'll be asked if you want to add oplog access. Click the Add MongoDB Oplog Access button to add it. Keep the Enable SSL? checkbox checked.

Exporting to IBM Cloud

After the oplog has been added, go back to your Exports view. Under Destination, you'll need to provide the mongodb:// connection string that you were provided for your Databases for MongoDB deployment on IBM Cloud which looks like:

You'll need to enter the admin user and password, or any user you've set up in Databases for MongoDB that has privileges to insert data. Enter the Databases for MongoDB URI in the destination box, and remember to change the database name in the URI to a database name that you want in your Databases for MongoDB deployment. The default database name in the URI is ibmclouddb. If you don't change the name, your data will be put into ibmclouddb and not the name of the database that you're exporting from.

Now, click on Export Database at the bottom of the Exports view. With the export started, it will take you back to your Exports view.

Under "status" you'll see "Starting" initially. This means that the export process is starting and during that time your Databases for MongoDB database is reading and writing the data from your Compose for MongoDB database. This includes all of the data and the collections within the specified database you're exporting.

After the "Starting" status, you will see an update to the status to "tailing". Now, if any writes are made to your Compose database, they will be carried over into your new Databases for MongoDB deployment.

There is a caveat. Only new data will be read and written from the Compose for MongoDB oplog. If a collection is created with new data, that collection will not be added to the new Databases for MongoDB database during the tailing process. If you want to include a new collection, you'll have to either wait and add it to your new database, or you'll have to cancel the tailing operation and start again.

As mentioned previously, tailing occurs for only three days, or until you decide to cancel the tailing operation. Once three days have passed, tailing will automatically be canceled and you will see "Cancelled" as the export status.

Importing Data Once Only

If for any reason, you just want to import the data from the Compose for MongoDB database to Databases for MongoDB without tailing, you can do that, too. Just don't select Enable Tailing.

From the Compose for MongoDB's Exports view, just add the Databases for MongoDB URI, including the database name that you want to move the data to. After that, you can click the Export Database button to export and you'll see the same status as "Starting". Then when the migration is complete you will see "Complete" as the status.

Porting your Application

Whether you choose either option, you should run a few checks to make sure that your data has been successfully migrated. These include data validation and structural compares. At the same time, since you will be using a much newer version of MongoDB, you'll need to make any necessary changes to your application that supports that database version. Consult the MongoDB release notes for database changes by version.

Once you've checked, and double-checked that your data is successfully stored in your new Databases for MongoDB database and that your application supports the much newer database version, you can swap out your Compose for MongoDB connection string with your new Databases for MongoDB connection string. At that point, all your new reads and writes will take place in your new deployment, and you can deprovision the Compose for MongoDB deployment.

Summary

Migrating your data from Compose for MongoDB to Databases for MongoDB is only a start. With lots of enhancements, added security and database features, you'll thoroughly love working with MongoDB on IBM Cloud. Compose makes it easy for you to choose which migration option is best, without disturbing your production databases while benefiting from all the improvements that MongoDB has to offer.

If you have any questions about migrating your data to IBM Cloud Databases for MongoDB, feel free to contact us at support@compose.com.

attribution Gerrit Vermeulen