Survey Your Mongo Land

Keyhole was still under active development after I published the
Peek at your MongoDB Clusters like a Pro with Keyhole: Part 1 post in December 2019.  Since then, I have added many features and also changed the format of output files from json to bson.  I’ll use this post to summarize new features and modified functions not available in the previous post. 


A couple months into Keyhole development, a clear direction was surfacing: I needed a tool to fetch all information of a MongoDB cluster in one shot.  I joked about this function the “Know the enemy and hit them where it hurts” feature.  The -allinfo option is perhaps the most commonly used one among users.  Using the -allinfo option, Keyhole collects host and build info, replication stats, server status, size and storage stats of databases and collections, index details and usages, and sharding configurations.  The information was collected from the primary member of a replica set or a mongos of a sharded cluster.  Later, I added the functions of collecting sharding configurations and expanded the retrieval of stats to all instances.  Below is an illustration of how Keyhole connects to all instances of a cluster.

The -redact option can be used together with the -allinfo option to mask collected sample documents.  This feature prevents exposing sensitive data such as PII or PHI.

Output in BSON Format

After downloading Keyhole, use the -allinfo option to collect information.  The -allinfo option replaces the previous -info -v option. Below is an example of retrieving stats from a cluster on Atlas.

keyhole --allinfo "mongodb+srv://"

You’ll find all collected data are saved in a file with a suffix -stats.bson.gz under directory out.  Drag the file to Maobi to view and download an HTML report.

Another way to view the output file is using Keyhole with the -print option. An example command is as follows:

keyhole --print out/<hostname>-stats.bson.gz

The -print option was added to read log messages from Keyhole’s output files and convert the stored bson object to a json string.

Servers Info

With collected stats and configurations from all mongo instances, Maobi summarized stats of all members of a replica set in a tabular view for comparisons.  In addition to the resources specifications, Maobi calculates replication headrooms and makes recommendations if the allocated oplog storage is not able to hold 24 hours worth of oplogs, based on the data found in the collection.  Below is an example of Servers Info.

Chunks Distribution

MongoDB uses the shard key associated with the collection to partition the data into chunks. A chunk contains a subset of sharded data.  A chunk can have no data due to data removal.  The balancer balances among shards by the number of chunks, not data size.  Therefore, manual maintenance may be required to merge chunks in order to balance data size among shards.  Maobi flags those the ratio of empty to total chunks over 20% in red ink.  Below is an example.

Collections and Indexes

There is no change in the collections and indexes section other than cosmetic changes.  Refer to Peek at your MongoDB Clusters like a Pro with Keyhole: Part 1 for details.

Final Words

Using Keyhole and Maobi is a revolutionary first step to diagnose the health of a MongoDB cluster.  I always believe presenting data is an efficient and convincing way to navigate a meeting by eliminating unnecessary guesses.  Using Keyhole and Maobi will definitely help you understand your MongoDB cluster better in many ways.


Popular posts from this blog

Build and Download Keyhole

Maobi - Reports Generator for Keyhole