MongoDB FTDC: The Open Source Overhaul
MongoDB FTDC: The Open Source Overhaul
One of the biggest challenges for the MongoDB community has always been the "tool gap."
Internal MongoDB engineers have access to powerful, proprietary tools like t2 for visualizing FTDC (Full-Time Diagnostic Data Capture) data. While t2 excels at revealing metric details, its interface is often considered dated. More frustratingly, MongoDB support engineers are constantly asked for the t2 tool by external customers, only to have to decline.
The reality is simple: External users and consultants don't have access to this essential internal visibility.
For years, my tool Keyhole has helped to bridge this gap, relying on the core module mongo-ftdc. This module has long provided an Assessment Panel view of critical FTDC metrics in high fidelity. However, while mongo-ftdc was functional, it had been sitting idle while MongoDB continued its rapid evolution and scale.
My continuous motivation is to promote the successful use of MongoDB. To achieve that, the community needs the same high-quality performance insights that internal teams enjoy. So, I took the reliable mongo-ftdc core and subjected it to a massive, much-needed optimization effort.
The Engine Upgrade: Faster and Ready for Scale
As MongoDB matured into versions 7.x and 8.x, the diagnostic FTDC files grew exponentially larger. The old parser was struggling to handle this data efficiently, turning essential analysis into a bottleneck.
I improved and optimized the parser to be faster, lighter, and more capable for modern database loads:
9× Faster: Load times are now near-instant, drastically reducing the time it takes to begin analysis.
Efficient: These optimizations led to 66% fewer memory allocations and eliminated millions of string operations.
Modern: The parser now fully supports the expanded metric set found in the latest MongoDB server versions.
Visualization That Closes the Gap
We don't need a custom, clunky UI. The goal is to provide a comprehensive, clear view similar to the internal Assessment Panel. By piping this enhanced data into the widely used Grafana platform, we achieve exactly that.
You get a professional visualization experience with clear metric grouping and modern table panels. This delivers the same quality of performance assessment that internal engineers rely on, using the open-source tools you are already comfortable with.
Safe Sharing (When You Need It)
A critical issue hindering community support is data privacy. You may want to share your diagnostics with a consultant or upload them to a GitHub issue, but doing so often means exposing sensitive network information.
To solve this, I've integrated a separate obfuscation utility into the mongo-ftdc toolkit.
You now have a simple, explicit step to run before sharing the file. This utility masks sensitive data—like hostnames, IPs, and replica set names—while meticulously preserving critical performance specs (hardware, OS, version). This finally makes community debugging sessions and workshops safer and more effective.
Why This Matters
Performance analysis shouldn't be a privilege. It is a fundamental necessity for running a successful MongoDB deployment. By upgrading the mongo-ftdc parser and adding this safe sharing capability, we are removing the friction from debugging and empowering the entire MongoDB community.
If you are running MongoDB at scale, you can now unlock the full value of your diagnostic data.
Try It Now
Optional: Use the obfuscation function for safe sharing.
Comments