MongoDB Diagnostics with AI: The Simagix Toolset and NotebookLM

The "Virtual DBA" Revolution

For years, the Simagix toolset has evolved to peel back the layers of MongoDB clusters, from log parsing to hardware metrics. However, even with the best diagnostic tools, a critical gap remained: correlation time. Manually cross-referencing a CPU spike in an FTDC file against a slow query pattern in a log file still required significant DBA effort and intuition.

I used the 2025 holiday break to close this gap. By combining the deep-dive analytics of the Simagix toolset with the reasoning power of NotebookLM, we have entered the era of the "Virtual DBA." We no longer just collect data; we use AI to synthesize it into a 360-degree view of cluster health in minutes.

The Simagix Arsenal

Our toolset is designed to bridge the gap between low-level database internals and high-level AI reasoning.

Tool	Role	Function
keyhole	The Collector	Snapshots configuration and statistics with built-in obfuscation.
maobi	The Visualizer	Provides deep-dive reports on cluster health and schema performance.
hatchet	The Log Parser	Audits MongoDB logs to identify slow query patterns and COLLSCANs.
mongo-ftdc	The Metric Decoder	Decodes binary metrics to reveal hardware and WiredTiger bottlenecks.

The Workflow: From Raw Data to AI Insights

This integrated workflow automates the transition from raw database artifacts to a searchable, AI-ready knowledge base. By following these three phases, you ensure your diagnostic data is properly captured, synthesized, and formatted for high-precision analysis.

Environment Preparation

Establish a secure and consistent environment by defining your connection string as a variable. This prevents sensitive credentials from being saved in your shell history and ensures that all subsequent commands use a unified source.

export MONGO_URI="mongodb+srv://user:password@host.example.mongodb.net/"

Note on Directory Structure: Before running the analysis pipeline, ensure your raw data is organized in your current working directory as follows:

• mongodb/: Place your system logs (mongod.log) here for hatchet.
• diagnostic.data/: Place your binary metrics (metrics.2025...) here for mongo-ftdc.

The Analysis Pipeline

Execute the core Simagix toolset to transform raw database artifacts into structured reports. This phase utilizes four specialized tools to capture the full spectrum of database health, bridging the gap between high-level statistics and granular hardware metrics.

• Step 1: Capture Cluster Stats (Keyhole) 

docker run --rm -v $(pwd):/home/simagix simagix/keyhole \     /keyhole -allinfo ${MONGO_URI} -obfuscate

• Step 2: Generate Visual Reports (Maobi) Transforms the compressed BSON statistics from Keyhole into a multi-dimensional diagnostic report. This step provides a deep dive into cluster configuration, collection-level statistics, and schema health. Beyond simple reporting, Maobi generates automated performance insights—highlighting storage fragmentation, identifying oversized documents, and flagging inefficient index-to-data ratios that impact memory overhead.

docker run --rm -v $(pwd):/home/simagix simagix/maobi \     /maobi {keyhole -allinfo output filename}

• Step 3: Parse Performance Logs (Hatchet) Analyzes raw MongoDB logs to reconstruct a high-definition view of database activity. Hatchet generates a comprehensive audit of operation types, identifies recurring slow query patterns, and ranks the "Top N" slowest operations. This allows you to pinpoint exactly which application queries are driving resource contention and identify unindexed operations (COLLSCANs) before they impact cluster stability.

docker run --rm -v $(pwd):/home/simagix simagix/hatchet \     /bin/hatchet -report mongodb/

• Step 4: Decode Hardware Metrics (mongo-ftdc) Decodes binary Full Time Diagnostic Data Capture (FTDC) files—the "black box" recorder of MongoDB. This tool translates raw metrics into human-readable data, revealing critical system-level bottlenecks such as CPU saturation, disk I/O wait, and WiredTiger cache pressure. By extracting these granular metrics, you can visualize the exact moment hardware constraints began to throttle database performance, providing the necessary evidence to correlate system stress with slow queries found in Hatchet.

docker run --rm -v $(pwd):/home/simagix simagix/ftdc \   /mftdc diagnostic.data/

NotebookLM Optimization: Convert to Markdown

Prepare your diagnostic data for AI consumption by converting HTML outputs into LLM-friendly Markdown. This final conversion is required because NotebookLM does not natively support HTML. Markdown is the ideal format for LLMs as it preserves the structural hierarchy of tables and headers, which is essential for achieving high reasoning accuracy and clean formatting in your final Google Doc reports.

Convert HTML to Markdown:

cd html for f in *.html; do     echo "converting $f"     pandoc "$f" -f html -t markdown -o "${f%.html}.md" done cd ..

Data Hygiene: Once the .md files are uploaded to NotebookLM, delete the local artifacts to keep your workspace clean.

rm -rf html/*.html *.bson.gz

Insights with NotebookLM

Once you’ve uploaded the Markdown files, NotebookLM synthesizes them into a unified "Source of Truth." It bridges the gap between raw logs and hardware metrics, allowing the AI to act as an investigative partner that correlates events across your specific diagnostic window.

To begin your investigation, use natural language to interrogate the data. These high-value prompts are designed to uncover deep architectural insights:

Scenario 1: Executive Health Summary

This prompt directs the AI to synthesize all available data into a high-level, actionable report for stakeholders:

Act as a Senior MongoDB DBA. Based on all uploaded reports, identify the single biggest bottleneck (e.g., storage waste or unindexed scans) and provide three prioritized, actionable recommendations for the engineering team.

Scenario 2: Performance & Correlation Forensics

This prompt forces the AI to cross-reference application-level latency (Hatchet) with system-level bottlenecks (mongo-ftdc):

Identify the top 5 most expensive queries in the Hatchet logs. Cross-reference these with the FTDC metrics: did we see a corresponding drop in CPU Idle time or a spike in WiredTiger cache pressure during these exact execution windows?

Scenario 3: Index & Schema Strategy

This prompt uses structural data (Maobi) to provide context for inefficient query patterns (Hatchet):

Based on the Maobi stats, identify collections with fragmentation above 50%. Suggest missing indexes identified in Hatchet that would eliminate the COLLSCANs currently impacting these specific collections.

References & Resources

The Simagix Toolset

Access the open-source tools used in this diagnostic pipeline. Each repository contains detailed documentation and Docker deployment guides.

• keyhole: MongoDB performance analysis and data collection tool.

• maobi: MongoDB BSON statistics and schema visualizer.

• hatchet: High-performance MongoDB log analyzer.

• mongo-ftdc: Binary FTDC decoder for hardware and WiredTiger metrics.

AI Analysis

• NotebookLM: Google’s AI-powered research and analysis tool used for synthesizing the Markdown diagnostic reports.

• Pandoc: The universal document converter used to transform HTML reports into AI-optimized Markdown.