Observing Fat Clients From the Inside: MCP Server for Java Desktop Apps
Everyone's talking about using AI to rewrite legacy systems. The server side is easy—functions with inputs and outputs. AI can read, trace, and reason about that with near-zero cost.
The hard part is the fat client.
Years of implicit state machines, event handler chains, undocumented UI behaviors that users depend on, rendering quirks that became features, interaction patterns that exist in no spec because they emerged from a decade of incremental patches.
The tribal knowledge doesn't live in code comments or Confluence. It lives in runtime behavior. Which fields disable when a combo selection changes. What validation fires on tab-out. What happens when someone pastes 10,000 order rows from Excel.
You can't rewrite what you can't observe.
Existing accessibility APIs (Java Access Bridge, UI Automation, AT-SPI) were designed for screen readers, not for AI agents. Wrong serialization, wrong transport, wrong granularity.
What's missing is a way to observe a running fat client from the inside: structured state, live component trees, user interaction recordings, all served as flat JSON over HTTP so an AI agent can consume it directly.
I needed this for Java Swing. It didn't exist. So I built it.
🔗 GitHub Repository:
https://github.com/your-username/java...
⚡ What java-swing-mcp Does:
Embeds an HTTP server inside any Swing application. One call—SwingMcpServer.start(9222)—and the live UI is exposed as structured JSON on localhost.
AI agents can:
✓ Read component state in real-time
✓ Execute actions programmatically
✓ Capture screenshots
✓ Record user interactions as reproducible scripts
✓ Run WCAG 2.1 contrast audits
Port 9222 is intentional—same default as Chrome DevTools Protocol.
🎯 Use Cases:
Legacy desktop application modernization
AI-assisted rewrite planning
Runtime behavior documentation
Accessibility auditing
Automated testing for fat clients
Understanding undocumented UI logic
💡 The Pattern Works Beyond Swing:
The implementation is Swing. The pattern works for any fat client where you have source access—JavaFX, WPF, Electron, Qt, you name it.
📚 Additional Resources:
Full documentation in the repo
Demo videos showing real-world usage
Model Context Protocol: https://modelcontextprotocol.io
#Java #LegacyModernization #AI #FatClient #DevTools #EnterpriseJava #MCP #AIAgents