Building an AI-Powered Test Generator on Top of Azure DevOps MCP + Claude Code

[PakAbhishek]

What I Built

I've been using the Azure DevOps MCP server extensively with Claude Code to build Smart Test Generator — a tool that uses AI to
automatically generate, review, and manage test cases by pulling context directly from Azure DevOps work items, repositories, and
existing test plans.

I noticed the Show and tell category could use more real-world usage stories, and I think enterprise examples help the team and
community understand where this technology shines and where it has friction. So here's mine.

How It Works

The core idea: instead of an engineer manually reading a user story, checking the code, looking at existing tests, and then writing new
test cases — the AI agent does all of that through MCP in a single automated loop.

The Five-Stage Pipeline

1. Requirements Context
The agent pulls the full picture of what needs to be tested using wit_get_work_item, wit_get_work_items_batch_by_ids, and
search_workitem. It reads acceptance criteria, linked work items, and parent/child relationships to understand scope — not just the
single story, but the feature it belongs to.

2. Code Context
Using repo_list_directory, search_code, and the PR tools (repo_get_pull_request_by_id, repo_list_pull_request_threads), the
agent understands how the feature was implemented. PR review comments are especially valuable — they surface edge cases and concerns
that reviewers already identified.

3. Existing Test Intelligence
Before generating anything, the agent reads what's already covered via testplan_list_test_plans, testplan_list_test_suites, and
testplan_list_test_cases. This is critical. Without this step, the AI generates redundant tests that duplicate existing coverage. With
it, the agent focuses on gaps.

4. Test Generation & Authoring
Informed by all three context sources, the agent generates test cases and writes them back via testplan_create_test_case,
testplan_add_test_cases_to_suite, and testplan_update_test_case_steps. The generated tests reference specific implementation details
from the code and trace directly to acceptance criteria from the work item.

5. Traceability
wit_link_work_item_to_pull_request and wit_add_artifact_link tie generated test cases back to their source requirements and code
changes. Every generated test has a clear audit trail — which requirement drove it, which code change it covers.

Which MCP Tools I Use Most

I exercise a significant portion of the tool surface across most of the nine domains. The ones I lean on hardest:

Domain	Tools	What For
Work Items	wit_get_work_item, wit_get_work_items_batch_by_ids, search_workitem, wit_add_artifact_link,
wit_link_work_item_to_pull_request	Pull requirements context, link generated tests back to source items
Repos	search_code, repo_list_directory, repo_get_pull_request_by_id, repo_list_pull_request_threads	Understand
implementation details and review feedback
Test Plans	testplan_list_test_cases, testplan_create_test_case, testplan_add_test_cases_to_suite,
testplan_update_test_case_steps	Read existing coverage, write new test cases, manage steps
Search	search_code, search_workitem	Cross-cutting queries to find related items across the organization
Pipelines	pipelines_get_builds, testplan_show_test_results_from_build_id	Check pass/fail history to inform test priorities

|

What Works Exceptionally Well

The unified surface area is the killer feature. Having work items, repos, test plans, and pipelines all accessible through a single
MCP server means the AI agent can build a complete picture of a feature without the user manually bridging between systems. Most MCP
servers connect to a single service. This one exposes an entire SDLC platform across nine domains, and that makes qualitatively
different workflows possible.

Claude Code as a client is seamless. The authentication flow, tool discovery, and execution work reliably. I run both the local
server and the remote MCP server (public preview) depending on the scenario.

The test plan tools are underrated. I read the team's Playwright success story and my experience confirms it — the test plan domain
is where MCP + AI agents create the most tangible, measurable value. Being able to read existing test cases, identify gaps, and
write new ones in a single agent loop is a genuine step change from the manual process.

Friction Points & Feature Requests

In the spirit of constructive feedback — these are the walls I've hit. Happy to file each as an individual issue if that's the preferred
workflow.

1. No bulk test case creation

When generating a suite of 15–20 test cases from a feature, I call testplan_create_test_case + testplan_add_test_cases_to_suite
sequentially for each one. A batch creation tool — similar to how wit_update_work_items_batch handles work items — would significantly
reduce round trips, improve reliability, and make the agent loop faster.

2. Shared step references in test case authoring

testplan_update_test_case_steps works for basic steps, but there's no way to reference existing shared steps programmatically. For
test generation, shared steps are critical — you don't want the AI recreating "Log in as admin" or "Navigate to the dashboard" in every
test case. Even read-only access to shared step definitions would help the agent recognize and reuse them.

3. Test results write-back

testplan_show_test_results_from_build_id lets me read results, but there's no tool to write test results or update test point
outcomes. Closing this gap would enable fully automated test execution reporting through MCP — run the tests, then record the outcomes,
all in one agent loop.

4. Direct WIQL execution

I often need dynamic queries like "all user stories in this sprint without linked test cases." The wit_get_query and
wit_get_query_results_by_id tools work great with saved queries, but a direct WIQL execution tool that accepts a query string would be
powerful for agent-driven ad hoc analysis without requiring pre-saved queries in the project.

5. Attachment metadata

Some test cases reference screenshots or documents attached to work items. There's currently no tool to read or list attachments. Even
read-only access to attachment metadata (file name, type, size) would help the AI understand the full context of a work item.

Beyond Test Generation

The test generation use case is what I built first, but the same MCP foundation enables:

• Impact analysis — "This PR changed these files. Which test cases cover this area? Which ones need updating?"
• Requirements coverage — "Which user stories in this sprint have no linked test cases?"
• Review context enrichment — Pulling PR threads and work item history to give code reviewers the full requirements and testing
  context in one place

Thank You

Thank you to @danhellem and the maintainer team for building and maintaining this. The Azure DevOps MCP server has become a core part of
my development workflow, and I genuinely believe the enterprise SDLC integration angle — not just code, but requirements, tests,
pipelines, and project management in one MCP surface — is what sets this apart in the MCP ecosystem.

Happy to answer questions, go deeper on any of this, or collaborate on any of the feature areas I mentioned.