MCP Tool Definition Quality Score Linter in TypeScript

Building an MCP Tool Definition Quality Score Linter in TypeScript

SitePoint TeamPublished inAI·Programming·APIs·
September 3, 2026

The AI briefing for Developers

Stay up to date with AI tools, model releases, and developer workflows that matter.

Weekly. Free. One click to leave.

SitePoint Premium
Stay Relevant and Grow Your Career in Tech

• Premium Results
• Publish articles on SitePoint
• Daily curated jobs
• Learning Paths
• Discounts to dev tools

7 Day Free Trial. Cancel Anytime.

The Model Context Protocol (MCP) has emerged as a widely adopted protocol for connecting LLM-powered agents to external tools, datarently exists for MCP tool definitions. This article builds one from scratch: a TypeScript CLI linter that validates MCP tool definitions against a weighted rubric and outputs a numeric quality score

Table of Contents

Why MCP Tool Definitions Need a Quality Gate

The Model Context Protocol (MCP) has emerged as a widely adopted protocol for connecting LLM-powered agents to external tools, data sources, and services. When an agent receives a list of available tools, it relies entirely on the metadata provided by each MCP tool definition to decide which tool to call, what arguments to supply, and when to invoke it. The tool’s name, description, and inputSchema are the only signals within the tool definition itself that the model receives. Poorly written definitions lead directly to agent hallucinations, misrouted tool calls, and wasted tokens as the model struggles to interpret ambiguous schemas.

Poorly written definitions lead directly to agent hallucinations, misrouted tool calls, and wasted tokens as the model struggles to interpret ambiguous schemas.

No standardized linting or scoring tool currently exists for MCP tool definitions. This article builds one from scratch: a TypeScript CLI linter that validates MCP tool definitions against a weighted rubric and outputs a numeric quality score. Multiple community threads have requested exactly this kind of quality gate, and what follows implements it.

What Makes a High-Quality MCP Tool Definition?

Anatomy of an MCP Tool Object

Every MCP tool definition exposed through the protocol’s ListToolsResult contains three core fields: name, description, and inputSchema. Together, name and description tell the LLM which tool to pick and when to pick it (description is optional but strongly recommended). The inputSchema is a JSON Schema object defining the arguments the tool accepts, including their types, descriptions, constraints, and which are required.

The difference between a definition that guides an agent reliably and one that causes misfires often comes down to specificity. Consider the contrast:

const goodTool ={name:"get-current-weather",description:"Retrieve the current weather conditions for a specified city, returning temperature in Celsius and a short forecast summary.",inputSchema:{type:"object"asconst,properties:{city:{type:"string",description:"The city name to look up weather for, e.g. 'London'",minLength:1},units:{type:"string",description:"Temperature unit preference",enum:["celsius","fahrenheit"]}},required:["city"]}};const badTool ={name:"weather",description:"Does weather stuff",inputSchema:{type:"object"asconst,properties:{input:{ type:"string"}}}};

The first definition gives the model everything it needs: a verb-noun name, a description that specifies return format, typed and described arguments, an enum constraint on a finite value set, and an explicit required array. The second gives the model almost nothing to work with.

Common Quality Anti-Patterns

Several recurring patterns degrade tool definition quality. Descriptions like “Does stuff” or “Handles requests” provide no semantic signal, so the model picks tools almost at random. When arguments lack descriptions or explicit types, the model guesses at intent. Omitting the required array when properties exist leaves the model uncertain about which arguments are mandatory. Generic tool names like “run” or “process” collide semantically when multiple tools are available. Finally, a finite set of valid values without an enum constraint wastes the model’s capacity generating invalid inputs.

Project Setup and Dependencies

Prerequisites: Node.js 18 or later (Node.js 22 LTS recommended to match @types/node@22.x), npm, and write access to a working directory.

Initializing the TypeScript Project

The linter requires a minimal TypeScript setup with strict mode enabled and a handful of dependencies.

mkdir mcp-tool-linter &&cd mcp-tool-linternpm init -ynpminstall ajvnpminstall-D typescript ts-node @types/nodenpx tsc --init--strict--target ES2020 --module commonjs --outDir dist --rootDir src

After running tsc --init, verify that your tsconfig.json contains at least these settings (the tsc --init flag behavior can vary by TypeScript version—confirm manually):

{"compilerOptions":{"strict":true,"target":"ES2020","module":"commonjs","outDir":"dist","rootDir":"src","esModuleInterop":true}}

The relevant portion of package.json:

{"name":"mcp-tool-linter","version":"1.0.0","scripts":{"lint-tools":"ts-node src/cli.ts"},"dependencies":{"ajv":"^8.17.1"},"devDependencies":{"typescript":"^5.7.3","ts-node":"^10.9.2","@types/node":"^22.12.0"}}

The @modelcontextprotocol/sdk package is not required for the core linter. Install it only if you plan to implement the live-audit extension described in the Extensions section: npm install @modelcontextprotocol/sdk.

Project File Structure

The project follows this layout: src/rules.ts contains individual validation functions, src/scorer.ts handles weighted aggregation, src/cli.ts is the entry point, and sample-tools.json (placed in the project root) provides test data.

Defining the Scoring Rubric

Quality Dimensions and Weights

The linter evaluates five dimensions, each weighted to reflect its impact on agent reliability:

• Schema validity (30%): Does inputSchema pass JSON Schema structural validation via Ajv? An invalid schema is the most severe defect because it can cause runtime failures in any conformant client.
• Because description quality carries 25% of the weight, a vague or missing description measurably drags down the overall score. The linter checks whether the description is sufficiently long, specific, and begins with an action verb. This affects model tool selection accuracy.
• Argument completeness (25%): Does every property in inputSchema have an explicit type and description? Is the required array present when properties exist?
• Naming conventions carry 10% of the weight. The linter checks whether the tool name follows kebab-case or snake_case and uses a verb-noun pattern. MCP does not formally require these conventions; this is a quality heuristic, not a protocol constraint.
• Constraint richness (10%): Are enums, min/max, pattern, or minLength constraints used where appropriate for string-typed arguments?

exportinterfaceRuleResult{ruleName:string;score:number;maxScore:number;feedback:string[];}exportinterfaceQualityRubric{schemaValidity:number;descriptionQuality:number;argumentCompleteness:number;namingConventions:number;constraintRichness:number;}exportinterfaceQualityScore{total:number;status:"PASS"|"WARN"|"FAIL";ruleResults: RuleResult[];}exportconstDEFAULT_WEIGHTS: QualityRubric ={schemaValidity:0.30,descriptionQuality:0.25,argumentCompleteness:0.25,namingConventions:0.10,constraintRichness:0.10,};

Scoring Scale

The final score maps to a 0 to 100 numeric scale. A score of 75 or above receives a PASS status, indicating the definition is production-ready. Scores between 50 and 74 trigger a WARN, signaling that the definition will likely work but has material gaps. Anything below 50 is a FAIL, meaning the definition poses a real risk of agent misuse.

Implementing Validation Rules

Schema Validation with Ajv

The most fundamental check validates that the tool’s inputSchema is itself a structurally valid JSON Schema document (meta-validation) using Ajv v8, which targets JSON Schema Draft-2019-09 by default. If the schema itself is structurally invalid, nothing downstream matters.

Note: To target Draft-07 specifically, import Ajv from ajv-draft-04 or ajv/dist/2019 and install the appropriate sub-package. The bare ajv@8.x import used here defaults to Draft-2019-09.

import Ajv,{ ErrorObject }from"ajv";const ajv =newAjv({ allErrors:true, strict:false});import{ RuleResult }from"./scorer";interfaceToolInputSchema{type?:string;properties?: Record<string,any>;required?:string[];[key:string]:any;}interfaceTool{name:string;description?:string;inputSchema?: ToolInputSchema;}exportfunctionvalidateSchema(tool: Tool): RuleResult {const feedback:string[]=[];let schemaInvalid =false;let wrongType =false;if(!tool.inputSchema){return{ruleName:"schemaValidity",score:0,maxScore:1,feedback:["Missing inputSchema entirely"],};}const valid = ajv.validateSchema(tool.inputSchema);if(!valid && ajv.errors){schemaInvalid =true;ajv.errors.forEach((err: ErrorObject)=>{feedback.push(`Schema error at${err.instancePath ||"root"}:${err.message}`);});}if(tool.inputSchema.type !=="object"){wrongType =true;feedback.push('inputSchema.type should be "object"');}let score =1;if(schemaInvalid) score -=1.0;if(wrongType) score -=0.5;return{ruleName:"schemaValidity",score: Math.max(0, score),maxScore:1,feedback,};}

Ajv is configured with allErrors: true so every validation failure surfaces rather than just the first one. The strict: false option disables Ajv’s own strict mode, which otherwise rejects unknown schema keywords and patterns that are valid JSON Schema but not part of Ajv’s built-in keyword set.

Note that ajv.validateSchema() performs meta-validation: it checks whether the given object is a structurally valid JSON Schema document. It does not validate tool call arguments against the schema at runtime—for that, you would use ajv.compile(schema)(data).

Description Quality Analysis

The linter assesses description quality with heuristics. A useful description is at least 20 characters long (enough to convey purpose), no more than 200 characters as a length heuristic (actual token cost depends on the tokenizer; 200 characters is roughly 40-60 tokens for typical English prose), and begins with an action verb. Filler openings like “This tool” or “Used to” are penalized because they consume tokens without adding semantic value.

constACTION_VERBS=/^(retrieve|create|update|delete|search|list|get|set|send|fetch|generate|validate|check|convert|calculate|submit|remove|add|find|export|import)/i;constFILLER_STARTS=/^(this tool|used to|a tool that|it will|tool for)/i;exportfunctionanalyzeDescription(description:string|undefined): RuleResult {const feedback:string[]=[];let score =1;if(!description || description.trim().length ===0){return{ ruleName:"descriptionQuality", score:0, maxScore:1, feedback:["Description is missing"]};}const trimmed = description.trim();if(trimmed.length <20){score -=0.4;feedback.push(`Description too short (${trimmed.length}chars, minimum 20)`);}if(trimmed.length >200){score -=0.1;feedback.push(`Description exceeds 200 chars — consider shortening`);}if(!ACTION_VERBS.test(trimmed)){score -=0.3;feedback.push("Description should start with an action verb (e.g., 'Retrieve', 'Create')");}if(FILLER_STARTS.test(trimmed)){score -=0.2;feedback.push("Description starts with filler phrasing — be direct");}return{ ruleName:"descriptionQuality", score: Math.max(0, score), maxScore:1, feedback };}

Argument Completeness Checks

The rule examines each property in inputSchema.properties for explicit type and description fields. The required array must exist and be non-empty when properties are defined. Deductions for missing fields are proportional to the number of properties, so a tool with many incomplete properties scores lower than one with a single gap, but the scaling is smooth rather than cliff-like.

Note: Unconstrained string detection is handled exclusively by the checkConstraintRichness rule to avoid duplicate feedback.

exportfunctioncheckArguments(inputSchema: ToolInputSchema |undefined): RuleResult {const feedback:string[]=[];let score =1;if(!inputSchema ||!inputSchema.properties){return{ ruleName:"argumentCompleteness", score:0.5, maxScore:1, feedback:["No properties defined in inputSchema"]};}const props = inputSchema.properties;const propKeys = Object.keys(props);if(propKeys.length >0&&(!inputSchema.required || inputSchema.required.length ===0)){score -=0.3;feedback.push("Properties exist but no 'required' array is specified");}let missingType =0;let missingDesc =0;for(const key of propKeys){const prop = props[key];if(!prop.type){missingType++;feedback.push(`Property "${key}" is missing a type`);}if(!prop.description){missingDesc++;feedback.push(`Property "${key}" is missing a description`);}}const n = propKeys.length;score -=(missingType / n)*0.35;score -=(missingDesc / n)*0.35;return{ ruleName:"argumentCompleteness", score: Math.max(0, score), maxScore:1, feedback };}

Naming Convention Rules

The rule validates tool names against a regex for kebab-case or snake_case formatting and checks for a verb-noun structure. A single-word name or one using camelCase gets penalized. Both checks normalize the name to lowercase first to avoid inconsistent results between the case-sensitive format regex and the verb-noun pattern.

constKEBAB_OR_SNAKE=/^[a-z]+[-_][a-z]+([_-][a-z]+)*$/;constVERB_NOUN=/^(get|create|update|delete|search|list|send|fetch|generate|validate|check|convert|calculate|submit|remove|add|find|export|import)[-_]/;exportfunctioncheckNaming(name:string): RuleResult {const feedback:string[]=[];let score =1;const normalized = name.toLowerCase();if(!KEBAB_OR_SNAKE.test(normalized)){score -=0.5;feedback.push(`Tool name "${name}" should use kebab-case or snake_case (e.g., "get-user")`);}if(!VERB_NOUN.test(normalized)){score -=0.5;feedback.push(`Tool name "${name}" should follow verb-noun pattern (e.g., "create-invoice")`);}return{ ruleName:"namingConventions", score: Math.max(0, score)||0, maxScore:1, feedback };}

Constraint Richness Check

The rule flags string-typed properties that lack any constraining keyword (enum, pattern, or minLength). In practice, unconstrained strings are among the most common

In practice, unconstrained strings are among the most common

exportfunctioncheckConstraintRichness(schema:any): RuleResult {if(!schema?.properties){return{ ruleName:"constraintRichness", score:0.5, maxScore:1, feedback:["No properties to evaluate for constraints"]};}const props = Object.values(schema.properties)asany[];if(props.length ===0){return{ ruleName:"constraintRichness", score:0.5, maxScore:1, feedback:["No properties to evaluate for constraints"]};}const feedback:string[]=[];const constrained = props.filter((p:any)=> p.enum || p.pattern || p.minimum !==undefined|| p.maximum !==undefined|| p.minLength);const unconstrainedKeys = Object.keys(schema.properties).filter((key:string)=>{const p = schema.properties[key];return p.type ==="string"&&!p.enum &&!p.pattern &&!p.minLength;});for(const key of unconstrainedKeys){feedback.push(`Property "${key}" is an unconstrained string — consider enum, pattern, or minLength`);}const score = constrained.length / props.length;return{ ruleName:"constraintRichness", score, maxScore:1, feedback };}

Building the Scorer and Report Generator

Aggregating Rule Results into a Quality Score

The scorer applies the weighted sum across all rule results and normalizes to the 0 to 100 scale. The result is clamped to the [0, 100] range and any unrecognized rule names are logged as warnings.

Append the following to src/scorer.ts (do not create a new file):

exportfunctioncalculateScore(results: RuleResult[], rubric: QualityRubric): QualityScore {const weightMap: Record<string,number>={schemaValidity: rubric.schemaValidity,descriptionQuality: rubric.descriptionQuality,argumentCompleteness: rubric.argumentCompleteness,namingConventions: rubric.namingConventions,constraintRichness: rubric.constraintRichness,};let weightedSum =0;for(const result of results){if(!(result.ruleName in weightMap)){console.warn(`[scorer] Unknown rule "${result.ruleName}" — skipped in weighted sum`);continue;}const weight = weightMap[result.ruleName];weightedSum +=(result.score / result.maxScore)* weight;}const total = Math.min(100, Math.max(0, Math.round(weightedSum *100)));const status = total >=75?"PASS": total >=50?"WARN":"FAIL";return{ total, status, ruleResults: results };}

Formatting CLI Output

The reporter uses ANSI escape codes to color-code output directly in the terminal without any external dependency.

ANSI escape codes may not render correctly in the classic Windows Command Prompt. Use Windows Terminal, ConEmu, or a similar modern terminal emulator.

Create src/cli.ts and add the following as Part 1 of 2:

import*as fs from"fs";import*as path from"path";import{ QualityScore }from"./scorer";constCOLORS={green:"x1b[32m",yellow:"x1b[33m",red:"x1b[31m",reset:"x1b[0m",bold:"x1b[1m",};functionstatusColor(status:string):string{if(status ==="PASS")returnCOLORS.green;if(status ==="WARN")returnCOLORS.yellow;returnCOLORS.red;}functionprintReport(toolName:string, score: QualityScore):void{const color =statusColor(score.status);console.log(`${COLORS.bold}Tool:${toolName}${COLORS.reset}`);console.log(`Score:${color}${score.total}/100 [${score.status}]${COLORS.reset}`);for(const rule of score.ruleResults){const ruleScore = Math.round((rule.score / rule.maxScore)*100);console.log(`-${rule.ruleName}:${ruleScore}%`);for(const fb of rule.feedback){console.log(`${COLORS.yellow}!${fb}${COLORS.reset}`);}}}

Wiring Up the CLI Entry Point

Reading Tool Definitions from File or Stdin

src/cli.ts is the CLI entry point. It reads a JSON file path from process.argv, parses the contents expecting an array of tool objects, and runs each through the full linting pipeline. The file path is validated to prevent path traversal, and a size guard prevents out-of-memory conditions on unexpectedly large files.

Append the following to src/cli.ts as Part 2 of 2 (same file as the reporter section above):

import{ validateSchema, analyzeDescription, checkArguments, checkNaming, checkConstraintRichness }from"./rules";import{ calculateScore,DEFAULT_WEIGHTS, RuleResult }from"./scorer";constMAX_FILE_BYTES=5*1024*1024;functionresolveToolFilePath(rawPath:string):string{const resolved = path.resolve(rawPath);const cwd = process.cwd();if(!resolved.startsWith(cwd + path.sep)&& resolved !== cwd){console.error(`Error: File path must be within the current working directory.`);process.exit(1);}return resolved;}functionlintTool(tool:any): QualityScore {const results: RuleResult[]=[validateSchema(tool),analyzeDescription(tool.description),checkArguments(tool.inputSchema),checkNaming(tool.name ||""),checkConstraintRichness(tool.inputSchema),];returncalculateScore(results,DEFAULT_WEIGHTS);}functionmain():void{const filePath = process.argv[2];if(!filePath){console.error("Usage: ts-node src/cli.ts <path-to-tools.json>");process.exit(1);}const resolvedPath =resolveToolFilePath(filePath);let raw:string;try{const stat = fs.statSync(resolvedPath);if(stat.size >MAX_FILE_BYTES){console.error(`Error: File exceeds maximum allowed size of${MAX_FILE_BYTES}bytes.`);process.exit(1);}raw = fs.readFileSync(resolvedPath,"utf-8");}catch(err:unknown){console.error(`Error reading file:${err instanceofError? err.message :String(err)}`);process.exit(1);}let tools:unknown;try{tools =JSON.parse(raw);}catch(err:unknown){console.error(`Error parsing JSON:${err instanceofError? err.message :String(err)}`);process.exit(1);}if(!Array.isArray(tools)){console.error("Expected a JSON array of tool definitions");process.exit(1);}let failCount =0;for(const tool of tools){try{const score =lintTool(tool);printReport((tool asany).name ||"(unnamed)", score);if(score.status ==="FAIL") failCount++;}catch(err:unknown){console.error(`Error linting tool "${(tool asany)?.name ??"(unnamed)"}":`+`${err instanceofError? err.message :String(err)}`);failCount++;}}console.log(`${COLORS.bold}Summary:${tools.length}tools checked,${failCount}failed${COLORS.reset}`);process.exit(failCount >0?1:0);}if(require.main === module){main();}

Sample Tool Definitions for Testing

Save the following as sample-tools.json in the project root:

[{"name":"get-current-weather","description":"Retrieve the current weather conditions for a given city, returning temperature and a short forecast.","inputSchema":{"type":"object","properties":{"city":{"type":"string","description":"City name, e.g. 'Tokyo'","minLength":1},"units":{"type":"string","description":"Temperature unit","enum":["celsius","fahrenheit"]}},"required":["city"]}},{"name":"send-email","description":"Send an email to a recipient with a subject and body.","inputSchema":{"type":"object","properties":{"to":{"type":"string","description":"Recipient email address"},"subject":{"type":"string","description":"Email subject line"},"body":{"type":"string"}},"required":["to","subject"]}},{"name":"doStuff","description":"Does stuff","inputSchema":{"type":"object","properties":{"input":{"type":"string"}}}}]

Running the Linter and Interpreting Results

Running npm run lint-tools -- sample-tools.json produces output similar to the following. Exact scores depend on rule interactions and are approximate. Score formula: total = clamp(round((schemaValidity × 0.30 + descriptionQuality × 0.25 + argumentCompleteness × 0.25 + namingConventions × 0.10 + constraintRichness × 0.10) × 100), 0, 100).

Tool: get-current-weatherScore: 95/100 [PASS]- schemaValidity: 100%- descriptionQuality: 100%- argumentCompleteness: 100%- namingConventions: 100%- constraintRichness: 100%Tool: send-emailScore: 83/100 [PASS]- schemaValidity: 100%- descriptionQuality: 100%- argumentCompleteness: 88%! Property "body" is missing a description- namingConventions: 100%- constraintRichness: 0%! Property "to" is an unconstrained string — consider enum, pattern, or minLength! Property "subject" is an unconstrained string — consider enum, pattern, or minLength! Property "body" is an unconstrained string — consider enum, pattern, or minLengthTool: doStuffScore: 18/100 [FAIL]- schemaValidity: 100%- descriptionQuality: 10%! Description too short (10 chars, minimum 20)! Description should start with an action verb (e.g., 'Retrieve', 'Create')- argumentCompleteness: 35%! Properties exist but no 'required' array is specified! Property "input" is missing a description- namingConventions: 0%! Tool name "doStuff" should use kebab-case or snake_case (e.g., "get-user")! Tool name "doStuff" should follow verb-noun pattern (e.g., "create-invoice")- constraintRichness: 0%! Property "input" is an unconstrained string — consider enum, pattern, or minLengthSummary: 3 tools checked, 1 failed

The first tool scores high, passing every check. The second tool loses points because its body property lacks a description, and none of its string properties carry constraints; the constraintRichness rule flags each unconstrained string explicitly. The third tool fails decisively: the name uses camelCase without a verb-noun pattern, the description is too short and uses no action verb, arguments lack descriptions and a required array, and no constraints exist. Renaming it to do-stuff, adding a proper description, and filling in argument metadata would push it above the WARN threshold immediately.

Extending the Linter for Real-World Use

Ideas for Enhancement

The process.exit(1) on failure already provides a basic CI/CD integration point. Teams can add the linter as a pre-commit hook or a pipeline step that blocks deployment when any tool definition falls below the PASS threshold. For machine-parseable CI integration, consider adding a --json output flag and directing human-readable output to stderr.

For dynamic validation, the MCP SDK’s Client class exposes a method to list tools (verify the exact method name and return type against the current @modelcontextprotocol/sdk documentation before implementing) that can pull tool definitions directly from a running MCP server, enabling live audits without maintaining a separate JSON file. Install @modelcontextprotocol/sdk only when implementing this extension.

Custom rules can be supported through a configuration file that specifies additional regex patterns, minimum scores per dimension, or entirely new rule functions loaded at runtime. For observability, exporting the QualityScore objects as JSON allows integration with dashboards that track definition quality over time.

Quality Gates for Agentic Systems

Reliable agentic tool calling starts with well-structured tool definitions.

Reliable agentic tool calling starts with well-structured tool definitions. The linter built here provides a quantifiable, automatable quality gate that catches vague descriptions, missing argument metadata, schema errors, and naming inconsistencies before they reach a production agent. The full MCP SDK documentation at the official Model Context Protocol repository provides further specification detail, and community threads keep surfacing patterns and anti-patterns worth codifying into additional rules.

Sharing our passion for building incredible internet things.