Explicit Criteria for Precision

Why vague instructions like 'be conservative' fail to improve precision, and how to replace them with explicit categorical criteria that dramatically reduce false positives in code review, data extraction, and CI/CD pipelines.

Quick Reference

→Explicit criteria = concrete rules that define exactly what constitutes a finding
→Vague instructions ('be conservative', 'only report high-confidence findings') do NOT improve precision
→High false-positive rates erode trust even when accurate categories exist
→Specific categorical criteria outperform confidence-based filtering every time
→Define severity levels with concrete code examples, not abstract descriptions
→Temporarily disable high-false-positive categories rather than adding 'be careful' instructions
→Each criterion should be independently verifiable by reading the code
→Exam Scenario 5 (CI/CD): precision is critical when findings block a merge
→Exam Scenario 6 (Structured Data Extraction): explicit field-level criteria prevent hallucinated data

Why Precision Matters More Than Recall

In production systems, a false positive is often more damaging than a missed finding. When a CI/CD pipeline flags 15 'issues' on every PR and 12 of them are noise, developers stop reading the output entirely. The 3 real issues get lost. This is the precision problem: once trust erodes, even accurate findings get ignored.

Exam trap: 'be conservative' does NOT work

A common exam scenario presents a code review system with high false positives and asks which fix is best. Adding general instructions like 'be conservative' or 'only report high-confidence findings' is always the WRONG answer. Claude cannot reliably self-assess confidence. The correct answer is always to provide explicit categorical criteria.

The core insight is this: Claude is excellent at following specific rules but unreliable at self-assessing its own confidence. When you say 'only report high-confidence findings,' you are asking the model to do something it fundamentally cannot do well. When you say 'flag comments only when the claimed behavior contradicts the actual code behavior,' you are giving it a concrete, verifiable check.

Vague vs. Explicit: The Critical Difference

Category	Vague (fails)	Explicit (works)
Comment accuracy	Check that comments are accurate	Flag a comment ONLY when the claimed behavior directly contradicts the actual code behavior (e.g., comment says 'returns null' but code throws an exception)
Security issues	Look for security vulnerabilities	Flag SQL queries that concatenate user input strings. Ignore parameterized queries even if the variable names look suspicious.
Performance	Identify performance problems	Flag loops that make network calls where the call count scales with input size. Ignore single network calls regardless of payload size.
Dead code	Remove unused code	Flag functions with zero call sites in the repository. Do NOT flag functions exported from public API modules, even if no internal callers exist.
Type safety	Improve type safety	Flag 'any' type assertions that suppress compiler errors on object property access. Ignore 'any' used in test fixtures or type guard functions.
Error handling	Improve error handling	Flag empty catch blocks that swallow errors silently. Ignore catch blocks that log the error before continuing.

Before and After: Code Review Prompt

Let's see the full transformation from a vague prompt to an explicit one. This example is a code review system prompt for a CI/CD pipeline (Exam Scenario 5).

Sign in to read this article

This is a premium article. Sign in with your Google account to continue.