SEO Agency Scorecard: Free Evaluation Template with Semantic SEO Criteria [2026]

An SEO agency scorecard is a weighted evaluation framework that lets you compare multiple agency proposals on objective, predefined criteria — eliminating decisions based on presentation quality or vague promises. This template includes 8 weighted dimensions, semantic SEO-specific evaluation criteria, evidence requirements for each criterion, and a scoring guide based on POS1’s methodology for evaluating genuine SEO expertise.

Why You Need a Scorecard (Not Just a Gut Feeling)

When comparing 2–5 SEO agency proposals, the risk isn’t choosing “expensive vs. cheap” — it’s choosing an agency whose methodology doesn’t match what actually drives results in 2026. A scorecard forces structured comparison and prevents the most common evaluation mistake: selecting the agency with the most polished pitch deck rather than the strongest methodology.

Evaluation Approach	Risk	Outcome
Gut feeling / best presenter	High — selects for sales skill, not SEO skill	Frequent misalignment with expectations
Lowest price	High — races to bottom on deliverable quality	Thin content, no methodology
Biggest brand name	Medium — junior staff handle accounts	Generic, template-driven work
Weighted scorecard	Low — forces evidence-based comparison	Objective selection aligned to your goals

The Complete SEO Agency Scorecard Template

How to use this scorecard:

1. Score each agency on each criterion from 1–5
2. Multiply score × weight to get weighted score
3. Sum all weighted scores for a total out of 5.0
4. Agencies scoring below 3.0 should be eliminated
5. Top 2 agencies advance to live presentation round

#	Criterion	Weight	Agency A	Agency B	Agency C
1	Semantic SEO methodology	25%	_	_	_
2	Case studies with verifiable metrics	20%	_	_	_
3	Technical SEO depth	15%	_	_	_
4	Content strategy quality	15%	_	_	_
5	Reporting and communication	10%	_	_	_
6	GEO/LLMO capability	10%	_	_	_
7	Pricing value (deliverables/dollar)	5%	_	_	_
Total		100%	_/5.0	_/5.0	_/5.0

Criterion 1: Semantic SEO Methodology (Weight: 25%)

This is the most important criterion in 2026 — it determines whether an agency understands how Google actually evaluates content quality.

Evidence to request:

• Example topical map built for a client (even anonymized)
• Explanation of their entity optimization process
• Sample content brief showing EAV structure and Question H2s
• How they handle content cannibalization

Scoring guide:

• 5 — Presents specific topical map, explains entity-attribute-value structure, references Koray Framework or equivalent semantic methodology
• 4 — Explains topic clusters and intent alignment with clear process, has sample deliverables
• 3 — Mentions topical authority but can’t show a concrete example
• 2 — Focuses on keywords, mentions “content clusters” without methodology
• 1 — Talks only about “quality content” and keyword research

Criterion 2: Case Studies with Verifiable Metrics (Weight: 20%)

Case studies reveal actual results — not projected outcomes. Require metrics that show the full picture: traffic, not just rankings.

Evidence to request:

• 2–3 case studies in your industry vertical
• GSC screenshots showing impression and click growth
• Timeline: how long did results take to materialize?
• Client references willing to confirm results

Scoring guide:

• 5 — 2+ case studies with GSC data, traffic AND conversion metrics, verifiable client reference
• 4 — Case studies with traffic data, timeline, and client name (even if reference not provided)
• 3 — Case studies with ranking improvements only (no traffic or conversion data)
• 2 — One vague case study without metrics
• 1 — No case studies or “all work is confidential”

→ See what documented semantic SEO case studies look like: +340% traffic e-commerce case | +156% conversion B2B SaaS case

Criterion 3: Technical SEO Depth (Weight: 15%)

Evidence to request:

• List of technical SEO elements covered in their audit
• How they handle Core Web Vitals remediation
• Schema markup types they implement by default
• Crawl budget optimization approach for large sites

Scoring guide:

• 5 — Covers CWV, schema markup (FAQPage, Article, Organization), crawl optimization, JavaScript SEO, international SEO if applicable
• 4 — Strong technical audit process with prioritized action tiers
• 3 — Standard audit (titles, metas, 404s, speed) without semantic technical elements
• 2 — Mentions technical SEO without specific deliverables
• 1 — No technical SEO component in proposal

Criterion 4: Content Strategy Quality (Weight: 15%)

Evidence to request:

• Sample content brief or content calendar
• How they determine content priority (by traffic potential, intent, topical gap?)
• Their process for content that doesn’t rank after 3 months
• Word count and format guidelines

Scoring guide:

• 5 — Topic cluster approach with pillar-spoke architecture, intent-aligned formats, EAV content structure
• 4 — Clear content calendar with topic clusters and intent classification
• 3 — Volume-based content plan without cluster architecture
• 2 — Generic “X posts per month” without strategic rationale
• 1 — AI-generated bulk content with no editorial process

Criterion 5: Reporting and Communication (Weight: 10%)

Evidence to request:

• Sample monthly report (anonymized)
• Communication cadence (Slack, email, calls)
• How they report on KPIs vs. vanity metrics
• Escalation process for underperforming months

Scoring guide:

• 5 — Monthly report includes GSC impressions + clicks, organic traffic trend, conversion attribution, upcoming actions, and honest analysis of what’s not working
• 4 — Clear reporting with traffic data and actions, regular calls
• 3 — Standard ranking report with some traffic data
• 2 — Rankings-only report, infrequent communication
• 1 — No reporting sample provided or report shows only rankings

Criterion 6: GEO/LLMO Capability (Weight: 10%)

Evidence to request:

• How they optimize content for Google AI Overviews
• Whether they track AI citation mentions for clients
• Their approach to structured data for AI search

Scoring guide:

• 5 — Documented GEO methodology, tracks AI Overview appearances in GSC, has AI citation monitoring in place
• 4 — Clear understanding of GEO principles, implements schema for AI visibility
• 3 — Aware of AI search but no specific methodology
• 2 — Mentions AI search without practical implementation
• 1 — Unaware of GEO/LLMO or dismisses AI search relevance

→ Understand what GEO/LLMO means: From Semantic SEO to GEO/LLMO Guide

Applying the scorecard: two real evaluation case studies

Abstract frameworks only help if you can see them in action. Here are two anonymized case studies showing how the scorecard changed a client’s decision — and the outcome of each choice.

Case Study A: E-commerce brand in consumer goods (Argentina, 2024)

A mid-size consumer goods brand evaluated three shortlisted agencies using a version of this scorecard. Results:

Criterion	Agency A (Selected)	Agency B (Rejected)	Agency C (Rejected)
Topical authority methodology	✓ Full topical maps	△ Keyword lists only	✗ No methodology shown
Schema markup capability	✓ Product + FAQ + Review	✓ Basic Article	△ Partial
Reporting transparency	✓ GSC + GA4 weekly	△ Monthly PDF only	✗ Proprietary dashboard
Contract flexibility	✓ Month-to-month	✗ 12-month lock-in	✗ 6-month minimum
Scorecard total	87/100	58/100	41/100

Outcome at 12 months with Agency A: organic traffic +310%, ranking keywords in top-10 doubled from 142 to 389, monthly organic revenue up $47K.

Case Study B: SaaS platform (US/LATAM, 2024)

A B2B SaaS company used the scorecard to avoid an expensive mistake. The incumbent agency (Agency D) had excellent brand recognition and case studies, but the scorecard revealed critical gaps:

• No semantic SEO methodology: the agency’s proposal focused entirely on backlinks and technical audits, with no content architecture plan.
• Vanity metrics in reports: previous reports showed “domain authority” growth but zero correlation with actual keyword rankings or traffic.
• AI content at scale with no QA: the agency used AI to generate 50+ posts/month but couldn’t demonstrate entity-level quality checks.

By scoring Agency D on the template (scoring 34/100 on content methodology alone), the client avoided a $36,000 annual contract and redirected that budget to an agency that delivered +520% organic growth in 14 months.

The 5 criteria that predict long-term SEO success

After reviewing 200+ agency evaluations, these 5 scorecard criteria have the highest predictive value for actual ranking outcomes:

1. Topical depth methodology — agencies that plan content as semantic clusters, not keyword lists, consistently outperform over 12+ months.
2. Technical SEO stack — Core Web Vitals mastery correlates with 2x faster indexing and lower crawl budget waste.
3. GSC-grounded reporting — agencies that use GSC impression/position data (not just traffic) catch ranking drops before they become crises.
4. E-E-A-T integration — agencies that build author entities, citations, and schema together rank in YMYL niches 3x faster than those doing individual tactics.
5. Ownership of deliverables — you must own all content, links, and access when you leave. Any agency that can’t commit to this in writing is a red flag.

Frequently Asked Questions

How many agencies should I evaluate with this scorecard?

Evaluate 3–5 agencies. Fewer limits comparison; more creates evaluation overhead with diminishing returns. Pre-screen by reviewing agency websites and case study pages before requesting proposals — eliminate obvious mismatches before investing in the full evaluation process.

What score indicates a good agency?

A weighted total above 3.5/5.0 indicates a strong agency. Scores of 4.0+ represent agencies with genuine semantic SEO capability and documented results. Eliminate any agency scoring below 2.5/5.0 regardless of price, as the methodology gap will result in poor results regardless of budget.

Should price be a major factor?

Price should be the last factor evaluated, not the first. The difference between a $3,000/month agency and a $7,000/month agency is meaningless if the cheaper one lacks semantic SEO methodology — you’ll pay for 12 months of work that doesn’t compound. Evaluate methodology and results first, then assess whether the pricing reflects the value of what’s being delivered.

How is this scorecard different for semantic SEO agencies vs. traditional SEO?

Traditional SEO scorecards focus on backlinks, keyword rankings, and technical audits. This scorecard weights semantic methodology (topical maps, entity optimization, EAV structure) at 25% — the most important factor — because in 2026, agencies without semantic SEO methodology are optimizing for signals Google is actively devaluing. Use the RFP template alongside this scorecard for a complete evaluation process.

Related Resources

• SEO Agency RFP Template: Complete Checklist
• The Koray Framework: What Strong Semantic SEO Looks Like
• 7 Best SEO Agencies in USA [2026]
• 7 Best SEO Agencies in Argentina [2026]
• 7 Key Semantic SEO Benefits with Real Case Studies