freecourses.digital
Lesson progress: 0 of 3 activities done
All lessons in Technical SEO Auditing Lesson 12.2 of 84

Comparing Before-and-After Crawl Results

Inside this lesson

  • How crawl comparison works
  • Which changes to look at first
  • How to read added, removed, and changed URLs
  • How to summarize the comparison for the report

Short answer

Comparing crawls means placing the baseline audit crawl beside the post-fix crawl to see what changed. Use your crawler's compare mode or match exports by URL, then review issue counts, added and removed URLs, and changed values like status codes, canonicals, and indexability. Improvements confirm fixes, while unexpected changes reveal regressions.

The clearest proof that an audit worked is a simple before-and-after table. The crawl comparison builds it for you.

How to compare

With compare mode: In Screaming Frog, switch to Compare mode, select the baseline and the new crawl, and run Crawl Analysis. Each tab shows Added, New, Removed, and Missing per issue.

Without compare mode: export both, match by URL, and add “changed” columns for status, indexability, canonical, title, and meta robots.

What to look at first

  1. Issue counts for every finding in your register.
  2. Indexability changes: pages newly indexable or newly non-indexable.
  3. Status code changes: new 4xx, 5xx, or redirects.
  4. Added URLs: unexpected new patterns.
  5. Removed URLs: pages that disappeared from the crawl, possibly orphaned.

Summarize it

FindingBeforeAfterStatus
IDX-03 Noindex on location pages400Fixed
RED-02 Links to 301s in footer1,2000Fixed
CAN-02 Canonical to HTTP486Partly fixed
NEW Non-200 canonicals on blog085New issue

Practice: improvement or regression?

Sort each item into the right group.

  • Noindexed location pages: 40 to 0

  • New 404s: 0 to 230

  • Redirect chains: 900 to 12

  • Indexable blog pages: 400 to 60

  • Duplicate titles on products: 1,200 to 0

Common mistakes

  • Looking only at fixed issues.
  • Ignoring removed URLs.
  • Comparing crawls with different settings.
1. What does a baseline crawl provide?
2. Indexable pages dropped sharply after fixes. What is it?
3. Without compare mode, how do you compare?

Your action

Next, you will verify redirects, canonicals, and indexability in detail.

Key terms

Baseline crawl
The original audit crawl used as the reference point.
Compare mode
A crawler feature that shows differences between two crawls, such as Screaming Frog's Crawl Comparison.
Added and removed URLs
URLs present in only one of the two crawls.
Changed URLs
URLs in both crawls with different values, such as a new status code or canonical.

Questions beginners ask

Can I compare without compare mode?

Yes. Export both crawls and match by URL with XLOOKUP, then flag differences in key columns.

What if URL structures changed?

Use URL mapping in compare mode, or a redirect map, so old URLs line up with new ones.

Are added URLs always bad?

No. New content is expected. Watch for unexpected patterns, like thousands of new parameter URLs.

How do I present the comparison?

A short table of issue counts before and after, plus notes on any new issues.

Related lessons

Want someone to handle SEO for your business while you learn? The team at Remar Digital can help.