Re-Crawling After Fixes
This lesson covers
- Why re-crawling is the first verification step
- When to use a full crawl or a list crawl
- How to keep settings comparable
- When to re-crawl after deployment
Short answer
Re-crawl after fixes using the same saved configuration as the original audit, so results are comparable. Use a list crawl of affected URLs for a quick check of specific fixes, and a full crawl to catch side effects across the site. Re-crawl soon after deployment, after caches clear, and record the date in the issue register.
The developer says, “Done, the canonicals are fixed.” That is the start of verification, not the end. Module 12 is about proving fixes work and making sure nothing else broke.
Two kinds of re-crawl
| Crawl | Use it to | When |
|---|---|---|
| List crawl | Check specific affected URLs from a finding | Right after a fix is deployed |
| Full crawl | Catch side effects and new issues sitewide | After major releases or each roadmap phase |
Keep it comparable
Load the saved configuration from Lesson 2.2: same user agent, rendering mode, speed, limits, exclusions, and API connections. Changing settings creates fake differences.
Timing
- Confirm the change is on the live site, not just staging.
- Ask for cache and CDN purges if needed.
- Run a list crawl of affected URLs.
- Schedule a full crawl at the end of the phase.
- Record the crawl date in the register.
Practice: list crawl or full crawl?
Sort each item into the right group.
-
Checking 48 room pages after a canonical fix
Answer: List crawl.
-
After a theme update across the whole site
Answer: Full crawl.
-
Confirming 120 redirects now go in one hop
Answer: List crawl.
-
End of the 'Next' roadmap phase
Answer: Full crawl.
-
After migrating to a new hosting provider
Answer: Full crawl.
Common mistakes
- New settings for the re-crawl.
- Crawling before caches clear.
- Only list crawls, so side effects go unseen.
Your action
Next, you will compare before-and-after crawl results.
Key terms
- Verification crawl
- A crawl run after fixes to confirm they work and nothing else broke.
- List crawl
- A crawl of a specific uploaded URL list rather than following links from a seed.
- Saved configuration
- The crawl settings file from the original audit, reused for comparison.
- Cache
- Stored copies of pages on servers or CDNs that can show old versions after a fix.
Questions beginners ask
How soon after deployment should I re-crawl?
Within a day, after purging caches. Re-crawl again a week or two later for a full check.
Why must settings match?
Different user agents, rendering, or limits change results. Matching settings makes differences meaningful.
Do I need a full crawl every time?
No. List crawls verify specific fixes quickly. Run a full crawl after major releases or at the end of each phase.
What if the fix seems missing?
Check caches, CDN, the deployed environment, and whether the change went to staging only.
Related lessons
Want someone to handle SEO for your business while you learn? The team at Remar Digital can help.