Identifying 5xx Server Errors
In this lesson
- What the common 5xx codes mean
- How 5xx errors affect Googlebot's crawling
- How to tell real server errors from crawl-caused ones
- What evidence developers need to fix them
Short answer
A 5xx error means the server failed to deliver a page. Find them in your crawl's response codes and in Search Console's Crawl Stats and Pages reports. Recheck each one at a slower crawl speed, because your own crawl can cause them, then give developers the URLs, times, and patterns.
During a 12.12 sale, a Davao gadget store’s product pages started returning 503 errors. The server could not handle the shoppers plus Googlebot. Sales stopped, and so did crawling. Server errors hurt people and search engines at the same time.
The codes you will meet
- 500: something broke in the site’s code, plugin, or database.
- 502: a CDN or proxy got a bad response from the origin server.
- 503: the server is overloaded or in maintenance.
- 504: the origin server took too long to answer the proxy.
Where to find them
- Your crawl: Response Codes, filter Server Error (5xx).
- Search Console Pages report: “Server error (5xx)” lists URLs Google could not fetch.
- Search Console Crawl Stats: check Host status and the breakdown by response. Spikes show when Googlebot hit problems.
- Server logs: confirm whether Googlebot itself received the errors.
Real errors or crawl-caused?
Your crawl can create the errors you report. Before reporting, recheck every 5xx URL in list mode at 1 URL per second.
- Still failing: likely a real error. Report it.
- Now returns 200: probably load or rate limiting. Note it as a server capacity risk, not a broken page.
- Fails only for some user agents: a firewall or bot protection rule may be blocking crawlers, including Googlebot.
What developers need
Give them URLs, time stamps, the user agent used, the response code, and any pattern (one template, one folder, only under load). Include the Search Console Crawl Stats chart if it shows spikes.
Practice: what does this suggest?
Sort each item into the right group.
-
Every /checkout/ URL returns 500 even at slow speed
Answer: Real site or code problem.
-
503s appeared at 10 URLs per second, gone at 1 per second
Answer: Crawl speed or firewall issue.
-
Search Console shows the same 500s for Googlebot
Answer: Real site or code problem.
-
403 and 503 only when using a crawler user agent
Answer: Crawl speed or firewall issue.
-
Blog pages return 502 after a CDN change
Answer: Real site or code problem.
Common mistakes
- Reporting 5xx errors without rechecking. Developers lose trust in the audit.
- Ignoring Crawl Stats. It shows what Googlebot experienced over 90 days.
- Treating a short maintenance 503 as a crisis. Duration and frequency matter.
Your action
Next, you will audit redirect chains and loops.
Key terms
- 500 Internal Server Error
- A general server failure, often caused by code bugs, plugin conflicts, or database problems.
- 502 Bad Gateway
- A server or proxy received an invalid response from another server behind it, common with CDNs and load balancers.
- 504 Gateway Timeout
- A server behind a proxy took too long to respond.
- Host status
- A section of Search Console Crawl Stats showing whether Google had problems reaching your server, robots.txt, or DNS.
Questions beginners ask
How do 5xx errors affect SEO?
Google slows its crawling when it sees many server errors. If important pages keep returning 5xx for days, Google may drop them from the index until they recover.
Is a 503 during maintenance safe?
Yes, for short periods. A 503 tells Google to come back later. Long or repeated 503s over several days can still cause pages to drop.
Why do I see 5xx errors only in my crawl?
Your crawl may be too fast for the server, or a firewall may be rate limiting you. Recrawl those URLs slowly before reporting them.
Who fixes 5xx errors?
Usually the developer or hosting provider. Your job is to supply clear evidence: affected URLs, time stamps, patterns, and how to reproduce the error.
Related lessons
Want someone to handle SEO for your business while you learn? The team at Remar Digital can help.