Server Log Analysis in WordPress [Guide 2021]

This guide discusses how to do server log analysis, directly from the WordPress Dashboard - using the IndexJet plugin.

How To Do Server Log Analysis with IndexJet

Server log analysis used to be something only very experienced SEO's worked on.

With IndexJet's ability to track Googlebot on your WordPress website, now anyone can get started with this important SEO technique.

How Often To Check Log Files?

The larger the website, the more often you want to check.

For a 50 page site: 1 check per month

For a 1000 page site: 2 checks per month

For a 100k page site: Check each week at minimum

In general, crawl budget is only an issue if your website has between 50k to 100k+ pages.

For any websites with 500 or even a 1000 pages it's not a problem.

Find Large Pages that Google could not crawl due to size

Issue: Crawl budget can be wasted on these large pages

Find pages or types of pages that Google did not crawl

IndexJet Workflow:

  1. Go to Main Dashboard
  2. Filter for Crawl Frequency 0 (zero)

Issue: Google can't access certain sections of your website i.e. domain.com/shoes/red/ladies/

Solution: Point internal links to these deeper pages, create specific sitemaps for these pages

Find 404 Errors that Googlebot landed on:

IndexJet Workflow:

Crawl Optimizer -> Filter for 404

Issue: You are sending Google bot into a page where no useful content is

Fix: Create redirects to relevant pages

Find 5xx Errors that Googlebot landed on:

IndexJet Workflow:

Crawl Optimizer -> Filter for 5xx

Issue: Google is getting an error response from server - Googlebot then slows down crawling/discovering your website

Fix: Create redirects to relevant pages

Find Redirects that Googlebot landed on:

IndexJet Workflow:

Crawl Optimizer -> Filter for 3xx status

Issue: Google landing on HTTP -> HTTPs or WWW -> NON-WWW

Fix: Reduce the amount of redirects to not waste crawl budgets. Fix this by links pointing to your website

all using HTTPs for example