This guide discusses how to do server log analysis, directly from the WordPress Dashboard - using the IndexJet plugin.
How To Do Server Log Analysis with IndexJet
Server log analysis used to be something only very experienced SEO's worked on.
With IndexJet's ability to track Googlebot on your WordPress website, now anyone can get started with this important SEO technique.
How Often To Check Log Files?
The larger the website, the more often you want to check.
For a 50 page site: 1 check per month
For a 1000 page site: 2 checks per month
For a 100k page site: Check each week at minimum
In general, crawl budget is only an issue if your website has between 50k to 100k+ pages.
For any websites with 500 or even a 1000 pages it's not a problem.
Find Large Pages that Google could not crawl due to size
Issue: Crawl budget can be wasted on these large pages
Find pages or types of pages that Google did not crawl
IndexJet Workflow:
- Go to Main Dashboard
- Filter for Crawl Frequency 0 (zero)
Issue: Google can't access certain sections of your website i.e. domain.com/shoes/red/ladies/
Solution: Point internal links to these deeper pages, create specific sitemaps for these pages
Find 404 Errors that Googlebot landed on:
IndexJet Workflow:
Crawl Optimizer -> Filter for 404
Issue: You are sending Google bot into a page where no useful content is
Fix: Create redirects to relevant pages
Find 5xx Errors that Googlebot landed on:
IndexJet Workflow:
Crawl Optimizer -> Filter for 5xx
Issue: Google is getting an error response from server - Googlebot then slows down crawling/discovering your website
Fix: Create redirects to relevant pages
Find Redirects that Googlebot landed on:
IndexJet Workflow:
Crawl Optimizer -> Filter for 3xx status
Issue: Google landing on HTTP -> HTTPs or WWW -> NON-WWW
Fix: Reduce the amount of redirects to not waste crawl budgets. Fix this by links pointing to your website
all using HTTPs for example