The Screaming Frog SEO Log File Analyser allows you to upload your log files, identify crawled URLs and analyse search bot data and behaviour for invaluable SEO insight.
View and analyse exactly which URLs Googlebot & other search bots are able to crawl, when and how frequently.
Get insight to which search bots crawl most frequently, how many URLs are crawled each day and the total number of bot events.
Discover all response codes, broken links and errors that search engine bots have encountered while crawling your site.
Find temporary and permanent redirects encountered by search bots, that might be different to those in a browser or simulated crawl.
Analyse your most and least crawled URLs & directories of the site, to identify waste and improve crawl efficiency.
Review the average bytes downloaded & time taken to identify large pages or performance issues.
Import a list of URLs and match against log file data, to identify orphan or unknown pages or URLs which Googlebot hasn’t crawled.
Import and match any data with a ‘URLs’ column against log file data. So import crawls, directives, or external link data for advanced analysis.