Screaming Frog requests pages and resources, follows links under the configured rules, and records what it finds. The output is a dataset from those conditions. It is not a complete list of Google’s decisions about the website.
Before starting, decide what you need to learn. A first crawl might investigate a category’s navigation and responses, rather than trying to collect every possible field from the entire site.
Set the crawl conditions
Install the current SEO Spider from the official publisher and check the limits of your edition. Licensing affects crawl size and available features; confirm the current quick-start guidance rather than assuming every option shown in an older recording is available.
Greenflare is mentioned as an alternative in the course. If using a different crawler, verify its current maintenance, platform support, and capabilities. The important skill is understanding the requests and output.
Record the starting URL, user agent, robots handling, rendering mode, inclusion rules, and rate. Agree the load with the team responsible for the site when necessary.
Start in Spider mode
Choose Spider mode and enter the starting URL. The crawler discovers further resources through the configured link-following behaviour. Its scope determines which hosts and paths it explores.
Watch the early responses before leaving it running. If the start page returns an access challenge or redirects to another host, the resulting crawl may not represent the intended section.
The progress percentage can change as more URLs are discovered. It is not a forecast of the total page count known in advance. Pause or stop when needed, and save the crawl before clearing or replacing the results.
Separate documents from supporting resources
Review Internal results and content types. HTML pages, images, scripts, and stylesheets are different resources. External URLs may be recorded without being explored in the same way as the configured internal scope.
Check Response Codes for redirects and errors. For an affected address, inspect its details and Inlinks. A missing destination becomes actionable when you can identify where the site links to it.
Review standard fields by page purpose
Titles, meta descriptions, canonicals, headings, word counts, and image alt fields can reveal patterns. Sort and filter them, then open representative pages.
A missing alt value on a decorative image needs different interpretation from a product image that conveys information. A low word count can be normal for a tool or concise reference page. Duplicate titles may indicate a shared template problem, but inspect the content before prescribing a fix.
Turn one row into a finding
Choose a suspicious result, reproduce it in the browser, locate the generating markup, and identify its scope. Export the useful evidence rather than forwarding every warning unchanged.
Continue with Spider versus List mode, crawl speed, storage, or custom extraction when that specific question arises. The broken pagination example demonstrates the full investigation from response to source template.
My technical SEO audit service covers crawl setup and interpretation. For teams building repeatable checks, SEO automation starts with a well-defined input and reliable output.