Spider mode explores a linked structure from a starting point. List mode checks a supplied set of URLs under its configured limits. They can overlap in results while answering different questions.
If you want to know whether page two is reachable from a category, a linked crawl is useful. If you want to recheck 200 URLs from a migration mapping, a controlled list may be more appropriate.
Use Spider mode for linked discovery
Start with a page that leads to the area being investigated. The crawler follows the permitted links and records relationships between discovered resources.
This can help identify incoming links to a broken address. It can also show that a collection was not reached under the current configuration. Before calling that an orphan-page finding, check restrictions, rendering, authentication, and whether the starting point could reach the section.
The first-crawl guide covers general setup. Keep those settings comparable when repeating a crawl after a fix.
Use List mode for a known set
Switch to List mode and provide the URLs using the available upload or paste options. A text file with one absolute URL per line is an easy input to retain and review.
https://example.com/category/
https://example.com/category/?page=2
https://example.com/products/example-item/
You can also obtain URLs from a sitemap using the tool’s supported import options. Confirm that a sitemap index has been expanded into the intended child URL sets. Merely recording the index address is not equivalent to checking every page it lists.
List-mode behaviour can be extended through configuration. Check link-following limits and related settings before assuming that the output contains only the supplied addresses or every possible linked destination.
Compare what each source can establish
A supplied URL returning 200 establishes its response in that test. It does not establish that visitors can navigate to it from the homepage. A linked crawl finding a page establishes a discovered route under that configuration, not a guarantee that Google indexed it.
A sitemap list is useful for the intended publication set. An Analytics or Search Console export may contain important URLs absent from the current navigation. A developer’s route list may include pages that should never be indexed.
Reconcile the datasets
Normalise carefully when comparing addresses. Decide how to handle fragments, trailing slashes, protocol, and parameters without collapsing URLs that actually return different content.
Review URLs found only in the supplied list. Open representative examples and search for their intended incoming links. Then inspect pages found only in the linked crawl: they may be valid, excluded from the sitemap intentionally, or generated by unwanted navigation patterns.
For extracting sitemap URLs with code, use the existing WordPress sitemap Python tutorial. For broken sources, continue with Inlinks.
My crawl and indexing analysis service covers these dataset comparisons. During a website migration, retaining the source mapping and final responses makes the redirect verification reviewable.