Find broken pagination and malformed internal links with Screaming Frog

A pagination link can look correct on the page and still send visitors to the wrong URL. An extra slash may trigger a redirect. The redirect may then end on a 404. You need to follow the request all the way to its destination to understand what is broken.

Another problem is an empty value inside a generated link. A template expects a coin name, product slug, or category ID, but the value is missing. The page still prints the link, and the same invalid URL can appear across many pages.

Both examples appear in my Screaming Frog course session on pagination and internal links. This article follows the investigation from the crawl result to the HTML that generated it. The paths below are simplified examples for practice.

Start with a crawl that can find the links

Open Screaming Frog in Spider mode, enter a starting URL, and run the crawl. For this investigation, start from a page that leads to the affected category or listing. The crawler needs a route to the source pages before it can collect their links.

Check the user agent, crawl restrictions, and speed before starting. If you are working on a business website, agree on the request rate with the team responsible for the server. A faster crawl is not useful if it causes server errors or gets the testing IP blocked.

Record the settings with your results. If the next crawl uses a different starting URL, user agent, or rendering mode, a change in the result may come from the test conditions.

This is one part of a technical SEO audit: reproduce the issue and identify where it enters the website’s navigation.

Check redirects and their final responses

Open the Response Codes tab. Review the Client Error (4xx) results for missing destinations, and the Redirection (3xx) results for URLs that send the request somewhere else.

A redirect is not automatically a mistake. For example, an old URL may intentionally lead to its replacement. The question here is whether the page is printing the intended destination and whether that destination works.

In the pagination example from the course, the printed URL contained an extra slash. Following the link exposed a redirect, and the destination returned a 404. Stopping at the first response would leave part of the problem unexplained.

The query-string version in the table is a route used in the practice example. Your website may use a different format. Removing a slash is only a fix if the resulting URL is the correct destination.

For redirects ending in errors, use Reports > Redirects > All Redirects and inspect the final status and address. Screaming Frog’s redirect report documentation explains the report types and the limits of incomplete crawls. A partial or blocked crawl may leave some responses unavailable.

Find the source using Inlinks

Select the affected URL in the upper results panel, then open Inlinks in the lower panel.

The From column identifies the page containing the link. The To column identifies its destination. Review the anchor text too; it can help you recognise the relevant control or content item when you open the source page.

1. Select a missing URL or an unexpected redirect.
2. Open Inlinks and choose a source page.
3. Note the destination and anchor text.
4. Open the source page in the browser.
5. Find the link and compare its actual URL with the crawl result.

If the source is itself a redirect, keep tracing until you reach the page that printed the original link. Screaming Frog’s broken-link investigation guide covers this case and the source/destination fields.

One invalid destination can have many inlinks. That does not necessarily mean many separate mistakes were made. A navigation component, recommendation block, or listing template may be printing the same bad link everywhere.

Export the source information when you need to work through a larger group. The Bulk Export > Response Codes > Client Error (4XX) Inlinks export is useful for keeping source pages and missing destinations together.

Locate the link in the HTML

A source page may contain hundreds of links. Searching its markup is usually easier than clicking through the whole page.

Open Inspect, select the Elements panel, and search for a distinctive part of the bad path. In the empty-slug example, search for /coins//. Check the element around the match so you can identify the item that generated it.

<!-- Simplified example: the coin slug is missing. -->
<a href="/coins//">Nano</a>

The visible label is present, but the destination is incomplete. That is why looking only at the displayed page can miss the problem.

For pagination, inspect the page-number link directly. A simplified example looks like this:

<!-- Incorrect destination in this practice example. -->
<a href="/catalogue//page/2/">2</a>

<!-- Working page-two route in this practice example. -->
<a href="/catalogue/?page=2">2</a>

Check the generated href, including its path, parameters, and slashes. The text “2” tells you which control you are inspecting; it does not tell you whether the destination is correct.

A control that works when clicked can still need a crawlability check. Google’s link guidance recommends anchors with usable href destinations. If the page relies on script-driven controls, a JavaScript rendering investigation can help establish which links are actually available.

Fix the shared template or data problem

After finding the element, check how its URL is built. Is the template adding a slash to a value that already ends with one? Is the item missing a slug? Does the pagination component use a different route from the one the server supports?

These are questions for the code and data that produce the link. The rendered HTML shows the output, but it does not identify the cause by itself.

For the missing coin slug, compare several items using the same component. If one record lacks a slug, correct that record and decide what the component should do when required data is missing. If every item prints the wrong path, inspect the shared URL-building rule.

A useful development ticket includes:

• A source page and the exact bad link it prints.
• The observed response, including any redirect and its final destination.
• The intended working URL.
• The page type or shared component affected.
• Steps to reproduce the result and check the replacement.

This approach is useful for website architecture and programmatic SEO, where URLs are often generated from shared rules. It also applies to classifieds and marketplace SEO: category pages, listings, and recommendation components can all depend on data that changes.

If the intended content has moved, a redirect may still be appropriate. Update the internal link to its intended current destination where possible, and preserve any redirect needed for old incoming URLs. Sending an invalid pagination link to the homepage would not restore access to page two.

Check that pagination still leads through the collection

A 200 response is the start of the recheck. Open the destination and confirm that it contains the expected results. A link labelled “2” should not silently show the same items as page one.

Review the next-page path from the first page, then continue through another page in the sequence. Check the mobile version as well if it uses a different component or output.

Google’s pagination guidance recommends distinct URLs for the pages and crawlable links between them. It also recommends a canonical URL for each page instead of pointing every page in the sequence at page one.

Keep those checks separate in your notes. A correctly formed link can point to the wrong content, and a working page can still carry an unintended canonical setting. When the remaining problem is discovery or index eligibility, it belongs in the crawl and indexing analysis.

Recheck the links after the fix

Start with the source pages you recorded. Confirm that they now print the intended URLs. Then request the destinations and review their responses.

Recheck the affected group using comparable crawl settings. Where browser caching or another cache layer is involved, make sure the test is seeing the updated output. Keep the original export so the before and after results can be compared.

• The source links contain the intended destination.
• The destination returns the expected response and content.
• The original malformed path is no longer printed by the affected component.
• Pagination continues through the collection.
• Other items using the shared component still work.
• The relevant mobile and rendered versions have been checked.

Record what changed and which URLs were tested. If the malformed URL still appears in a later crawl, return to its inlinks. Another component may be generating it, or a source page may still be serving an older version.