Broken links are a routine annoyance on small websites. On large WordPress sites, they are a systemic infrastructure problem.
A 404 on a WooCommerce product page during a traffic spike costs real revenue. A broken internal link on a high-authority post wastes the link equity that took months to build. An unresolved redirect chain after a domain migration quietly bleeds crawl budget while your rankings drop and nobody can explain why.
This post covers how broken links accumulate on larger sites, how to detect them without damaging server performance, how to fix them at scale using WP-CLI, and how to build a process that prevents them from coming back.
Why Broken Links Accumulate Faster Than You Expect
On a site with 20 pages, broken links are easy to spot and fix manually. On a site with 500 posts, an active content team, a WooCommerce catalog, and a recent redesign behind it, broken links are almost guaranteed to be spreading in the background without anyone noticing.
The most common causes are structural, not careless. Content gets deleted without a redirect. A permalink structure changes during a platform upgrade. An HTTPS migration leaves HTTP links buried in post content. A plugin update changes the slug of a custom post type. A product is discontinued and its URL removed from the catalog. Each of these events creates broken links across every post that ever linked to that URL.
Multi-author environments make this worse. When writers, editors, and developers all have the ability to publish, move, or delete content, no single person has a complete picture of what links where. The result is a slow accumulation of dead ends that no individual is responsible for fixing.
What Broken Links Actually Cost
The cost goes beyond a bad user experience. On a site that depends on organic traffic, broken links create compounding SEO damage that takes time to recover from.
Wasted Crawl Budget
Google allocates a fixed crawl budget to each site based on its authority and server responsiveness. When Googlebot follows a link and hits a 404, it has spent part of that budget on a dead end. On a site with hundreds of broken links, a meaningful percentage of every crawl goes toward discovering errors rather than indexing new or updated content. This is why pages on large sites sometimes take weeks to appear in search results even after being published.
Lost Link Equity
Internal links distribute authority across your site. When a post links to a page that no longer exists, that link equity stops moving. On sites that have built up a strong internal linking structure, a wave of deletions without redirects can quietly reduce the authority of their most important pages without any obvious trigger to investigate.
External broken links are less damaging to SEO but still hurt credibility. When your content points readers to a resource that returns 404, it signals that the content has not been maintained.
Direct Revenue Impact
For WooCommerce stores, broken product or category links are a direct revenue problem. A user who clicks a broken link from a blog post or an email campaign does not go looking for an alternative on your site. They leave. If the broken link is in a paid ad or a promotional email, the cost is immediate and measurable.
Choosing the Right Detection Approach for Your Site
Not every tool is appropriate for every site. The wrong choice either misses problems or creates new ones by overloading the server during a scan.
Google Search Console (Always On, Always First)
Google Search Console is the baseline. It reports 404 errors that Googlebot has actually encountered while crawling your site. Under Indexing, check Pages and filter for “Not Found (404)” to see which URLs are returning errors and which pages are linking to them. This is passive detection that costs nothing and has no server impact. The limitation is that it only shows you what Google has crawled recently, so newly broken links may take time to appear.
Screaming Frog (For Serious Site Audits)
Screaming Frog is a desktop crawler that works the way Googlebot does. You give it your site URL, it follows every link it finds, and it reports all response codes including 404, 301, 302, and 410. The free plan handles up to 500 URLs. The paid license removes that limit and adds features like JavaScript rendering and scheduled crawls.
For a comprehensive audit, run Screaming Frog during off-peak hours. The crawler makes HTTP requests to every URL it finds, which can stress a server that is already near capacity. On managed hosting platforms like WPVIP or Kinsta, this is rarely a problem. On shared hosting or underpowered VPS setups, schedule the crawl at night.
The Broken Link Checker Plugin: Use With Caution
The Broken Link Checker plugin by WPMU DEV is the most commonly recommended tool for WordPress, and it works well on small to medium sites. However, its default behavior of continuous background checking creates real performance problems on high-traffic sites. The plugin uses WP-Cron to queue link checks, which runs on every page load. On a site with thousands of posts and active traffic, this background process competes with serving real visitors.
The cloud mode added in recent versions mitigates this by offloading scanning to WPMU DEV servers rather than running locally. If you use this plugin, switch to cloud mode and disable continuous monitoring in favor of scheduled scans.
For enterprise sites or WooCommerce stores with significant traffic, external tools like Screaming Frog or Ahrefs Site Audit are more appropriate than any server-side plugin.
Ahrefs Site Audit (For Link Equity-Aware Auditing)
Ahrefs Site Audit goes beyond basic 404 detection. It reports broken links alongside their estimated link equity, redirect chains, and pages that have lost backlinks because the destination page was deleted. If you need to understand which broken links are actually costing you in terms of search authority, Ahrefs gives you that picture in a way that Google Search Console and Screaming Frog alone cannot.
The WP-CLI Approach: Fixing Broken Links at Scale
For engineers with server access, WP-CLI is the fastest and most reliable way to fix large numbers of broken links, especially after a migration. It handles serialized data correctly, does not require installing temporary plugins, and runs far faster than any admin interface.
Bulk URL Replacement After a Migration
The most common enterprise broken link scenario is a domain change or HTTP-to-HTTPS migration that leaves old URLs hardcoded throughout the database. WP-CLI search-replace handles this correctly, including the serialized data that breaks when you run raw SQL UPDATE statements.
# Always run with --dry-run first to preview what will change
wp search-replace 'http://old-domain.com' 'https://new-domain.com' --all-tables --dry-run
# Execute when ready
wp search-replace 'http://old-domain.com' 'https://new-domain.com' --all-tablesThe --all-tables flag ensures the replacement runs across every database table, not just the default WordPress tables. This matters on sites with page builders or plugins that store URLs in custom tables. Run the dry run first and review the output. For large databases, add the --verbose flag to see which tables are being updated.
Auditing Post Content for Known Broken Patterns
When you know a specific URL pattern is broken, you can query the database directly via WP-CLI to find which posts contain it before deciding how to fix each instance.
# Search post content for a specific broken URL
wp post list --post_status=publish --fields=ID,post_title
--post_type=post,page
--search='old-broken-url'
--format=tableThis gives you a list of every post that references the broken URL, which you can then either fix via search-replace or update individually through the editor.
Bulk Redirect Setup via WP-CLI
If you use the Redirection plugin, you can import a CSV of redirects rather than creating them one by one in the admin. For very large redirect sets, creating them programmatically via a custom WP-CLI command or a direct database insert into the wp_redirection_items table is more reliable than the plugin interface at scale.
For teams managing redirects on WPVIP or Pantheon, platform-level redirect configuration in vip-config or pantheon.yml is preferred over plugin-based redirects, as it runs at the server level before WordPress loads and carries no PHP overhead.
The Most Common Cause: Post-Migration Link Rot
Most enterprise broken link crises happen after a migration. Domain changes, permalink structure updates, HTTPS migrations, and CMS-to-CMS migrations all introduce broken links at scale if the URL changes are not handled systematically before go-live.
The reliable pre-migration approach is to crawl the current site with Screaming Frog, export all URLs, map them to their post-migration equivalents, and build a redirect list before switching DNS. Running WP-CLI search-replace immediately after migration to update hardcoded URLs in the database covers the internal link layer. The redirect list covers incoming backlinks and external references that point to old URLs.
After going live, run Google Search Console daily for the first two weeks and monitor for new 404 errors. Fix each one with a 301 redirect as soon as it appears. This two-week window is when most migration-related broken links surface.
Redirect Chains and the Hidden Crawl Cost
A redirect chain occurs when one URL redirects to another URL that itself redirects again before reaching the final destination. For example: /old-page redirects to /interim-page which redirects to /new-page. Both users and Googlebot follow the full chain on each request.
Redirect chains build up over time on sites that have gone through multiple redesigns. Each migration adds another layer. The performance cost is real: each redirect in the chain adds an HTTP round trip, increasing page load time for users and consuming more crawl budget per URL for Googlebot.
Screaming Frog flags redirect chains automatically. Ahrefs Site Audit calls them out in its crawl issues report. When you find them, collapse the chain: update the original redirect to point directly to the final destination, removing the intermediate hops entirely. This is particularly important for any URL that receives significant backlink equity, since redirect chains reduce the amount of link equity that passes through.
How to Fix Broken Links: The Decision Framework
Every broken link falls into one of three categories, and the right fix depends on which category it belongs to.
Update the Link
When the target content still exists but at a different URL, update the link to the new address. This is the cleanest resolution because it removes the broken link without creating a redirect dependency. Use WP-CLI search-replace for bulk updates when the same URL appears across many posts. Use the WordPress editor for isolated cases.
Set Up a 301 Redirect
When the original content has been deleted but a relevant replacement exists, set up a 301 redirect from the old URL to the new one. A 301 passes most of the link equity from the old URL to the new destination, which protects your SEO while ensuring users and search engine crawlers reach working content.
Use the Redirection plugin for site-wide redirect management in WordPress. For large numbers of redirects or on managed hosting platforms, server-level redirect configuration is more reliable and performs better than plugin-based redirects.
Remove the Link
When the content the link pointed to is permanently gone and there is no suitable replacement, remove the link rather than redirecting to an unrelated page. A redirect to an irrelevant destination is a worse experience than no link at all, and Google can detect when redirect targets are poorly matched to the original URL.
Prevention: Building a Process That Stops Broken Links Before They Start
Detection and fixing are reactive. The more effective long-term investment is building a workflow that prevents broken links from accumulating in the first place.
Redirect on Delete
The most impactful prevention measure is making redirect creation mandatory whenever a post or page is deleted. This can be enforced at the editorial level through a process requirement, or enforced technically through a mu-plugin that triggers a redirect creation flow when a post moves to trash. Plugins like Yoast SEO Premium and Rank Math Pro both include delete-triggered redirect prompts that remind editors to create a redirect before confirming deletion.
Stable URL Structure
The best time to design your permalink structure is before publishing any content, because changing it later breaks every existing internal and external link to your site. If a permalink structure change is unavoidable, run WP-CLI search-replace across the database before going live, and prepare a redirect list that maps old URLs to new ones.
For sites that use post IDs in custom meta or third-party integrations, avoid building logic that relies on slugs or URL patterns that may change over time. Post IDs remain stable; slugs do not.
Scheduled Audits
Set a recurring calendar reminder to run a link audit every four to six weeks. For high-traffic sites or active WooCommerce stores, monthly is the appropriate frequency. Screaming Frog supports scheduled crawls that export results automatically. Google Search Console sends email alerts when new crawl errors appear, which serves as a passive continuous monitor between formal audits.
Do not rely on continuous server-side link checking plugins as your primary monitoring mechanism on large sites. The crawl budget they consume on your server is not worth the real-time detection benefit when a properly configured Search Console alert achieves the same outcome with no server cost.
Editorial Verification Checklist
For multi-author sites, add a link verification step to the editorial publishing checklist. Before any post goes live, every internal link should be clicked and confirmed as working. This takes two minutes per post and catches broken links introduced by the author before they reach production. For posts that reference products, policy pages, or resource documents that change frequently, note those links for re-verification during future content reviews.
Common Questions
Yes, with the right timing. Screaming Frog makes HTTP requests to your site the same way any visitor would. On well-resourced managed hosting, it runs without issue. On underpowered servers or during peak traffic, schedule the crawl during low-traffic hours. You can also throttle the crawl speed in Screaming Frog settings to reduce the request rate.
No. WP-CLI search-replace is designed to handle serialized PHP data correctly. It deserializes, replaces, and reserializes the data, which preserves the correct byte counts that raw SQL REPLACE statements break. Running a raw SQL UPDATE on serialized data corrupts it. Always use WP-CLI for URL replacements in WordPress databases.
Monthly for most large sites. Weekly for active WooCommerce stores where product pages change frequently. Always immediately after a migration, major redesign, or permalink structure change. Google Search Console provides continuous passive monitoring between formal audits, so enable email alerts for new crawl errors.
No. Broken internal links cause more SEO damage because they disrupt your site structure and interrupt the flow of link equity between your own pages. Broken external links are a credibility and user experience problem but have a smaller direct effect on your rankings. Prioritize fixing internal broken links first.
Use a 301 redirect for any change that is permanent. A 301 tells search engines the old URL has moved permanently and passes link equity to the new destination. A 302 signals a temporary move and does not reliably transfer link equity. For cleaning up broken links from deleted pages, 301 is the correct choice in almost every case.
Broken Links Are a Maintenance Problem, Not Just a Technical One
The sites that keep broken links under control are not the ones with the best tools. They are the ones with a consistent process. A monthly Screaming Frog crawl, a redirect-on-delete editorial habit, and a reliable WP-CLI workflow for migrations will keep most large WordPress sites clean.
If your site has accumulated a significant backlog of broken links or you are preparing for a migration and want to avoid the typical post-go-live broken link crisis, this is the kind of systematic audit and cleanup work covered in an Ongoing WordPress Partnership. You can also start with a focused Strategy Session to audit your current link health and build a prioritized fix plan.
