Scrabble tiles spelling 'SEO' on a wooden surface. Ideal for digital marketing themes.

Discovered – Currently Not Indexed: Why It’s Rarely Crawl Budget on WordPress

BySplinternet Marketing

If Google Search Console is filling up with “Discovered – currently not indexed” or “Crawled – currently not indexed”, the reflex diagnosis is crawl budget.

For most small and mid-size WordPress and WooCommerce sites in the U.S., that’s usually the wrong first move.

Google’s documentation on crawl budget is explicit: it primarily matters for very large sites or sites with extremely high URL churn. If you’re not operating at massive scale or publishing millions of URLs, crawl demand is rarely your binding constraint.

When URLs sit in “Discovered” or “Crawled – currently not indexed,” Google has typically made a prioritization or quality decision—not hit a bot ceiling.

Common causes:

  • Duplicate or near-duplicate URLs competing for canonical selection.
  • Signals consolidating to a different canonical.
  • Thin, templated, or low-value archive pages.
  • Accidental noindex directives.
  • Unstable or incomplete rendering.

Google’s canonical guidance explains that when duplicates exist, signals consolidate to the selected canonical. Non-selected versions may appear as not indexed—even though their signals are counted elsewhere.

And per Google’s robots meta tag documentation, a noindex meta tag or X-Robots-Tag removes a page from Search. No index means no eligibility for search visibility.

Need help checking this on your WordPress, Google Ads, Analytics, local SEO, or website setup? Splinternet Marketing can review the issue and help you prioritize the next fix.

Where WordPress and WooCommerce Sites Create the Problem

This is usually structural, not budgetary.

1. Faceted navigation and filter sprawl.
WooCommerce attributes (color, size, price, layered filters) can generate parameterized URLs at scale. Most combinations do not represent distinct search demand. If they’re crawlable and internally linked, they create duplication and prioritization noise.

2. Tag, date, and search archives.
Themes and plugins often output indexable tag archives, author archives, and internal search URLs. WordPress plugin documentation confirms that plugins can register routes and endpoints programmatically. Each endpoint increases crawl surface area.

3. Canonical conflicts.
SEO plugins, theme-level canonicals, and WooCommerce templates can disagree. If internal links, XML sitemaps, and canonicals point to different URL variants (HTTP vs HTTPS, trailing slash variants, parameters), Google may select its own canonical and exclude others.

4. Soft 404 patterns.
Empty categories, expired products returning 200 status codes, or thin location pages often land in “Crawled – currently not indexed.” Google crawled them and determined they don’t merit inclusion.

5. Rendering dependencies.
Google renders pages before final indexing decisions. If primary content relies heavily on delayed JavaScript or fails intermittently, Google may see less content than users do. That can affect indexation decisions.

What to do next

Before editing robots.txt or blaming crawl budget, run this focused audit this week:

  1. Segment by template. In Search Console, group affected URLs by page type (product, category, blog, tag, parameter). Patterns matter more than individual pages.
  2. Validate canonicals at scale. Confirm self-referencing canonicals on URLs intended to rank. Ensure internal links and sitemaps reference the same version.
  3. Check index directives. Inspect for unintended noindex meta tags or X-Robots-Tag headers, especially after migrations or staging pushes.
  4. Rationalize filters. Decide intentionally which attribute or filtered URLs deserve indexation. Reduce crawlable combinations that don’t map to meaningful queries.
  5. Clean thin archives. Noindex or consolidate weak tag, date, and empty category pages rather than letting them compete for crawl attention.
  6. Test rendering. Use URL Inspection to confirm Google can render and see primary content without relying entirely on late-loading scripts.
  7. Improve internal linking. Ensure priority pages are reachable within a few clicks and linked contextually from authoritative pages.

Only after duplication, canonicals, and directives are clean should you consider crawl controls. Robots.txt manages crawling, not indexing, and blocking URLs can prevent Google from understanding duplication and consolidation relationships.

Crawl budget matters for massive ecommerce catalogs and highly dynamic platforms. For most WordPress and WooCommerce installs under a few hundred thousand URLs, “Discovered” and “Crawled – currently not indexed” are structure and quality signals—not bot scarcity.

Fix discovery and duplication first. Touch crawl controls last.

Sources

Need help checking this on your WordPress, Google Ads, Analytics, local SEO, or website setup? Splinternet Marketing can review the issue and help you prioritize the next fix.

This article is for informational purposes only and reflects general marketing, technology, website, and small-business guidance. Platform features, policies, search behavior, pricing, and security conditions can change. Verify current requirements with the relevant platform, provider, or professional advisor before acting. Nothing in this article should be treated as legal, tax, financial, cybersecurity, or other professional advice.

Editorial note: Splinternet Marketing articles are researched from cited platform, documentation, regulatory, and industry sources. AI may assist with drafting and review; final content is checked for source support, practical usefulness, and platform/date accuracy before publication.

Related Posts

Splinternet Marketing