Crawl
Can search engines discover important URLs?
Topical Authority
Build a Technical Foundation Search Engines Can Access
Find and fix the technical barriers that affect crawling, indexation, rendering, performance and the accessibility of your website’s important content.
The Foundation
Technical SEO Makes Your Website Accessible to Search Systems
Technical SEO focuses on the technical conditions that affect how search engines can crawl, render, interpret and index a website, while also supporting a usable experience for visitors.
It sits underneath everything else. A strong content architecture only works if the pages it describes can actually be reached and processed.
Render
Can important page content be processed correctly?
Index
Are the right pages eligible to appear in search?
Perform
Does the website provide a fast and usable experience?
The Journey
How Search Systems Move Through Your Website
Six stages, in order. A problem at any one of them limits everything after it, which is why technical work follows this sequence rather than a list of warnings.
-
What happens
Search systems find out that a URL exists, usually through a link or a sitemap entry.
What can go wrongA page nothing links to, and that appears in no sitemap, may simply never be found.
What I check- Internal links
- XML sitemaps
- Orphan pages
- Navigation paths
-
What happens
Search systems request the URL and follow the accessible paths they find on it.
What can go wrongBlocked paths, redirect chains and error responses can stop a crawl reaching what matters.
What I check- Robots.txt
- Status codes
- Redirects
- Crawl paths
-
What happens
The page is processed so its content, links and layout can be read as a finished document.
What can go wrongContent that depends on scripts that fail, stall or are blocked may not be available when the page is processed.
What I check- Blocked resources
- JavaScript content
- Links in markup
- Lazy loading
-
What happens
The page becomes eligible to be stored and served. Eligibility is not the same as a guarantee of inclusion.
What can go wrongAn accidental noindex, a canonical pointing elsewhere, or near-duplicate URLs competing with each other.
What I check- Noindex directives
- Canonical tags
- Duplicate URLs
- Indexable page set
-
What happens
The subject of the page, and its relationship to the rest of the site, is interpreted.
What can go wrongA flat hierarchy, weak internal linking or markup that contradicts the visible content all blur the picture.
What I check- Site hierarchy
- Internal linking
- Structured data
- Heading structure
-
What happens
The page can be shown to someone, and their experience of it begins with how quickly it loads.
What can go wrongSlow responses, layout that shifts under the reader, or a page that fails on a phone.
What I check- Core Web Vitals
- Mobile usability
- Server response
- Resource loading
The System
See Your Website as a Technical System
Technical issues rarely sit on their own. Each subsystem below affects the others, so a fix in one place often changes what matters somewhere else.
Hover any node to highlight its connection to the website
Crawlability
- Robots.txt
- Internal links
- Sitemaps
- Status codes
Indexation
- Noindex
- Canonicals
- Duplicate URLs
- Indexable pages
Rendering
- JavaScript content
- Blocked resources
- Lazy loading
- Links in markup
Architecture
- URL structure
- Hierarchy depth
- Orphan pages
- Pagination
Performance
- LCP
- INP
- CLS
- Page speed
Structured data
- Schema types
- Validation
- Accuracy
- Entity consistency
Mobile
- Responsive layout
- Viewport
- Touch targets
- Readability
Security
- HTTPS
- Certificate validity
- Mixed content
- Redirect consistency
Fixing any single item on this list does not on its own guarantee a ranking change. The value is in removing barriers so the rest of the work can be seen at all.
The Workflow
My Technical SEO Workflow
Seven stages, run in order. Diagnosis comes before prioritisation, and prioritisation before any fix, so effort goes where it actually changes something.
-
01
Discover
Collect technical data and understand the website.
-
02
Crawl
Analyze URLs, links, status codes and crawl paths.
-
03
Diagnose
Identify technical problems and their likely impact.
-
04
Prioritize
Separate critical issues from low-priority improvements.
-
05
Fix
Implement technical recommendations.
-
06
Validate
Re-crawl and verify changes.
-
07
Monitor
Track technical health over time.
Discover
→
Diagnose
→
Fix
→
Validate
→
Monitor
Crawlability
Make Important Pages Easy to Discover
Search engines need accessible paths to discover important URLs. How much of a site gets crawled, and how often, varies between websites, so the goal is to remove obstacles rather than to chase a fixed number.
Good crawl path
- Crawler
- Homepage
- Category
- Service
- Supporting page
Every step returns a 200 and links onward. Depth stays shallow enough that important pages are a few clicks from the homepage.
versus
Broken path
- Crawler
- Homepage
- Redirect chain
- 404 not found
- Page never reached
The page may be perfectly good. Nothing that leads to it works, so none of that matters.
What gets reviewed
- Internal links
- Robots.txt
- XML sitemaps
Response handling
- Status codes
- Redirects and chains
- Soft 404s
Path quality
- Click depth
- Crawl paths
- Dead ends
Waste
- Parameter URLs
- Infinite spaces
- Low-value duplicates
Indexation
Make Sure the Right Pages Are Indexable
Not every URL needs to be indexed. The goal is to help ensure important, unique pages are technically eligible for indexing while unnecessary or duplicate URLs are handled appropriately.
Indexable
- 200 status
- Self-referencing canonical
- No accidental noindex
- Unique purpose
versus
Problem
- Noindex
- Blocked in robots.txt
- Duplicate of another URL
- Canonical points elsewhere
- 4xx response
Eligibility is not inclusion. Making a page technically indexable means removing the reasons it could be excluded. Whether it is then stored and served depends on factors outside any technical fix.
Rendering
Make Important Content Accessible During Rendering
Modern websites can rely heavily on JavaScript. Technical SEO should verify that important content, links and elements remain accessible to search systems and users once the page is processed.
The practical test is simple: if the important content and links are still there when the page is rendered, the risk is low. If they only appear after a script succeeds, that script becomes a dependency worth checking.
- HTMLStructure
- CSSPresentation
- JavaScriptBehaviour
- Rendered pageResult
- Content · Links · NavigationWhat must survive
Architecture
Turn Your Information Architecture Into a Clean Technical Structure
The hierarchy decided in content architecture has to survive contact with the CMS. URLs, directories and breadcrumbs are where that structure either becomes visible or quietly disappears.
- Logical hierarchy
- Consistent URLs
- Relevant directories
- Breadcrumbs
- Internal relationships
Canonicalisation
Control Duplicate and Competing URLs
The same page can often be reached through several addresses. Left alone, those variants compete with each other and split the signals between them.
A canonical tag states which version is preferred. It is treated as a strong hint rather than an absolute instruction, so it works best when the rest of the site agrees with it: consistent internal links, redirects and sitemap entries pointing the same way.
- Canonical tags
- Duplicate pages
- URL parameters
- HTTP and HTTPS
- WWW and non-WWW
- Trailing slash consistency
- Redirects
- URL A?ref=nav
- URL BNo trailing slash
- URL CWWW variant
- Canonical signalA strong hint
- Preferred URLOne address
Connections
Connect Important Pages Through Internal Links
Important pages should have meaningful pathways from other relevant pages. A page with no incoming internal link depends entirely on a sitemap entry to be found at all.
What internal linking should do
- Give every important page a route in
- Use contextual, descriptive anchors
- Connect core and supporting pages both ways
- Keep click depth reasonable
This is the technical half of a job that starts in content architecture and is informed by entity SEO: the structure decides which relationships exist, and the links make them real.
Crawler Guidance
Guide Crawlers Without Creating Unnecessary Barriers
Two small files with outsized consequences. Both are worth reviewing precisely because they are easy to set once and never look at again.
XML sitemap
Helps communicate important URLs
- Indexable URLs only
- Canonical versions only
- 200 status responses
- A clean, current URL set
Why it matters
A sitemap listing redirects, 404s or noindexed URLs sends mixed messages about what the site considers important. Submitting a URL does not mean it will be indexed.
Robots.txt
Provides crawler instructions
- Review blocked paths
- Avoid accidental blocking
- Keep directives intentional
- Do not block needed CSS or JS
An important distinction
Robots.txt provides crawler directives. It is not an indexing control mechanism: a blocked URL can still be indexed if other signals point to it, and blocking a page is not the same as removing it.
Machine-Readable
Add Structured Data Where It Accurately Represents the Page
Structured data can help communicate supported information in a standardized format. It describes what is already on the page rather than adding anything new.
Which types make sense follows from the entities a page is actually about, which is where entity SEO and the technical layer meet.
- Web page
- Content
- Schema
- Machine-readable information
Types worth considering
- Organization
- Person
- Service
- Article
- Product
- LocalBusiness
- BreadcrumbList
JavaScript
Technical SEO for JavaScript-Driven Websites
JavaScript is not a problem in itself. The question is whether important content and links depend on it succeeding, and what happens when it does not.
Accessible content
- Content present in the markup
- Real anchor links
- Navigation available without interaction
- Images lazy-loaded, not hidden
versus
Content dependent on broken or delayed scripts
- Content injected after a failed request
- Links that are click handlers, not anchors
- Menus that never render
- Resources blocked from being fetched
Rendering approach
- Client-side rendering
- Server-side rendering
- Static generation
Content delivery
- Dynamic content
- Lazy loading
- Hidden content
Links
- JavaScript links
- Anchor markup
- Navigation availability
Failure modes
- Blocked scripts
- Slow execution
- Rendering errors
Performance
Performance Is Part of the Technical Foundation
Core Web Vitals are user-experience performance metrics. Good page experience and performance can support usability and overall site quality.
Largest Contentful Paint
How long until the main content of the page has actually appeared. Usually decided by server response time, render-blocking resources and the size of the hero image.
Cumulative Layout Shift
How much the layout moves while loading. Usually caused by images without dimensions, late-loading fonts or content injected above what someone is already reading.
Interaction to Next Paint
How quickly the page responds when someone interacts with it. Usually a question of how much work the main thread is doing at the moment of the tap or click.
Assets
- Image optimization
- Font loading
- Lazy loading
Code
- Unused CSS
- Unused JavaScript
- Resource prioritization
Delivery
- Server response
- Caching
- Compression
Stability
- Image dimensions
- Reserved space
- Third-party scripts
Performance work supports usability and the overall quality of the experience. It is not a guarantee of ranking improvement on its own.
Devices
Technical SEO Should Work Across Devices
One website, one set of content, adapting to the screen it is shown on. Anything that works on a desktop and fails on a phone is a technical problem, not a design preference.
Layout
- Responsive layout
- Viewport configuration
- No horizontal scrolling
Readability
- Readable text sizes
- Adequate contrast
- No clipped content
Interaction
- Navigation on small screens
- Touch targets
- Tap spacing
Parity
- Same content across devices
- Content accessibility
- Mobile performance
The Audit
What I Check During a Technical SEO Audit
Grouped by subsystem, so findings arrive organised rather than as one long undifferentiated list of warnings.
Crawlability
- Robots.txt
- Internal links
- Status codes
- Redirects
Indexation
- Noindex directives
- Canonicals
- Duplicate URLs
- Indexable pages
Architecture
- URL structure
- Hierarchy depth
- Orphan pages
- Pagination
Performance
- LCP
- INP
- CLS
- JavaScript
- CSS
- Images
Mobile
- Responsive layout
- Navigation
- UX and touch targets
Structured data
- Schema types
- Validation
- Entity consistency
Sitemaps
- XML sitemap
- Indexable URLs
- Sitemap errors
Prioritisation
Not Every Technical Issue Deserves the Same Priority
Audit tools flag everything they can detect. Deciding what to act on is a separate judgement based on the website’s context, issue severity, how many URLs are affected and how much work the fix takes.
High impact · Low effort
Fix first
An accidental noindex on a key page, a broken canonical, a blocked resource. Small changes with disproportionate consequences.
High impact · High effort
Plan
URL restructures, rendering changes, migrations. Worth doing, but they need scheduling, testing and a rollback plan.
Low impact · Low effort
Quick improvement
Tidy-ups worth bundling into other work. Useful, rarely urgent, and easy to overweight because they are satisfying to close.
Low impact · High effort
Defer
Warnings that are technically true and practically irrelevant for this website right now. Documented, then left alone.
A red warning is not the same as a priority. The same issue can be critical on one website and irrelevant on another, depending on how many URLs it touches and whether those URLs matter.
The Output
From Technical Noise to an Action Plan
Findings arrive grouped by subsystem with a clear status against each, so the first question after an audit is never “where do we start?”
Crawlability
Healthy
Paths are clear and status codes are consistent. No action needed.
Indexation
Priority
Something is preventing important pages from being eligible. Fixed first.
Architecture
Review
Works, but the hierarchy or URLs drift from the intended structure.
Performance
Review
Usable, with specific assets or scripts holding back the experience.
Mobile
Healthy
Layout, navigation and touch targets behave across screen sizes.
Structured data
Review
Present but incomplete, or describing more than the page actually contains.
Internal linking
Priority
Important pages with no route in. Usually one of the fastest things to fix.
The statuses above are an illustration of how findings are presented. They are not a score for any real website, and no numerical health score is assigned to yours without an actual audit.
The Change
What Technical Cleanup Actually Changes
A technical audit does not make a website new. It removes the things getting in the way of what is already there.
Before
- Broken redirects
- Orphan pages
- Duplicate URLs
- Slow resources
- Wrong canonical
- Blocked resources
cleanup
After
- Clear crawl paths
- Correct canonicalization
- Connected pages
- Cleaner URL system
- Improved performance foundation
Every website starts from a different position, so the specific findings and the scale of the change differ in each case. This is an illustration of the type of improvement technical work produces, not a prediction of results.
Deliverables
What My Technical SEO Work Includes
Everything is documented so your developer can act on it directly, whether the implementation is theirs or mine.
Technical SEO Audit
A full review of the technical conditions affecting crawling, rendering, indexation and performance.
Crawlability Analysis
How search systems can reach your important URLs, and where those paths break down.
Indexation Analysis
Which pages are technically eligible for indexing, and which are excluded by accident.
Canonical & Duplicate URL Review
Competing URL variants, parameter handling and canonical consistency across the site.
Site Architecture Review
URL structure, hierarchy depth and whether the intended architecture survived implementation.
Internal Linking & Orphan Page Analysis
Which important pages lack incoming links, and where the linking contradicts the structure.
XML Sitemap & Robots.txt Review
Whether both files say what you intend, and whether anything is blocked by accident.
Core Web Vitals & Performance Review
LCP, INP and CLS alongside the asset, code and delivery issues behind them.
Structured Data Review
Whether existing markup is valid, accurate and consistent with the page it describes.
Technical SEO Implementation Roadmap
Every finding ordered by impact and effort, so the sequence of work is already decided.
Fit
Who Needs Technical SEO?
Existing websites
Technical problems accumulated over time, often invisible until something is looked for.
Redesigns
Protect SEO when changing architecture or URLs, before the change goes live.
Large websites
Complex crawling, indexing and internal linking requirements across many templates.
Content-heavy websites
Large numbers of URLs require stronger technical control over what is indexable.
Why work with me?
Technical SEO Should Support the Bigger Semantic Structure
I approach technical SEO as part of the wider website system, connecting technical accessibility with semantic structure, content architecture, entities and topical coverage.
That changes what gets prioritised. A technical issue on a page central to your topical structure matters more than the same issue on a page that supports nothing, and a list of warnings sorted by severity alone cannot tell you which is which.
About DanishHow it fits together
- 01 Semantic SEO
- 02 Topical Authority
- 03 Entity SEO
- 04 Content Architecture
- 05 Technical SEO
Questions
Technical SEO FAQs
Technical SEO covers the technical conditions that affect how search engines can crawl, render, interpret and index a website, and how usable that website is for visitors. It includes crawl paths, status codes, canonicals, site architecture, rendering, structured data, performance and mobile usability.
Because everything else depends on it. Content and structure can only do their job if the pages can be reached, processed and understood. Technical SEO removes the barriers rather than adding new signals.
On-page SEO is mostly about what a page says: its content, headings, internal links and how well it addresses a search need. Technical SEO is about the conditions that let that page be accessed and processed at all. They overlap around things like internal linking and page structure.
Crawlability, indexation, site architecture and URLs, canonicals and duplicates, internal linking and orphan pages, rendering and JavaScript, structured data, Core Web Vitals and performance, mobile usability, and the sitemap and robots.txt files.
Crawlability is how easily search systems can reach the URLs on a website. It depends on accessible internal links, correct status codes, sensible redirects, an accurate sitemap and a robots.txt file that does not block anything important.
Indexation is whether a URL is technically eligible to be stored and served in search results. Eligibility depends on things like the status code, canonical tags and noindex directives. Not every URL on a site needs to be indexed.
A canonical URL is the version of a page you want treated as the preferred one when several addresses show similar content. The canonical tag communicates that preference. It is treated as a strong hint rather than an absolute instruction, so consistent internal links, redirects and sitemap entries matter alongside it.
An XML sitemap can help communicate which URLs a website considers important, which is particularly useful on large sites or for pages with few internal links. It should list only canonical, indexable URLs that return a 200. Including a URL in a sitemap does not mean it will be indexed.
Technical SEO can remove technical barriers and improve accessibility, crawlability, indexation and user experience. Rankings depend on many factors, so technical fixes alone do not guarantee ranking improvements.
There is no universal schedule. Larger or frequently changing websites may need more frequent monitoring, while smaller stable websites may require periodic reviews.
Fix the foundation
Find the Technical Issues Holding Your Website Back
Get a prioritized technical SEO roadmap focused on the issues that matter most to your website’s crawlability, indexation, architecture and performance.