E-Commerce Technical SEO: Crawl Budget, Pagination,...

E-Commerce Technical SEO: Crawl Budget, Pagination,...

person JasaSEO.id Team
calendar_today 25 Jan 2026
schedule 7 min read
bolt

TL;DR (Ringkasan Singkat)

Kalau website Anda punya 10,000 halaman tapi crawl budget cuma 2,000 halaman/hari, maka 8,000 halaman tidak akan pernah di-index. Untuk e-commerce besar (10,000+ halaman), ini sangat penting karena Google tidak akan crawl semua halaman.

format_list_bulleted

Daftar Isi

expand_more

E-Commerce Technical SEO: Crawl Budget, Pagination, Faceted Navigation

"Website toko online saya punya 10,000 produk, tapi Google cuma index 2,000. Kenapa?" Ini adalah masalah klasik yang dialami hampir setiap toko online besar. Mereka berpikir: "Kalau saya punya banyak produk, pasti semua di-index Google."

  • Salah besar.*

Google punya crawl budget terbatas untuk setiap website. Kalau website Anda punya 10,000 halaman tapi crawl budget cuma 2,000 halaman/hari, maka 8,000 halaman tidak akan pernah di-index.

  • Kesimpulan:* Technical SEO untuk e-commerce bukan optional, tapi mandatory untuk survival.

E-Commerce vs Regular Website Technical SEO

Aspek Regular Website E-Commerce Website
**Halaman** 10-1,000 10,000-1,000,000+
**Duplicate Content** Rare Very common (variants, filters)
**Crawl Budget** Not a concern Critical issue
**Site Speed** Important Mission-critical (conversion impact)
**URL Structure** Simple Complex (categories, filters, pagination)
**Schema Markup** Article, Organization Product, Offer, Review, Breadcrumb List

Crawl Budget Optimization

1.1. Apa Itu Crawl Budget?

  • Definisi:* Jumlah halaman yang Googlebot crawl di website Anda per hari.

    build Schema Generator

    Gunakan Schema Generator secara gratis untuk membantu optimasi Anda.

    Coba Sekarang Gratis

  • Faktor yang Mempengaruhi:*

  • Site Authority: Website authority tinggi = crawl budget lebih besar

  • Site Speed: Website cepat = lebih banyak halaman di-crawl

  • Crawl Errors: Banyak 404/500 = crawl budget berkurang

  • Site Size: 10,000 halaman vs 100,000 halaman

  • Cara Cek Crawl Budget:*

  • Google Search Console → Settings → Crawl Stats

  • Lihat "Total crawl requests" per hari

  • Compare dengan total halaman di website

  • Example:*

  • Total halaman: 50,000

  • Crawl budget: 2,000 halaman/hari

  • Problem: 48,000 halaman mungkin tidak ter-crawl

1.2. Cara Optimize Crawl Budget

Strategy 1: Block Low-Value Pages

  • Robots.txt:*
User-agent: * Disallow: /cart/ Disallow: /checkout/ Disallow: /my-account/ Disallow: /thank-you/ Disallow: /?add-to-cart= Disallow: /*?sort= Disallow: /*?filter=
  • Why:* Halaman ini tidak punya SEO value, waste crawl budget.

Strategy 2: Fix Crawl Errors

  • Common Errors:*

  • 404 errors (broken links)

  • 500 errors (server errors)

  • Redirect chains (A → B → C → D)

  • Solution:*

  • Fix 404: Redirect ke relevant page atau remove link

  • Fix 500: Improve server stability

  • Fix redirects: Direct redirect (A → D)

Strategy 3: Improve Site Speed

  • Logic:* Website cepat = Googlebot crawl lebih banyak halaman dalam waktu sama.

  • Target:*

  • Server response time < 200ms

  • Page load time < 2 seconds

1.3. Prioritize Important Pages

  • XML Sitemap Priority:*
<url> <loc>https://example.com/</loc> <priority>1.0</priority> </url> <url> <loc>https://example.com/category/sepatu-pria/</loc> <priority>0.8</priority> </url> <url> <loc>https://example.com/product/nike-air-max/</loc> <priority>0.6</priority> </url>

  • Internal Linking:*

  • Homepage → Category pages (strong links)

  • Category pages → Product pages (strong links)

  • Product pages → Related products (moderate links)

Pagination Best Practices

2.1. The Pagination Problem

  • Scenario:*

  • Category page: 500 products

  • 20 products per page

  • = 25 pagination pages

  • Issues:*

  • Duplicate Content: Page 1, 2, 3 punya content similar

  • Link Equity Dilution: Backlinks spread across 25 pages

  • Crawl Budget Waste: Googlebot crawl 25 pages untuk 1 category

2.2. Pagination Solutions

Solution 1: Load More Button

  • How it Works:*

  • Initial load: 20 products

  • Click "Load More": Load next 20 products

  • URL tetap sama (no pagination URLs)

  • Pros:*

  • No duplicate content

  • Better UX

  • Crawl budget efficient

  • Cons:*

  • Need JavaScript implementation

  • Googlebot must render JavaScript

Solution 2: Infinite Scroll

  • How it Works:*

  • Auto-load products saat user scroll down

  • No pagination URLs

  • SEO Implementation:*

<link rel="next" href="?page=2"> <link rel="prev" href="?page=1">

  • Pros:*

  • Excellent UX

  • No duplicate content

  • Cons:*

  • Complex implementation

  • Must provide pagination links for Googlebot

Solution 3: View All Page

  • How it Works:*

  • Provide "View All" link (show all 500 products)

  • Canonical from paginated pages → View All page

  • Pros:*

  • All products on 1 page (SEO-friendly)

  • No duplicate content

  • Cons:*

  • Slow page load (500 products)

  • Poor UX for mobile

Faceted Navigation Solutions

3.1. The Faceted Navigation Problem

  • What is Faceted Navigation?* Filter system untuk narrow down products:

  • Color: Hitam, Putih, Merah

  • Size: 39, 40, 41, 42

  • Price: < 500k, 500k-1jt, > 1jt

  • Brand: Nike, Adidas, Puma

  • The Problem:*

  • 3 colors × 4 sizes × 3 price ranges × 3 brands = 108 URL combinations

  • For 10 categories = 1,080 URLs

  • For 100 categories = 10,800 URLs

  • Impact:*

  • Massive duplicate content

  • Crawl budget waste

  • Thin content (filter pages with 0-5 products)

3.2. Faceted Navigation Solutions

Solution 1: Parameter Handling (Google Search Console)

  • Setup:*

  • Google Search Console → Legacy Tools → URL Parameters

  • Add parameters: color, size, price, brand

  • Set to "Let Googlebot decide" or "No URLs"

  • Pros:*

  • Easy setup

  • Google handles it automatically

  • Cons:*

  • Not 100% reliable

  • Google may still index some filter pages

Solution 2: Canonical Tags

  • Implementation:*
<!-- Filter page --> <link rel="canonical" href="https://example.com/category/sepatu/">

  • Pros:*

  • Clear signal to Google

  • Consolidate ranking signals

  • Cons:*

  • Filter pages won't rank independently

Solution 3: Robots.txt + Noindex

  • Robots.txt:*
User-agent: * Disallow: /*?color= Disallow: /*?size= Disallow: /*?price=
  • Meta Tag* (for filter pages):
<meta name="robots" content="noindex, follow">

  • Pros:*

  • Complete control

  • No crawl budget waste

  • Cons:*

  • Must implement on all filter pages

Duplicate Content Solutions

4.1. Common Duplicate Content Issues

Issue 1: Product Variants

  • Problem:*

  • Nike Air Max Hitam Size 42

  • Nike Air Max Hitam Size 43

  • Same description, different size

  • Solution:*

<link rel="canonical" href="https://example.com/product/nike-air-max/">

Issue 2: Manufacturer Descriptions

  • Problem:*

  • 100 toko pakai deskripsi sama dari supplier

  • Solution:*

  • Tulis unique descriptions (minimum 300 kata)

  • Add value: use cases, comparisons, local context

Issue 3: Cross-Selling Pages

  • Problem:*

  • "Customers also bought" pages

  • "Similar products" pages

  • Solution:*

  • Noindex these pages

  • Or use canonical to main category page

Site Speed Optimization

5.1. Why Speed Matters for E-Commerce

  • Conversion Impact:*

  • 1 second delay = 7% drop in conversion

  • 3 second delay = 40% bounce rate

  • SEO Impact:*

  • Core Web Vitals = ranking factor

  • Slow site = less crawl budget

5.2. Core Web Vitals for E-Commerce

  • Target Metrics:*

  • LCP (Largest Contentful Paint): < 2.5 seconds

  • INP (Interaction to Next Paint): < 200ms

  • CLS (Cumulative Layout Shift): < 0.1

  • Common Issues:*

  • Large product images (not optimized)

  • Too many product reviews loaded

  • Heavy JavaScript (product sliders, filters)

5.3. Speed Optimization Checklist

  • Image Optimization:*

  • [ ] Compress images (< 200 KB)

  • [ ] Use WebP format

  • [ ] Lazy load images (below fold)

  • [ ] Responsive images (srcset)

  • Caching:*

  • [ ] Browser caching enabled

  • [ ] Server-side caching (Redis, Varnish)

  • [ ] CDN for static assets

  • Code Optimization:*

  • [ ] Minify CSS/JS

  • [ ] Remove unused CSS/JS

  • [ ] Defer non-critical JavaScript

  • Database Optimization:*

  • [ ] Index database tables

  • [ ] Cache database queries

  • [ ] Optimize product search queries

Pagination Methods Comparison

Method SEO Impact UX Implementation Best For
**Numbered Pagination** Medium (duplicate content) Good Easy Small catalogs (< 100 products)
**Load More Button** Excellent Excellent Medium Medium catalogs (100-1000)
**Infinite Scroll** Good (with proper implementation) Excellent Hard Large catalogs (1000+)
**View All Page** Excellent Poor (slow load) Easy Small catalogs only

Faceted Navigation Solutions Comparison

Solution Effectiveness Complexity Crawl Budget Impact
**Parameter Handling (GSC)** Medium Easy Medium
**Canonical Tags** High Medium High (saves budget)
**Robots.txt** Very High Easy Very High
**Noindex Meta Tag** Very High Medium Very High
**Combination (Canonical + Noindex)** Excellent Medium Excellent

FAQ: Pertanyaan Umum E-Commerce Technical SEO

  • Q: Apa itu crawl budget dan kenapa penting untuk e-commerce?* A: Crawl budget adalah jumlah halaman yang Googlebot crawl per hari. Untuk e-commerce besar (10,000+ halaman), ini sangat penting karena Google tidak akan crawl semua halaman. Prioritaskan halaman penting dan block halaman low-value.

  • Q: Bagaimana cara mengatasi duplicate content di toko online?* A: Solusi: 1) Canonical tags untuk product variants, 2) Parameter handling untuk filter pages, 3) Tulis unique product descriptions, 4) Noindex untuk low-value pages.

  • Q: Apakah pagination mempengaruhi SEO?* A: Ya. Pagination yang salah bisa menyebabkan duplicate content, crawl budget waste, dan link equity dilution. Solusi: gunakan 'Load More' button atau infinite scroll.

  • Q: Bagaimana cara optimasi faceted navigation?* A: Faceted navigation bisa create jutaan URL duplicate. Solusi: 1) Parameter handling, 2) Canonical tags, 3) Robots.txt untuk block filter combinations, 4) Noindex untuk low-value filter pages.

  • Q: Apakah semua halaman produk harus di-index?* A: Tidak. Prioritaskan: 1) Best-selling products (index), 2) Out of stock products (noindex atau redirect), 3) Low-quality products (improve atau noindex), 4) Duplicate variants (canonical).

  • Q: Berapa kecepatan loading ideal untuk halaman produk?* A: Target: LCP < 2.5 detik, INP < 200ms, CLS < 0.1. Setiap 1 detik delay = 7% drop conversion rate.

Butuh Bantuan Technical SEO untuk E-Commerce?

Technical SEO untuk e-commerce sangat complex. Satu kesalahan kecil (misal: accidentally noindex semua product pages) bisa menghancurkan traffic organik Anda. Kami menyediakan Technical SEO Services khusus untuk e-commerce dengan focus pada crawl budget optimization, duplicate content handling, dan Core Web Vitals improvement.

Pelajari Artikel Terkait:

read_more Artikel Terkait

Article

Aged Domain untuk E-commerce: Worth It atau Tidak?...

Pelajari selengkapnya tentang topik ini....

calendar_today 2026-02-05
Article

Category Page SEO: Optimasi Halaman Kategori E-Commerce...

Pelajari selengkapnya tentang topik ini....

calendar_today 2026-01-25
Article

Customer Reviews SEO: Senjata Rahasia E-Commerce yang...

Pelajari selengkapnya tentang topik ini....

calendar_today 2026-01-25

Butuh Bantuan SEO Profesional?

Tim ahli kami siap membantu website Anda ranking di halaman 1 Google.

Konsultasi Gratis arrow_forward SEO Audit