E-Commerce Technical SEO: Crawl Budget, Pagination,...

E-Commerce Technical SEO: Crawl Budget, Pagination,...

person JasaSEO.id Team
calendar_today 2026-01-25
schedule 7 min read
bolt

TL;DR (Ringkasan Singkat)

# E-Commerce Technical SEO: Crawl Budget, Pagination, Faceted Navigation

format_list_bulleted

Daftar Isi

expand_more

E-Commerce Technical SEO: Crawl Budget, Pagination, Faceted Navigation

"Website toko online saya punya 10,000 produk, tapi Google cuma index 2,000. Kenapa?" Ini adalah masalah klasik yang dialami hampir setiap toko online besar. Mereka berpikir: "Kalau saya punya banyak produk, pasti semua di-index Google." Salah besar.

Google punya crawl budget terbatas untuk setiap website. Kalau website Anda punya 10,000 halaman tapi crawl budget cuma 2,000 halaman/hari, maka 8,000 halaman tidak akan pernah di-index.

Kesimpulan: Technical SEO untuk e-commerce bukan optional, tapi mandatory untuk survival.

E-Commerce vs Regular Website Technical SEO

Aspek Regular Website E-Commerce Website
Halaman 10-1,000 10,000-1,000,000+
Duplicate Content Rare Very common (variants, filters)
Crawl Budget Not a concern Critical issue
Site Speed Important Mission-critical (conversion impact)
URL Structure Simple Complex (categories, filters, pagination)
Schema Markup Article, Organization Product, Offer, Review, Breadcrumb List
  • Crawl Budget Optimization (Prioritize important pages)
  • Pagination Best Practices (Avoid duplicate content)
  • Faceted Navigation Solutions (Handle filter URLs)
  • Site Speed Optimization (Core Web Vitals)

Crawl Budget Optimization

1.1. Apa Itu Crawl Budget?

Definisi: Jumlah halaman yang Googlebot crawl di website Anda per hari. Faktor yang Mempengaruhi:
  • Site Authority: Website authority tinggi = crawl budget lebih besar
  • Site Speed: Website cepat = lebih banyak halaman di-crawl
  • Crawl Errors: Banyak 404/500 = crawl budget berkurang
  • Site Size: 10,000 halaman vs 100,000 halaman
Cara Cek Crawl Budget:
  • Google Search Console → Settings → Crawl Stats
  • Lihat "Total crawl requests" per hari
  • Compare dengan total halaman di website
Example:
  • Total halaman: 50,000
  • Crawl budget: 2,000 halaman/hari
  • Problem: 48,000 halaman mungkin tidak ter-crawl

1.2. Cara Optimize Crawl Budget

#### Strategy 1: Block Low-Value Pages

Robots.txt:
User-agent: * Disallow: /cart/ Disallow: /checkout/ Disallow: /my-account/ Disallow: /thank-you/ Disallow: /?add-to-cart= Disallow: /*?sort= Disallow: /*?filter=

Why: Halaman ini tidak punya SEO value, waste crawl budget.

#### Strategy 2: Fix Crawl Errors

Common Errors:
  • 404 errors (broken links)
  • 500 errors (server errors)
  • Redirect chains (A → B → C → D)
Solution:
  • Fix 404: Redirect ke relevant page atau remove link
  • Fix 500: Improve server stability
  • Fix redirects: Direct redirect (A → D)
#### Strategy 3: Improve Site Speed Logic: Website cepat = Googlebot crawl lebih banyak halaman dalam waktu sama. Target:
  • Server response time < 200ms
  • Page load time < 2 seconds

1.3. Prioritize Important Pages

XML Sitemap Priority:
<url> <loc>https://example.com/</loc> <priority>1.0</priority> </url> <url> <loc>https://example.com/category/sepatu-pria/</loc> <priority>0.8</priority> </url> <url> <loc>https://example.com/product/nike-air-max/</loc> <priority>0.6</priority> </url>

Internal Linking:

  • Homepage → Category pages (strong links)
  • Category pages → Product pages (strong links)
  • Product pages → Related products (moderate links)

Pagination Best Practices

2.1. The Pagination Problem

Scenario:
  • Category page: 500 products
  • 20 products per page
  • = 25 pagination pages
Issues:
  • Duplicate Content: Page 1, 2, 3 punya content similar
  • Link Equity Dilution: Backlinks spread across 25 pages
  • Crawl Budget Waste: Googlebot crawl 25 pages untuk 1 category

2.2. Pagination Solutions

#### Solution 1: Load More Button

How it Works:
  • Initial load: 20 products
  • Click "Load More": Load next 20 products
  • URL tetap sama (no pagination URLs)
Pros:
  • No duplicate content
  • Better UX
  • Crawl budget efficient
Cons:
  • Need JavaScript implementation
  • Googlebot must render JavaScript
#### Solution 2: Infinite Scroll How it Works:
  • Auto-load products saat user scroll down
  • No pagination URLs
SEO Implementation:
<link rel="next" href="?page=2"> <link rel="prev" href="?page=1">

Pros:

  • Excellent UX
  • No duplicate content
Cons:
  • Complex implementation
  • Must provide pagination links for Googlebot
#### Solution 3: View All Page How it Works:
  • Provide "View All" link (show all 500 products)
  • Canonical from paginated pages → View All page
Pros:
  • All products on 1 page (SEO-friendly)
  • No duplicate content
Cons:
  • Slow page load (500 products)
  • Poor UX for mobile

Faceted Navigation Solutions

3.1. The Faceted Navigation Problem

What is Faceted Navigation? Filter system untuk narrow down products:
  • Color: Hitam, Putih, Merah
  • Size: 39, 40, 41, 42
  • Price: < 500k, 500k-1jt, > 1jt
  • Brand: Nike, Adidas, Puma
The Problem:
  • 3 colors × 4 sizes × 3 price ranges × 3 brands = 108 URL combinations
  • For 10 categories = 1,080 URLs
  • For 100 categories = 10,800 URLs
Impact:
  • Massive duplicate content
  • Crawl budget waste
  • Thin content (filter pages with 0-5 products)

3.2. Faceted Navigation Solutions

#### Solution 1: Parameter Handling (Google Search Console)

Setup:
  • Google Search Console → Legacy Tools → URL Parameters
  • Add parameters: color, size, price, brand
  • Set to "Let Googlebot decide" or "No URLs"
Pros:
  • Easy setup
  • Google handles it automatically
Cons:
  • Not 100% reliable
  • Google may still index some filter pages
#### Solution 2: Canonical Tags Implementation:
<!-- Filter page --> <link rel="canonical" href="https://example.com/category/sepatu/">

Pros:

  • Clear signal to Google
  • Consolidate ranking signals
Cons:
  • Filter pages won't rank independently
#### Solution 3: Robots.txt + Noindex Robots.txt:
User-agent: * Disallow: /*?color= Disallow: /*?size= Disallow: /*?price=

Meta Tag (for filter pages):

<meta name="robots" content="noindex, follow">

Pros:

  • Complete control
  • No crawl budget waste
Cons:
  • Must implement on all filter pages

Duplicate Content Solutions

4.1. Common Duplicate Content Issues

#### Issue 1: Product Variants

Problem:
  • Nike Air Max Hitam Size 42
  • Nike Air Max Hitam Size 43
  • Same description, different size
Solution:
<link rel="canonical" href="https://example.com/product/nike-air-max/">

#### Issue 2: Manufacturer Descriptions

Problem:

  • 100 toko pakai deskripsi sama dari supplier
Solution:
  • Tulis unique descriptions (minimum 300 kata)
  • Add value: use cases, comparisons, local context
#### Issue 3: Cross-Selling Pages Problem:
  • "Customers also bought" pages
  • "Similar products" pages
Solution:
  • Noindex these pages
  • Or use canonical to main category page

Site Speed Optimization

5.1. Why Speed Matters for E-Commerce

Conversion Impact:
  • 1 second delay = 7% drop in conversion
  • 3 second delay = 40% bounce rate
SEO Impact:
  • Slow site = less crawl budget

5.2. Core Web Vitals for E-Commerce

Target Metrics:
  • LCP (Largest Contentful Paint): < 2.5 seconds
  • INP (Interaction to Next Paint): < 200ms
  • CLS (Cumulative Layout Shift): < 0.1
Common Issues:
  • Large product images (not optimized)
  • Too many product reviews loaded
  • Heavy JavaScript (product sliders, filters)

5.3. Speed Optimization Checklist

Image Optimization:
  • [ ] Compress images (< 200 KB)
  • [ ] Use WebP format
  • [ ] Lazy load images (below fold)
  • [ ] Responsive images (srcset)
Caching:
  • [ ] Browser caching enabled
  • [ ] Server-side caching (Redis, Varnish)
  • [ ] CDN for static assets
Code Optimization:
  • [ ] Minify CSS/JS
  • [ ] Remove unused CSS/JS
  • [ ] Defer non-critical JavaScript
Database Optimization:
  • [ ] Index database tables
  • [ ] Cache database queries
  • [ ] Optimize product search queries

Pagination Methods Comparison

Method SEO Impact UX Implementation Best For
Numbered Pagination Medium (duplicate content) Good Easy Small catalogs (< 100 products)
Load More Button Excellent Excellent Medium Medium catalogs (100-1000)
Infinite Scroll Good (with proper implementation) Excellent Hard Large catalogs (1000+)
View All Page Excellent Poor (slow load) Easy Small catalogs only

Faceted Navigation Solutions Comparison

Solution Effectiveness Complexity Crawl Budget Impact
Parameter Handling (GSC) Medium Easy Medium
Canonical Tags High Medium High (saves budget)
Robots.txt Very High Easy Very High
Noindex Meta Tag Very High Medium Very High
Combination (Canonical + Noindex) Excellent Medium Excellent

FAQ: Pertanyaan Umum E-Commerce Technical SEO

Q: Apa itu crawl budget dan kenapa penting untuk e-commerce? A: Crawl budget adalah jumlah halaman yang Googlebot crawl per hari. Untuk e-commerce besar (10,000+ halaman), ini sangat penting karena Google tidak akan crawl semua halaman. Prioritaskan halaman penting dan block halaman low-value. Q: Bagaimana cara mengatasi duplicate content di toko online? A: Solusi: 1) Canonical tags untuk product variants, 2) Parameter handling untuk filter pages, 3) Tulis unique product descriptions, 4) Noindex untuk low-value pages. Q: Apakah pagination mempengaruhi SEO? A: Ya. Pagination yang salah bisa menyebabkan duplicate content, crawl budget waste, dan link equity dilution. Solusi: gunakan 'Load More' button atau infinite scroll. Q: Bagaimana cara optimasi faceted navigation? A: Faceted navigation bisa create jutaan URL duplicate. Solusi: 1) Parameter handling, 2) Canonical tags, 3) Robots.txt untuk block filter combinations, 4) Noindex untuk low-value filter pages. Q: Apakah semua halaman produk harus di-index? A: Tidak. Prioritaskan: 1) Best-selling products (index), 2) Out of stock products (noindex atau redirect), 3) Low-quality products (improve atau noindex), 4) Duplicate variants (canonical). Q: Berapa kecepatan loading ideal untuk halaman produk? A: Target: LCP < 2.5 detik, INP < 200ms, CLS < 0.1. Setiap 1 detik delay = 7% drop conversion rate.

Butuh Bantuan Technical SEO untuk E-Commerce?

Technical SEO untuk e-commerce sangat complex. Satu kesalahan kecil (misal: accidentally noindex semua product pages) bisa menghancurkan traffic organik Anda. Kami menyediakan Technical SEO Services khusus untuk e-commerce dengan focus pada crawl budget optimization, duplicate content handling, dan Core Web Vitals improvement.

Konsultasi Gratis untuk audit technical SEO toko online Anda.

Pelajari Artikel Terkait:

Butuh Bantuan SEO Profesional?

Tim ahli kami siap membantu website Anda ranking di halaman 1 Google.

Konsultasi Gratis arrow_forward SEO Audit