Web Crawler Documentation
Meet Jakka Bot
Your tireless guardian. A quiet force that crawls the web, finding issues before your users do. Polite, thorough, and always respectful of your server.
USER AGENT
jakka-bot
What Jakka Bot Does
What Jakka Bot Does
Our crawler visits your pages, runs 530+ tests, and reports back with actionable insights.
Our crawler visits your pages, runs 530+ tests, and reports back with actionable insights.
Verify Business Accuracy
Check approved services, locations, offers, contact details, business claims and brand information against the delivered content.
Review Requirement Evidence
See the approved instruction beside the observed result and the evidence supporting each finding.
Check Exact Project Instructions
Verify required pages, sections, order, images, logos, CTA wording, destinations, content rules and exclusions.
Correct With Context
Review specific corrections grounded in the customer's business, project, brand and target market.
Test Functional Customer Journeys
Execute required forms, bookings, applications and checkout journeys. Verify successful completion and the required integration outcome.
Verify the Re-test
Check the implemented correction against the original requirement and retain its result for the handover decision.
Verify Business Accuracy
Check approved services, locations, offers, contact details, business claims and brand information against the delivered content.
Check Exact Project Instructions
Verify required pages, sections, order, images, logos, CTA wording, destinations, content rules and exclusions.
Test Functional Customer Journeys
Execute required forms, bookings, applications and checkout journeys. Verify successful completion and the required integration outcome.
Review Requirement Evidence
See the approved instruction beside the observed result and the evidence supporting each finding.
Correct With Context
Review specific corrections grounded in the customer's business, project, brand and target market.
Verify the Re-test
Check the implemented correction against the original requirement and retain its result for the handover decision.
Polite by Design
Jakka Bot is built to be a good citizen of the web. Here's how we ensure minimal impact on your servers.
Respects robots.txt
By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.
Concurrent Request Limits
We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.
Honors
Crawl-Delay
When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.
Technical Specifications
Everything developers need to know about Jakka Bot
Crawler Identity
User-Agent String
jakka-bot
Full
User-Agent
Mozilla/5.0 (compatible; jakka-bot/1.0; +https://jakka.ai/jakka-bot)
Rendering Engine
Chromium (headless)
JavaScript
Enabled
Crawl Behavior
Default Delay
1 second between pages
Respects robots.txt
Yes (default)
Crawl-Delay Support
Yes
Error Backoff
Automatic on 4xx/5xx
robots.txt Configuration
Control how Jakka Bot crawls your site
โฆฟ Allow Jakka Bot (Default)
copy
# Allow Jakka Bot to crawl your site User-agent: jakka-bot
Allow: /
# Optional: Set a crawl delay (in seconds) Crawl-delay: 2
# Point to your sitemap Sitemap: https://yoursite.com/sitemap.xml
โฆฟ Block Jakka Bot
copy
# Block Jakka Bot from your entire site User-agent: jakka-bot
Disallow: /
# Or block specific sections
User-agent: jakka-bot
Disallow: /admin/
Disallow: /private/
Disallow: /staging/
Verify Your Site Ownership
Verified site owners get additional control over how Jakka Bot crawls their properties. Unlock the ability to crawl areas normally blocked by robots.txt and access advanced crawl settings.
Crawl restricted areas (with your permission)
Custom crawl speed settings
Test password-protected staging sites
Polite by Design
Polite by Design
Jakka Bot is built to be a good citizen of the web. Here's how we ensure minimal impact on your servers.
Respects robots.txt
By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.
Concurrent Request Limits
We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.
Honors
Crawl-Delay
When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.
Respects robots.txt
By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.
Concurrent Request Limits
We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.
Honors
Crawl-Delay
When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.
Technical Specifications
Technical Specifications
Everything developers need to know about Jakka Bot
Crawler Identity
User-Agent String
jakka-bot
Full
User-Agent
Mozilla/5.0 (compatible; jakka-bot/1.0; +https://jakka.ai/jakka-bot)
Rendering Engine
Chromium (headless)
JavaScript
Enabled
Crawl Behavior
Default Delay
1 second between pages
Respects robots.txt
Yes (default)
Crawl-Delay Support
Yes
Error Backoff
Automatic on 4xx/5xx
Crawler Identity
User-Agent String
jakka-bot
Full
User-Agent
Mozilla/5.0 (compatible; jakka-bot/1.0; +https://jakka.ai/jakka-bot)
Rendering Engine
Chromium (headless)
JavaScript
Enabled
Crawl Behavior
Default Delay
1 second between pages
Respects robots.txt
Yes (default)
Crawl-Delay Support
Yes
Error Backoff
Automatic on 4xx/5xx
robots.txt Configuration
Control how Jakka Bot crawls your site
โฆฟ Allow Jakka Bot (Default)
copy
# Allow Jakka Bot to crawl your site User-agent: jakka-bot
Allow: /
# Optional: Set a crawl delay (in seconds) Crawl-delay: 2
# Point to your sitemap Sitemap: https://yoursite.com/sitemap.xml
โฆฟ Block Jakka Bot
copy
# Block Jakka Bot from your entire site User-agent: jakka-bot
Disallow: /
# Or block specific sections
User-agent: jakka-bot
Disallow: /admin/
Disallow: /private/
Disallow: /staging/
Verify Your Site Ownership
Verified site owners get additional control over how Jakka Bot crawls their properties. Unlock the ability to crawl areas normally blocked by robots.txt and access advanced crawl settings.
Crawl restricted areas (with your permission)
Custom crawl speed settings
Test password-protected staging sites
robots.txt Configuration
Control how Jakka Bot crawls your site
โฆฟ Allow Jakka Bot (Default)
copy
# Allow Jakka Bot to crawl your site User-agent: jakka-bot
Allow: /
# Optional: Set a crawl delay (in seconds) Crawl-delay: 2
# Point to your sitemap Sitemap: https://yoursite.com/sitemap.xml
โฆฟ Block Jakka Bot
copy
# Block Jakka Bot from your entire site User-agent: jakka-bot
Disallow: /
# Or block specific sections
User-agent: jakka-bot
Disallow: /admin/
Disallow: /private/
Disallow: /staging/
Verify Your Site Ownership
Verified site owners get additional control over how Jakka Bot crawls their properties. Unlock the ability to crawl areas normally blocked by robots.txt and access advanced crawl settings.
Crawl restricted areas (with your permission)
Custom crawl speed settings
Test password-protected staging sites
Frequently Asked Questions
Common questions about Jakka Bot
Why is Jakka Bot crawling my site?
Will Jakka Bot slow down my server?
Why does Jakka Bot load JavaScript?
Can I request crawl data be deleted?
Does Jakka Bot respect noindex/nofollow?
How do I report abusive crawling?
Common questions about Jakka Bot
Frequently Asked Questions
Why is Jakka Bot crawling my site?
Will Jakka Bot slow down my server?
Why does Jakka Bot load JavaScript?
Can I request crawl data be deleted?
Does Jakka Bot respect noindex/nofollow?
How do I report abusive crawling?
Frequently Asked Questions
Common questions about Jakka Bot
Why is Jakka Bot crawling my site?
Will Jakka Bot slow down my server?
Why does Jakka Bot load JavaScript?
Can I request crawl data be deleted?
Does Jakka Bot respect noindex/nofollow?
How do I report abusive crawling?
