Web Crawler Documentation

Meet Jakka Bot

Your tireless guardian. A quiet force that crawls the web, finding issues before your users do. Polite, thorough, and always respectful of your server.

USER AGENT

jakka-bot

USER AGENT

jakka-bot

What Jakka Bot Does

Our crawler visits your pages, runs 530+ tests, and reports back with actionable insights.

Full Site Discovery

Starts from your homepage, follows links, and reads sitemaps to find every page that matters.

Full Site Discovery

Starts from your homepage, follows links, and reads sitemaps to find every page that matters.

JavaScript Rendering

Uses headless Chromium to render SPAs, React, Vue, and Angular apps just like a real browser.

JavaScript Rendering

Uses headless Chromium to render SPAs, React, Vue, and Angular apps just like a real browser.

530+ Automated Tests

SEO, accessibility, performance, security, UX, and content checks on every page it visits.

530+ Automated Tests

SEO, accessibility, performance, security, UX, and content checks on every page it visits.

Screenshot Capture

Takes visual snapshots of each page for layout analysis and visual regression testing.

Screenshot Capture

Takes visual snapshots of each page for layout analysis and visual regression testing.

Asset Tracking

Takes visual snapshots of each page for layout analysis and visual regression testing.

Asset Tracking

Takes visual snapshots of each page for layout analysis and visual regression testing.

Security Analysis

Checks HTTPS, security headers, mixed content, and common vulnerability patterns.

Security Analysis

Checks HTTPS, security headers, mixed content, and common vulnerability patterns.

Context-aware AI that knows your voice

Most AI tools generate bland, generic copy. AI Studio learns your brand rules, personas, and tone before suggesting anything.

Respects robots.txt

By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.

Respects robots.txt

By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.

Respects robots.txt

By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.

Respects robots.txt

By default, Jakka Bot obeys your robots.txt rules and avoids restricted areas. Only verified site owners can override this for their own properties.

Honors
Crawl-Delay

When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.

Honors
Crawl-Delay

When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.

Honors
Crawl-Delay

When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.

Honors
Crawl-Delay

When you set a crawl-delay in robots.txt, we strictly follow it for HTML page requests. Your rate limits are respected.

Adaptive Speed Control

If we encounter 4xx or 5xx errors, we automatically reduce crawl speed. We back off when your server is under stress.

Adaptive Speed Control

If we encounter 4xx or 5xx errors, we automatically reduce crawl speed. We back off when your server is under stress.

Adaptive Speed Control

If we encounter 4xx or 5xx errors, we automatically reduce crawl speed. We back off when your server is under stress.

Adaptive Speed Control

If we encounter 4xx or 5xx errors, we automatically reduce crawl speed. We back off when your server is under stress.

Concurrent Request Limits

We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.

Concurrent Request Limits

We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.

Concurrent Request Limits

We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.

Concurrent Request Limits

We limit parallel requests to avoid overwhelming your server. Static resources are cached to reduce repeat fetches.

Context-aware AI that knows your voice

Most AI tools generate bland, generic copy. AI Studio learns your brand rules, personas, and tone before suggesting anything.

Crawler Identity

User-Agent String

jakka-bot

Full
User-Agent

Mozilla/5.0 (compatible; jakka-bot/1.0; +https://jakka.ai/jakka-bot)

Rendering Engine

Chromium (headless)

JavaScript

Enabled

Crawl Behavior

Default Delay

1 second between pages

Respects robots.txt

Yes (default)

Crawl-Delay Support

Yes

Error Backoff

Automatic on 4xx/5xx

robots.txt Configuration

Control how Jakka Bot crawls your site

⦿ Allow Jakka Bot (Default)

copy

# Allow Jakka Bot to crawl your site User-agent: jakka-bot Allow: /

# Optional: Set a crawl delay (in seconds) Crawl-delay: 2

# Point to your sitemap Sitemap: https://yoursite.com/sitemap.xml

⦿ Block Jakka Bot

copy

# Block Jakka Bot from your entire site User-agent: jakka-bot Disallow: /

# Or block specific sections User-agent: jakka-bot Disallow: /admin/ Disallow: /private/ Disallow: /staging/

IP Addresses for Whitelisting

Download List

If you need to whitelist Jakka Bot in your firewall, use these IP ranges.

52.24.148.0/24

54.186.192.0/24

35.82.64.0/24

44.234.128.0/24

52.40.96.0/24

54.200.64.0/24

35.167.32.0/24

44.226.192.0/24

IP addresses may change. We recommend checking this page periodically or subscribing to our status page for updates.

Verify Your Site Ownership

Verified site owners get additional control over how Jakka Bot crawls their properties. Unlock the ability to crawl areas normally blocked by robots.txt and access advanced crawl settings.

Crawl restricted areas (with your permission)

Custom crawl speed settings

Test password-protected staging sites

Verify Your Site

Frequently Asked Questions

Common questions about Jakka Bot

Why is Jakka Bot crawling my site?

Will Jakka Bot slow down my server?

Why does Jakka Bot load JavaScript?

Can I request crawl data be deleted?

Does Jakka Bot respect noindex/nofollow?

How do I report abusive crawling?

Why is Jakka Bot crawling my site?

Will Jakka Bot slow down my server?

Why does Jakka Bot load JavaScript?

Can I request crawl data be deleted?

Does Jakka Bot respect noindex/nofollow?

How do I report abusive crawling?

Buy template