# Scrape, crawl, map and search the web with AnakinTools

> Agno adds AnakinTools, a toolkit for the Anakin.io scrape, crawl, map and search APIs. Import it from agno.tools.anakin and set ANAKIN_API_KEY.

- Published: 2026-10-12
- Author: aryan-anakin
- Categories: Changelog
- Canonical: https://www.agno.com/blog/scrape-crawl-map-and-search-the-web-with-anakintools
- Markdown: https://www.agno.com/blog/scrape-crawl-map-and-search-the-web-with-anakintools.md

Agno's `AnakinTools` sends scrape, crawl and map requests to Anakin.io as jobs, then polls each job every `poll_interval` seconds (2 by default) until it completes, so the model gets page content or links back from one tool call. Search is a single synchronous request.

| Method          | Flag            | Anakin endpoint     |
| --------------- | --------------- | ------------------- |
| `scrape_url`    | `enable_scrape` | `POST /url-scraper` |
| `crawl_website` | `enable_crawl`  | `POST /crawl`       |
| `map_website`   | `enable_map`    | `POST /map`         |
| `search_web`    | `enable_search` | `POST /search`      |

Only `scrape_url` is on by default, and `all=True` turns on all four.

```python
from agno.agent import Agent
from agno.tools.anakin import AnakinTools

agent = Agent(tools=[AnakinTools(all=True)], markdown=True)

agent.print_response("Scrape https://docs.agno.com/introduction/ and summarize it")
agent.print_response("List the URLs reachable from https://docs.agno.com")
```

Against a stand-in for the Anakin API, one `scrape_url` call submitted the job and checked it three times before handing the result back.

```text
# real output: AnakinTools(poll_interval=0.01).scrape_url("https://docs.agno.com/introduction")
POST /url-scraper {'url': 'https://docs.agno.com/introduction', 'generateJson': False, 'useBrowser': False, 'country': 'us'}
GET /url-scraper/job_1
GET /url-scraper/job_1
GET /url-scraper/job_1
scrape_url -> {"status": "completed", "markdown": "# Agno\nBuild agents..."}
```

`max_wait_time` (120 seconds by default) caps the wait. A job that fails or runs past it comes back as an error string the model can read.

```text
# real output: AnakinTools(poll_interval=0.01, max_wait_time=0.05) on a job that never finishes
Error scraping https://example.com: Job job_2 was still 'processing' after 0.05s. Retry later or raise max_wait_time.
```

Scrape and crawl route through a proxy in the `country` you set (`"us"` by default), and `use_browser=True` renders JavaScript-heavy pages in a headless browser.

See the [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/91_tools/anakin_tools.py), and learn more about [toolkits](https://docs.agno.com/tools/toolkits/overview) in the documentation.