[`agentic_crew.tools.scraping_tools`](#module-agentic_crew.tools.scraping_tools)
Module Contents
Section titled “Module Contents”Classes
Section titled “Classes”CrawlWebsiteTool | A tool for crawling a website and scraping its content. |
|---|
log |
|---|
agentic_crew.tools.scraping_tools.log
Section titled “agentic_crew.tools.scraping_tools.log”‘getLogger(…)’
class agentic_crew.tools.scraping_tools.CrawlWebsiteTool
Section titled “class agentic_crew.tools.scraping_tools.CrawlWebsiteTool”Bases: crewai_tools.ScrapeWebsiteTool
A tool for crawling a website and scraping its content.
This tool extends ScrapeWebsiteTool to support crawling multiple pages starting from a given URL.
‘CrawlWebsiteTool’
description : str
Section titled “description : str”‘Crawl a website from a given URL and scrape its content.’
_run(url: str) → str
Section titled “_run(url: str) → str”Crawls a website and returns the scraped content.
Args: url: The URL to start crawling from.
Returns: The scraped content of the website.
_scrape_content(soup: bs4.BeautifulSoup) → str
Section titled “_scrape_content(soup: bs4.BeautifulSoup) → str”Scrapes the readable content from a BeautifulSoup object.