Website Lookup Methods
There are three primary ways to integrate website data. Choosing the right method depends on your specific needs regarding data freshness, reliability, and the target website’s structure.1. Live Web Search
This method works similarly to a standard search engine (like Google). You can set it up to include domains, which narrows the results to a specific website, or you can exclude domains to remove certain websites from the search.- Best for:
- Big websites with a lot of pages, like e-commerce sites that can’t be imported.
- Search for general information across the internet.
- Limitations: It relies on the search engine’s indexing and search algorithms.
2. Website Import
This method crawls specific web pages and stores their contents, optionally including those of the subpages, directly in the Typewise database. It features an advanced search algorithm that can find information more reliably and faster.- Best for: Static websites, FAQs or knowledge bases.
- Limitations: This method has a capacity limit of 100 pages with a maximum depth of 10 subpages, after which it stops looking for new pages. It is not suitable for indexing entire large webshops or sites with thousands of pages.
3. Live Website Lookup
This method accesses a specific website in real-time to get the latest information. It supports dynamically generated URLs, e.g. to get an order status. The AI Agent decides how to fill the dynamic parameters automatically based on the context.- How it works: You provide a URL template with placeholders.
- Example 1:
https://www.hotel-search.com/search?minPrice={minimum_price}&maxPrice={maximum_price}(Agent fills in price range) - Example 2:
https://www.online-shop.com/en/search?q={product}(Agent searches for a specific product) - Example 3:
https://www.search-engine.com/en/search/?q={search_query}(Agent inputs general search terms)
- Example 1:
- Best for: Sites with predictable search URL structures (e.g., e-commerce sites, booking platforms). It is highly effective when you need to pass dynamic values like product names, dates, or price ranges.
- Limitations: This method will only analyze a single page of the website. It won’t follow links to other pages.
- Recommendation: Only use this method if you need dynamic parameters or the latest information. If the URL is static, Website Import is usually a better choice.
See also
- Live web search: Search the web in real time
- Website import: Import and index website content
- Live website lookup: Query websites in real-time