Intranet crawler

Overview

The Intranet crawler enables your assistants to search and retrieve information from your organization’s internal intranet. Built on the Model Context Protocol (MCP) standard, it gives your team authenticated access to intranet content directly within Intric conversations.

Each customer receives a separate deployment — a dedicated container, persistent volume, vector database, and ingress path — sharing only the container image. Deployments cannot share databases. Onboarding is an engineer-run procedure; there is no self-serve onboarding flow.

The crawler indexes content behind login-protected intranets using credentials you provide during setup. This includes employee handbooks, policy documents, internal wikis, and other resources that require authentication.

Tools and resources

The crawler exposes three MCP tools and one resource:

intranet_get_usage_guide

Returns this usage guide. The crawler’s own instructions tell the assistant to call this tool first in every conversation. Call it before any other crawler tool.

search_intranet

Search indexed intranet content and retrieve relevant information in response to user queries. Every user who can reach the tool sees every indexed page. The index only ever contains what the crawl account could reach during crawling, so careful service account scoping is critical to prevent over-indexing. There is no query-time permission check.

correct_intranet_answer

Correct an answer — not the index. Use this to suggest a better answer for a specific search result. The suggestion is routed to administrators who can review, edit, and approve changes. Accepted corrections are indexed immediately and take precedence over crawled content in future answers.

crawler_stats resource

Returns information about the current crawl state, including pages indexed, last crawl time, and crawl errors.

How it works

The crawler operates as a custom MCP server configured through the crawler’s own admin panel at https://mcp-servers.intric.ai/<customer>-intranet/admin. The panel is Swedish-first with an English toggle, and is protected by HTTP Basic authentication. Once configured:

1. The crawler periodically visits configured URLs on your intranet
2. It authenticates using the credentials you provide
3. Content is extracted, processed, and indexed
4. Assistants with the Intranet crawler tool can search this content
5. Users can suggest corrections through a dedicated interface
6. Administrators review corrections in the crawler’s admin panel

Intranet crawler modes

Standard crawling

The crawler visits your intranet directly on a scheduled basis. Scheduled crawls run daily at 02:00 UTC by default, with an additional incremental news crawl running every 30 minutes. Changing an interval requires a pod restart. This mode requires network access to your intranet and credentials configured during setup.

Pre-crawled data mode

An alternative setup where the crawler never touches your network. Instead, you crawl your own intranet using your own tools and hand over a data file to Intric, who index it. This mode:

• Requires no network access to your intranet from Intric’s side
• Requires no crawl credentials
• Uses manual (not scheduled) updates — send a new data file when content changes

This mode is positioned for security-sensitive customers who cannot grant Intric access to their intranet. Warning: a pre-crawled-mode reindex destroys accepted corrections, as it deletes the vector collection and replaces it with a freshly built one.

Security and permissions

The crawler operates under the permissions of the account configured during setup. This means:

• The index only ever contains content that the crawl account could reach
• All indexed content becomes searchable by every user of an assistant with the tool attached — there is no per-user filtering at query time
• Careful service account scoping is critical to prevent over-indexing

Important: Review your intranet’s permission model before configuring the crawler. If your crawl account has broad access rights, consider whether all indexed content should be searchable by all users of the assistant.

Security classification rules apply to the Intranet crawler tool just as they do to other tools in Intric. An assistant can only use the crawler if it meets the security requirements of the Space it belongs to.

Document citations

When a search result points to a PDF or other document, clicking the citation takes the user to the parent page that linked to the document — not to the document itself. This is by design.

Advanced options

Additional configuration options are available for specific use cases. Contact us for details on:

• Extra crawl sources — index a second authenticated site into the same knowledge base, with its own credentials
• Additional seed URLs — pages that no link on your intranet reaches
• External-link crawling — one hop outside the intranet (default off, capped at 500 pages)
• Blocked-URL quarantine and junk-page purging
• Forced JavaScript rendering
• SharePoint search-based discovery

Next steps

Ready to enable intranet search for your organization?

• Contact us at support@intric.ai to initiate setup and coordinate with our technical team
• Read the article below to understand the setup process and prepare the necessary information.

Installation and configuration guide

Read more