DCC AI Readiness Checker

A web application to analyze websites for AI readiness and optimization. Built by firecrawl and extended by the the Data Competence Center Basel-Stadt team.

Note: This is a clone of firecrawl/ai-ready-website.

https://github.com/user-attachments/assets/9df66bdd-4eb4-4c2f-93c5-39b03a34b841

https://github.com/user-attachments/assets/0cfc0728-ac43-4d11-be01-7ae95bd6bdc0

Features:

• AI Discoverability — Full 4-stage pipeline (discover, summarize, select, analyze) to test whether an LLM can find and answer questions about your site, including a page inspector and search results ranking.
• PDF Report Export — Download a detailed PDF report of the analysis results.
• Scrape → Crawl — Deeper site coverage via Firecrawl's crawl.
• Any OpenAI-Compatible API — Works with OpenAI, Groq, Ollama, LM Studio, or any provider that implements the OpenAI standard API.
• Self-Hosted Firecrawl — Compatible with a local Firecrawl instance via FIRECRAWL_API_URL.

Setup

1. Install dependencies:

bun install

2. Copy the environment template and fill in your keys:

cp .env.example .env.local

3. Run the development server:

bun dev

Open http://localhost:3000 to view the application.

Environment Variables

Variable	Required	Description
FIRECRAWL_API_KEY	Yes	Firecrawl API key for web crawling. Get one at firecrawl.dev.
OPENAI_API_KEY	Yes	API key for OpenAI or any OpenAI-compatible LLM provider.
FIRECRAWL_API_URL	No	Custom Firecrawl API URL (e.g. for self-hosted instances). Defaults to Firecrawl cloud.
OPENAI_BASE_URL	No	Override the base URL to use Ollama, Groq, LM Studio, etc. Defaults to OpenAI cloud.
AI_MODEL	No	Model name used for analysis. Defaults to gpt-4o.
AI_MAX_TOKENS	No	Maximum tokens for AI completion responses. Defaults to 4000.
GROQ_API_KEY	No	Groq API key (alternative LLM provider). Used for status checks.

Features

• AI Readiness Analysis for websites
• Real-time scoring and recommendations
• Visual charts and metrics
• SEO and accessibility checks
• PDF report export with detailed analysis results

AI Discoverability

Test whether an LLM can find and answer questions about your website's content. The pipeline runs in four stages:

1. Discover — Crawls the website via Firecrawl and searches the web for the user's question. Crawl depth is configurable (0–50 pages); setting it to 0 enables search-only mode. All search results (including external domains) are tracked with their ranking position.

2. Summarize — Each discovered page is summarized by an LLM in batched parallel calls (8 pages per batch). Produces a concise title, 1–2 sentence summary, and 5–8 keywords per page. Requires OPENAI_API_KEY; falls back to raw previews if unavailable.

3. Select — The LLM reviews all page summaries and selects the most relevant ones for answering the question. Uses summary, keywords, and headings for semantic matching across languages.

4. Analyze — The LLM reads the selected pages' full content and produces an answer with per-page quality scores (relevance, clarity, completeness, accuracy), evidence, and actionable recommendations.

Inspector & Search Visibility

• Page Inspector — Card grid showing every discovered page with a detail modal that reveals exactly what the AI saw and produced at each pipeline stage (Summarize, Select, Analyze).
• Search Results Ranking — Lists all search results (including competitors/external domains) with position numbers, so you can see where your domain ranks for a given query.
• Match Highlighting — Search-sourced pages show query term match counts and highlighted context snippets.

API Routes

Route	Method	Description
/api/discoverability	POST	Crawl + search a website. Returns pages, search results, and domain stats.
/api/discoverability-summarize	POST	Batch-summarize discovered pages via LLM. Returns title, summary, keywords per page.
/api/discoverability-select	POST	LLM page selection based on summaries. Returns selected indices and reasoning.
/api/discoverability-analysis	POST	LLM analysis of selected pages. Returns answer, scores, evidence, recommendations.
/api/check-config	GET	Checks if OpenAI API key is configured.