Skip to main content

Typesense Search - Proof of Concept

What Has Been Done​

I've set up a complete Typesense search integration for your Docusaurus site. Here's what's been configured:

✅ Completed Setup​

  1. Dependencies Installed

    • docusaurus-theme-search-typesense - Docusaurus theme for Typesense
    • typesense - Typesense client library

  2. Docker Configuration

    • docker-compose.yml - Docker Compose file for local Typesense server
    • Configured to run on http://localhost:8108
    • Uses API key: mambu_docs_search_key

  3. Indexing Script

    • scripts/index-to-typesense.js - Complete indexing solution
    • Automatically parses all .md and .mdx files
    • Extracts frontmatter, content, and headings
    • Detects content types (User Guide, API v2, API v1, etc.)
    • Creates weighted search schema

  4. Docusaurus Configuration

    • Updated docusaurus.config.ts to use Typesense theme
    • Commented out old search plugin
    • Configured search parameters with field weights

  5. NPM Scripts

    • npm run typesense:start - Start Typesense server
    • npm run typesense:stop - Stop Typesense server
    • npm run typesense:index - Index all documentation
    • npm run typesense:setup - One-command setup

How to Test​

Step 1: Start Docker​

First, make sure Docker Desktop is running on your Mac:

open -a Docker

Wait for Docker to start completely (check the Docker icon in the menu bar).

Step 2: Start Typesense Server​

npm run typesense:start

Verify it's running:

curl http://localhost:8108/health

Should return: {"ok":true}

Step 3: Index Your Documentation​

npm run typesense:index

This will:

  • Scan all documentation files
  • Parse content and metadata
  • Create search records
  • Upload to Typesense

You should see output like:

🚀 Starting Typesense indexing...
✅ Created new collection with schema
📂 Scanning for documentation files...
Found 250 files
✅ Parsed 245 documents (5 skipped)
📦 Batch 1: 100 indexed, 0 failed
📦 Batch 2: 100 indexed, 0 failed
📦 Batch 3: 45 indexed, 0 failed
✨ Indexing complete!
📊 Total documents indexed: 245

Step 4: Start Docusaurus​

npm run clear
npm run start
  1. Open http://localhost:3000
  2. Click the search box or press Cmd+K
  3. Try these test queries:
    • "deposit account"
    • "create loan"
    • "transaction"
    • "payment channel"

What to Look For​

🎯 Search Quality Indicators​

Relevance:

  • Most relevant results should appear first
  • Title matches should rank higher than content matches
  • User Guide results should appear before API docs (when relevant)

Context:

  • Each result shows a description/preview
  • You can see the content type (User Guide, API v2, etc.)
  • Full path/breadcrumb is visible

Speed:

  • Results should appear instantly (under 100ms)
  • No lag or delay while typing

Features:

  • Typo tolerance: Try "deposite" or "acconut"
  • Partial matching: Try "depo" or "trans"
  • Multi-word queries: Try "create deposit account"

📊 Comparing with Current Search​

Compare these aspects:

AspectCurrent SearchTypesense
Speed?< 100ms
Relevance?Weighted ranking
Typo tolerance?Built-in
Context150 charsCustomizable
GroupingManual JSNative sorting
PerformanceClient-sideServer-side

Search Configuration​

Field Weights​

The search is configured with these weights:

query_by: 'title,description,content,headings'
query_by_weights: '4,3,1,2'

This means:

  • Title (4x) - Highest priority
  • Description (3x) - High priority
  • Headings (2x) - Medium priority
  • Content (1x) - Base priority

Content Type Priority​

Results are sorted by content type priority:

  1. User Guide (priority: 1)
  2. API v2 Reference (priority: 2)
  3. API v1 Reference (priority: 3)
  4. Streaming API (priority: 4)
  5. Functions API (priority: 5)
  6. Payments API (priority: 6)
  7. Other Documentation (priority: 7)

Troubleshooting​

Docker Not Running​

Error: cannot connect to docker daemon

Solution:

open -a Docker
# Wait for Docker to start, then retry
npm run typesense:start

Port Already in Use​

Error: port 8108 is already allocated

Solution:

# Find what's using the port
lsof -i :8108

# Stop existing Typesense container
docker stop mambu-docs-typesense
docker rm mambu-docs-typesense

# Start again
npm run typesense:start

Indexing Fails​

Error: Connection refused or ECONNREFUSED

Solution:

  1. Make sure Typesense is running:

    curl http://localhost:8108/health

  2. Check container logs:

    docker logs mambu-docs-typesense

  3. Restart Typesense:

    npm run typesense:stop
    npm run typesense:start
    sleep 3
    npm run typesense:index

Search Not Working​

Error: Search box doesn't show results

Solution:

  1. Check browser console for errors

  2. Verify collection exists:

    curl http://localhost:8108/collections/mambu_docs \
      -H "X-TYPESENSE-API-KEY: mambu_docs_search_key"

  3. Test search directly:

    curl "http://localhost:8108/collections/mambu_docs/documents/search?q=deposit&query_by=title" \
      -H "X-TYPESENSE-API-KEY: mambu_docs_search_key"

  4. Rebuild Docusaurus:

    npm run clear
    npm run build
    npm run serve

Testing Checklist​

Use this checklist to evaluate the search:

  • Docker is running
  • Typesense server started successfully
  • Indexing completed without errors
  • Docusaurus builds successfully
  • Search box appears on the site
  • Simple query returns results (e.g., "deposit")
  • Results are relevant to the query
  • User Guide results appear first (when applicable)
  • Preview text is helpful and contextual
  • Typo tolerance works (e.g., "deposite")
  • Partial matching works (e.g., "depo")
  • Search is fast (< 100ms)
  • Results look visually organized

Next Steps After Testing​

If Search Quality is Good ✅​

  1. Customize Result Display

    • Enhance visual styling
    • Add content type badges/colors
    • Improve grouping headers

  2. Add Advanced Features

    • Faceted search filters (filter by content type)
    • Search analytics tracking
    • Recent searches
    • Search suggestions

  3. Plan Production Deployment

    • Set up Typesense Cloud or self-hosted server
    • Create production API keys
    • Set up CI/CD for automatic indexing
    • Configure environment variables

If Search Needs Improvement 🔧​

  1. Adjust Field Weights

    • Edit docusaurus.config.ts
    • Change query_by_weights values
    • Re-test search quality

  2. Improve Indexing

    • Edit scripts/index-to-typesense.js
    • Adjust content parsing
    • Add more metadata fields
    • Improve description extraction

  3. Fine-tune Ranking

    • Adjust content type priorities
    • Add boost factors for specific pages
    • Implement custom ranking rules

Files Created/Modified​

New Files​

  • ✅ docker-compose.yml - Typesense server configuration
  • ✅ scripts/index-to-typesense.js - Indexing script
  • ✅ TYPESENSE_POC.md - This file

Modified Files​

  • ✅ package.json - Added Typesense scripts
  • ✅ docusaurus.config.ts - Added Typesense configuration
  • ✅ .gitignore - Added typesense-data directory

Unchanged (for reference)​

  • SEARCH_SOLUTION.md - Previous search implementation
  • SEARCH_IMPROVEMENTS.md - Previous improvements
  • src/client-modules/group-search-results-simple.js - Old grouping logic

Questions to Consider​

After testing, think about:

  1. Is the search faster than the current solution?
  2. Are results more relevant?
  3. Does typo tolerance help users?
  4. Is the preview text more useful?
  5. Do users benefit from content type ordering?
  6. Are there any missing features from the old search?
  7. Is the setup/maintenance complexity worth the improvement?

Resources​

Need Help?​

If you encounter issues:

  1. Check the Troubleshooting section above
  2. Review the indexing script output for errors
  3. Test Typesense directly with curl commands
  4. Check browser console for JavaScript errors
  5. Verify Docker is running and container is healthy

Ready to test? Start with Docker, then run npm run typesense:setup to get everything running!