Components

This page describes each component of the Code Search system in detail.

API Server

The API server is the central hub for all client interactions.

Responsibilities

• Serve REST API endpoints
• Handle search requests
• Manage connections and repositories
• Queue indexing jobs
• Serve the web UI

Endpoints

All API endpoints are prefixed with /api/v1/:

Endpoint	Description
/api/v1/search	Code search
/api/v1/repos	Repository management
/api/v1/connections	Connection management
/api/v1/jobs	Job monitoring

Configuration

server:
  addr: ":8080"
  read_timeout: 15s
  write_timeout: 60s

Health Check

GET /health

Returns 200 OK when healthy.

Indexer Service

The indexer runs as a separate process and handles all repository operations.

Responsibilities

• Discover repositories from connections
• Clone and update repositories
• Build search indexes
• Process job queue

Components

Scheduler

Manages periodic tasks:

• Connection syncing (repository discovery)
• Repository updates (git fetch)
• Index rebuilding

Worker Pool

Concurrent job processing:

• Configurable worker count
• Job prioritization
• Failure handling

Cloner

Git operations:

• Initial clone
• Incremental fetch
• Branch management

Index Builder

Search index creation:

• File analysis
• Language detection
• Zoekt index building

Configuration

indexer:
  concurrency: 2
  index_path: "./data/index"
  repos_path: "./data/repos"
  reindex_interval: 1h
  zoekt_bin: "zoekt-git-index"

Zoekt

Zoekt is the search engine powering Code Search.

What is Zoekt?

Zoekt is a fast text search engine intended for source code, originally developed by Google. It uses trigram indexing for fast substring matches.

How It Works

1. Indexing: Files are split into trigrams (3-character sequences)
2. Index: Trigrams are stored in an inverted index
3. Search: Query trigrams are matched against the index
4. Ranking: Results are ranked by match quality

Index Structure

./data/
├── repos/
│   ├── github.com/
│   │   └── myorg/
│   │       └── api/
│   │           └── .git/
│   └── gitlab.com/
│       └── ...
└── index/
    └── myorg_api_v1.zoekt

Configuration

zoekt:
  url: "http://localhost:6070"
  index_path: "./data/index"
  shards: 0

Database (PostgreSQL / MySQL)

Code Search supports both PostgreSQL and MySQL for storing metadata. The database driver is auto-detected from the connection URL.

Schema

Connections Table

CREATE TABLE connections (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255) UNIQUE NOT NULL,
    type VARCHAR(50) NOT NULL,
    url TEXT,
    token_encrypted BYTEA,
    config JSONB,
    status VARCHAR(50),
    last_synced_at TIMESTAMP,
    created_at TIMESTAMP DEFAULT NOW()
);

Repositories Table

CREATE TABLE repositories (
    id SERIAL PRIMARY KEY,
    connection_id INTEGER REFERENCES connections(id),
    name VARCHAR(255) NOT NULL,
    url TEXT NOT NULL,
    default_branch VARCHAR(255),
    status VARCHAR(50),
    last_indexed_at TIMESTAMP,
    created_at TIMESTAMP DEFAULT NOW()
);

Jobs Table

CREATE TABLE jobs (
    id SERIAL PRIMARY KEY,
    type VARCHAR(50) NOT NULL,
    status VARCHAR(50) NOT NULL,
    repository_id INTEGER REFERENCES repositories(id),
    payload JSONB,
    error TEXT,
    created_at TIMESTAMP DEFAULT NOW(),
    started_at TIMESTAMP,
    finished_at TIMESTAMP
);

Configuration

database:
  url: "postgres://codesearch:codesearch@localhost:5432/codesearch?sslmode=disable"
  max_open_conns: 25
  max_idle_conns: 5
  conn_max_lifetime: 5m

Redis

Redis provides job queuing and distributed locking.

Job Queue

Jobs are queued in Redis lists:

code-search:queue:pending    # Pending jobs
code-search:queue:running    # Running jobs

Distributed Locks

Prevent concurrent operations on the same repository:

code-search:lock:repo:123   # Lock for repo ID 123

Configuration

redis:
  addr: "localhost:6379"
  password: ""
  db: 0

Web UI

The web UI is a Next.js application.

Pages

Page	Description
/	Search interface
/repos	Repository list
/connections	Connection management
/jobs	Job monitoring

Stack

• Framework: Next.js 14
• Styling: TailwindCSS
• Components: shadcn/ui
• State: React Query

Build

cd web
pnpm install
pnpm build

CLI

The CLI provides command-line access to all features.

Architecture

cmd/cli/
├── main.go
├── cmd/
│   ├── root.go
│   ├── search.go
│   ├── replace.go
│   ├── repo.go
│   ├── find.go
│   └── config.go
└── client/
    └── client.go

Communication

The CLI communicates with the API server via HTTP REST.

// Example API call
func (c *Client) Search(ctx context.Context, query string) (*SearchResponse, error) {
    resp, err := c.http.Get(c.baseURL + "/api/v1/search?q=" + query)
    // ...
}

Next Steps

• Data Flow - Request and indexing flows
• Indexing - How indexing works