discord -> sqlite -> answers

Server history you can actually search.

Discrawl mirrors Discord guilds into local SQLite so you can grep, query, and run analytics on org memory without depending on Discord search. Bring a bot token, or read everything offline from a Git snapshot.

Get started View on GitHub

bot api sync

Fan out across every guild a bot can see. Channels, threads, members, attachments, mentions, FTS5 - all into one SQLite file.

desktop wiretap

Read local Discord Desktop cache for classifiable messages and proven DMs. No user token. No selfbot. Auth tokens never extracted.

fts + semantic

unicode61 tokenizer for fast literal search. Optional embeddings (OpenAI, Ollama) for semantic and hybrid recall.

git-backed mirrors

Publish a sharded NDJSON snapshot to a private repo. Readers subscribe, search offline, and never need a bot token.

live tail

Gateway tail keeps the archive warm. Periodic repair sweeps catch anything the live stream missed.

offline analysis

digest, analytics, members, raw read-only sql against the local archive.

Discrawl

Mirror Discord guilds into local SQLite. Search server history without depending on Discord search. Bring a bot token, or read everything offline from a Git snapshot.

#What it does

  • discovers every guild a bot can access and syncs channels, threads, members, and message history into SQLite
  • maintains FTS5 indexes for fast literal search; optional embeddings for semantic and hybrid recall
  • imports classifiable Discord Desktop cache messages with wiretap, including proven DMs under @me
  • tails the Gateway for live updates with periodic repair sweeps
  • publishes the archive as sharded NDJSON snapshots in a private Git repo so readers can search offline with no Discord credentials
  • exposes read-only SQL, channel/member directories, mention queries, digests, and trend analytics

#Pick your path

  • New here? Read Install and run discrawl init.
  • Already have a bot? Jump to sync and search.
  • Just want to read a shared archive? Use subscribe - no token needed.
  • Need DM search? wiretap imports local Discord Desktop cache.
  • Want semantic search? Configure Embeddings, then run embed.

#At a glance

export DISCORD_BOT_TOKEN="..."
discrawl init
discrawl doctor
discrawl sync --full
discrawl search "panic: nil pointer"
discrawl tail

#Sections

  • Start - install, configure, set up the Discord bot, security notes, contact
  • Guides - sync sources, wiretap internals, search modes, embeddings, Git snapshots, data layout
  • Commands - one page per CLI command

#Where to file issues

https://github.com/steipete/discrawl/issues. Contact: steipete@gmail.com.