sync

Refreshes SQLite from one or both archive sources.

By default, sync runs both live/local sources and does not import the Git snapshot first:

• Discord bot-token sync for bot-visible guild data
• local Discord Desktop cache import for classifiable cached messages and proven DMs

Use update when you want to pull/import the shared Git snapshot. Routine imports upsert changed shards without deleting local cache rows. If you intentionally want a sync run to import the snapshot before live deltas, pass --update=auto for the configured stale update mode (merge by default) or --update=force for an exact replacement. --no-update is accepted as an explicit no-op alias for the default.

Run one explicit --full pass when you want a complete historical guild archive. Use plain sync afterward for frequent latest-message and desktop-cache refreshes.

#Usage

discrawl sync
discrawl sync --update=auto
discrawl sync --update=force
discrawl sync --no-update
discrawl sync --full
discrawl sync --full --all
discrawl sync --guild 123456789012345678
discrawl sync --guilds 123,456 --concurrency 8
discrawl sync --source both      # default: bot API + desktop cache
discrawl sync --source discord   # bot API only; aliases: key, bot, api
discrawl sync --source wiretap   # desktop cache only; aliases: desktop, cache
discrawl sync --guild 123456789012345678 --all-channels
discrawl sync --channels 111,222 --since 2026-03-01T00:00:00Z
discrawl sync --with-embeddings
discrawl sync --with-media

#Sources

#Bot sync modes

#Flags

• --source <both|discord|wiretap> - which archive sources to read
• --update <auto|force|never> - apply the configured stale snapshot update mode, force an exact replacement, or skip snapshot import before live deltas
• --full - historical backfill (slow on large guilds)
• --all-channels - broader incremental sweep across every stored channel/thread
• --latest-only - explicit latest-only run (also the default for untargeted sync)
• --all - ignore default_guild_id and fan out across every discovered guild
• --guild <id> / --guilds <id,id> - target specific guilds
• --channels <id,id> - target specific channels (forum ids expand to threads)
• --since <RFC3339> - limit initial history and --full backfill to messages at or after this timestamp
• --concurrency <n> - override worker count (default auto-sized: floor 8, cap 32)
• --skip-members - refresh guild/channel/message data without crawling members
• --with-members - refresh guild members even during the default latest-only sync; fail if the member crawl cannot complete
• --with-embeddings - also enqueue changed messages into embedding_jobs
• --with-media - after sync, download missing attachment media into cache_dir/media

#Notes

• --latest-only is the default for untargeted sync. Use --all-channels to opt out without doing a full historical crawl.
• --with-media records expired or removed Discord CDN URLs as failed fetches with the HTTP status, commonly 404.
• --with-media updates the local cache only; run publish --push afterward to include cached non-DM media in the Git backup as gzip-compressed files.
• --since does not mark older history as complete, so a later sync --full without --since can continue the backfill.
• Targeted bot syncs (--source discord --channels <ids>) refresh Desktop-imported channel placeholders and their parent metadata from Discord before crawling messages. Unavailable metadata or missing parent or permission evidence fails the sync visibly; ordinary cached channels keep their existing behavior. Ancestor metadata refreshes do not crawl ancestor messages or reset history and archive cursors. This is still a message sync: --since limits initial history and backfill, not fetching new messages after an existing cursor.
• Long runs emit periodic progress logs to stderr.
• Heartbeat logs (message sync waiting) name the oldest active channel and per-channel page activity if in-flight channels stop completing for a while.
• Every run ends with a message sync finished summary.
• Each channel crawl has a bounded runtime budget; pathological channels are deferred and retried next sync.
• Guild and archived-thread pagination reports a cursor error if Discord repeats a page instead of continuing indefinitely.
• Full message pages with missing or repeated cursors stop the channel crawl with an error and preserve its last usable backfill checkpoint.
• Retryable failures and unavailable-channel markers are tracked per channel; stale unavailable markers are cleared after a later successful crawl.
• A channel that fails with missing access carries an unavailable marker, and routine syncs pass over it while the marker is inside its seven-day window. The next routine sync after the marker reaches seven days retries the channel. To retry sooner: sync --guild <id> --full attempts every channel in the guild immediately, marked ones included, and sync --channels <id> retries a single channel immediately. doctor reports unavailable_markers_active (inside the window, so currently passed over) and unavailable_markers_expired (due for another attempt).
• Full sync retries marked channels even when their older history is complete. When only unavailable channels remain in the cached backlog, it also discovers new channels; the retry and discovery phases do not request the same channel twice.
• Marker cleanup is best-effort, so one missing local sync-state row cannot crash the run.
• Member refresh is best-effort and gives up after five minutes without a caller-supplied deadline. Routine latest-only syncs skip it unless --with-members is set.
• Routine refreshes keep a per-parent archived-thread cursor, so they discover threads archived between runs without rescanning the historical thread catalog.
• When the archive is already complete, sync --full reuses backlog markers and the same incremental thread discovery instead of revisiting every stored archived thread.
• Completed channels with no stored messages are verified with a full crawl when Discord still reports a last message. Routine sync skips channels verified empty; --full rechecks them. A --since window does not initiate this recovery.
• Interrupted recovery resumes from its own saved page checkpoint on the next unwindowed sync, including after cancellation or process termination. Messages ingested in the meantime do not mark the older history complete.
• Failed recovery leaves partial history resumable. After cancellation, restoring a still-empty channel's completion marker uses the existing five-second failure-cleanup budget.

#See also

• Sync sources
• tail
• update