Skip to content

feat: add Database.merge() and sqlite-utils merge command#724

Open
abhiyadav2345 wants to merge 1 commit intosimonw:mainfrom
abhiyadav2345:feature/merge-databases
Open

feat: add Database.merge() and sqlite-utils merge command#724
abhiyadav2345 wants to merge 1 commit intosimonw:mainfrom
abhiyadav2345:feature/merge-databases

Conversation

@abhiyadav2345
Copy link

@abhiyadav2345 abhiyadav2345 commented Mar 22, 2026

Closes #491.

Adds the ability to merge tables from one or more source SQLite databases into a destination database.

Python API

# Merge all tables from two source databases
db.merge(["one.db", "two.db"])

# Add missing columns automatically
db.merge(["one.db", "two.db"], alter=True)

# Replace rows with conflicting primary keys
db.merge(["one.db"], replace=True)

# Skip rows with conflicting primary keys
db.merge(["one.db"], ignore=True)

# Only merge specific tables
db.merge(["one.db", "two.db"], tables=["cats", "dogs"])

# Source can be a Database object or a file path
db.merge([other_db, "another.db"])

CLI

sqlite-utils merge combined.db one.db two.db
sqlite-utils merge combined.db one.db two.db --alter
sqlite-utils merge combined.db one.db two.db --replace
sqlite-utils merge combined.db one.db two.db --ignore
sqlite-utils merge combined.db one.db two.db --table mytable

Implementation notes

  • Tables not in the destination are created with the source schema and rows
  • Tables that already exist have rows inserted (no-op if same rows with same PKs, unless --replace or --ignore)
  • Primary keys are preserved from the source table by default; --pk overrides this
  • Virtual tables (FTS indexes etc.) are skipped automatically
  • FTS shadow tables (e.g. docs_fts_data) are also skipped via parent name detection
  • merge() returns self for chaining

Tests

17 new tests covering the Python API and CLI — all passing, no regressions in existing 1042 tests.

Docs

Added a Merging databases section to docs/cli.rst with examples for all options.


📚 Documentation preview 📚: https://sqlite-utils--724.org.readthedocs.build/en/724/

Implements the ability to merge tables from one or more source SQLite
databases into a destination database, as requested in simonw#491.

Python API:
    db.merge([src1, src2], alter=True, replace=False, ignore=False, tables=None)
  - source_dbs can be Database objects or file paths
  - Tables not in dest are created; existing tables have rows inserted
  - alter=True adds missing columns to existing destination tables
  - replace=True overwrites rows with matching primary keys
  - ignore=True skips rows with conflicting primary keys
  - tables= limits which tables are merged
  - Virtual tables and their shadow tables are automatically skipped

CLI:
    sqlite-utils merge combined.db one.db two.db [options]
  - Supports --alter, --replace, --ignore, --pk, --table, --load-extension

Closes simonw#491
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ability to merge databases and tables

1 participant