Skip to content

Migrations

Generating the schema from the entities is enough for a prototype, but an application that lives for a while needs a history: which changes were applied, in what order, and how to apply the next one on a database that already has data. That is what turso-orm-migration provides. This page explains how migrations are declared, how the migrator applies them, why each one is atomic, and how to run them.

How it works

The migrator keeps a bookkeeping table in your database, turso_migrations(version TEXT PRIMARY KEY, applied_at INTEGER), with one row per applied migration; migration_table_name() on the migrator renames it, for a database shared by several migrators or migrated by another tool before. Every operation starts by making sure that table exists and reading it; then it walks your declared migrations in order and runs the ones whose state does not match: up applies the pending ones, down reverts the applied ones, newest first.

Each migration runs inside its own BEGIN IMMEDIATE transaction, and the bookkeeping insert or delete goes through that same transaction before the commit. Two consequences:

  • Taking the write lock up front means a busy database fails at BEGIN, before any schema change, rather than halfway through.
  • SQLite DDL is transactional, so if a migration fails, its tables, its indexes and its version row are all rolled back together. The database is left exactly as before, and the migrator can be run again after the fix. There is never a "version 3 applied but table missing" state.

Declaring a migration

A migration is a module whose name carries its version, holding a struct that implements MigrationTrait. The DeriveMigrationName derive takes the migration name from the module path, so every struct can simply be called Migration and the file name is the single source of truth for ordering:

//! Creates the `post` table.

use turso_orm_migration::prelude::*;

/// The migration.
#[derive(DeriveMigrationName)]
pub struct Migration;

#[async_trait]
impl MigrationTrait for Migration {
    async fn up(&self, manager: &SchemaManager<'_>) -> Result<(), DbErr> {
        manager
            .create_table(
                Table::create()
                    .table("post")
                    .if_not_exists()
                    .col(ColumnDef::integer("id").primary_key().auto_increment())
                    .col(ColumnDef::text("title").not_null())
                    .col(ColumnDef::text("text").not_null()),
            )
            .await
    }

    async fn down(&self, manager: &SchemaManager<'_>) -> Result<(), DbErr> {
        manager.drop_table(Table::drop().table("post")).await
    }
}

The naming convention is m<yyyymmdd>_<seq>_<what>: a date, a sequence number for several migrations on the same day, and a description. Keep the three parts: the migrator compares names as strings to know which ones ran.

up receives a SchemaManager, a thin view over the migration's transaction. It offers the DDL operations and a few catalog lookups:

Method What it does
create_table(Table::create()...) Runs a CREATE TABLE built with the SQL builder.
alter_table(Table::alter()...) ALTER TABLE, including Turso's ALTER COLUMN ... TO and column renames.
drop_table(Table::drop()...) DROP TABLE.
create_index(CreateIndex::new()...), drop_index(...) Indexes, including Turso's USING fts.
has_table(name), has_column(table, column), has_index(table, name) Inspect the catalog through sqlite_schema and pragma_table_info.
get_connection() The transaction itself, for anything else.

down reverts what up did. It has a default implementation that fails with DbErr::Migration, so a migration that cannot be reverted says so explicitly instead of silently doing nothing; a down that is wrong is worse than none.

Data migrations

Because get_connection() hands out the migration's transaction, a migration can also move or seed data, and it benefits from the same atomicity. The entity API works on it as on any connection:

//! Seeds two posts so that the API has something to list on first start.

use entity::post;
use entity::prelude::Post;
use turso_orm::prelude::*;
use turso_orm_migration::prelude::*;

/// The migration.
#[derive(DeriveMigrationName)]
pub struct Migration;

#[async_trait]
impl MigrationTrait for Migration {
    async fn up(&self, manager: &SchemaManager<'_>) -> Result<(), DbErr> {
        // The connection is the migration's own transaction, so the seed is
        // rolled back together with the schema change if anything fails.
        Post::insert_many([
            post::ActiveModel {
                title: Set("Hello, Turso".to_owned()),
                text: Set("The first post, inserted by a migration.".to_owned()),
                ..Default::default()
            },
            post::ActiveModel {
                title: Set("Second post".to_owned()),
                text: Set("Also seeded.".to_owned()),
                ..Default::default()
            },
        ])
        .exec(manager.get_connection())
        .await?;
        Ok(())
    }

    async fn down(&self, manager: &SchemaManager<'_>) -> Result<(), DbErr> {
        Post::delete_many()
            .filter(post::Column::Title.is_in(["Hello, Turso", "Second post"]))
            .exec(manager.get_connection())
            .await?;
        Ok(())
    }
}

Two details are worth noting. The seed uses insert_many through the entity, so the column list comes from the model and cannot drift from the code. And down deletes exactly the rows up inserted, filtered by title, rather than emptying the table, which may hold user data by then.

A migration and the entity it uses can drift

A migration refers to the entity as it is today, but the migration describes the schema as it was then. If a later migration adds a required column to post, this seed would start failing on a fresh database. Data migrations through the entity are convenient early in a project; for a schema that keeps changing, prefer explicit SQL or a copy of the model frozen inside the migration module.

The migrator

The migrator is a type that lists the migrations, oldest first. That list is the only thing to implement; up, down, status and the rest come from MigratorTrait:

//! The migrator of the axum example and its migrations, oldest first.
//!
//! Each migration lives in a module named `m<date>_<seq>_<what>`; the
//! `DeriveMigrationName` derive takes the migration name from that module,
//! so every struct is simply called `Migration`.

pub use turso_orm_migration::prelude::*;

mod m20240101_000001_create_post_table;
mod m20240101_000002_seed_posts;

/// The migrator; `migrations` is the only method to implement.
pub struct Migrator;

#[async_trait]
impl MigratorTrait for Migrator {
    fn migrations() -> Vec<Box<dyn MigrationTrait>> {
        vec![
            Box::new(m20240101_000001_create_post_table::Migration),
            Box::new(m20240101_000002_seed_posts::Migration),
        ]
    }
}
Method Effect
up(db, None) / up(db, Some(n)) Applies every pending migration, or the first n.
down(db, None) / down(db, Some(n)) Reverts every applied migration, or the last n, newest first.
status(db) Each declared migration with an applied flag, in order.
refresh(db) down everything, then up: rebuilds the schema through the migrations.
fresh(db) Drops every table including the bookkeeping one, then up. For development databases.
reset(db) down everything and stop.

A migration that is in the bookkeeping table but no longer in the list is left alone; one that is in the list but not in the table is pending.

Running migrations

The migrator is a library: how and when it runs is your decision. The web examples call Migrator::up(&db, None) at start-up, before the server binds its port, which is right for a service that owns its database. They also ship a small binary for operating by hand:

//! A minimal migration CLI: `up`, `down`, `status`, `fresh`, `refresh`, `reset`.
//!
//! The database path comes from `DATABASE_URL` in the environment or in the
//! `.env` file next to the workspace root, like the server.
//!
//! ```text
//! cargo run -p migration -- status
//! ```

#![allow(clippy::print_stdout, reason = "a CLI reports on stdout")]

use migration::{Migrator, MigratorTrait};
use turso_orm_migration::prelude::*;

/// Parses the single command argument and runs it against the database.
#[tokio::main]
async fn main() -> Result<(), DbErr> {
    dotenvy::dotenv().ok();
    let url = std::env::var("DATABASE_URL").unwrap_or_else(|_| "axum_example.db".to_owned());
    let db = Database::connect(ConnectOptions::new(url)).await?;
    let command = std::env::args().nth(1).unwrap_or_else(|| "up".to_owned());
    match command.as_str() {
        "up" => Migrator::up(&db, None).await?,
        "down" => Migrator::down(&db, Some(1)).await?,
        "fresh" => Migrator::fresh(&db).await?,
        "refresh" => Migrator::refresh(&db).await?,
        "reset" => Migrator::reset(&db).await?,
        "status" => {}
        other => {
            eprintln!(
                "unknown command `{other}`; expected up, down, fresh, refresh, reset or status"
            );
            std::process::exit(2);
        }
    }
    for status in Migrator::status(&db).await? {
        let mark = if status.applied { "applied" } else { "pending" };
        println!("{mark:8} {}", status.name);
    }
    Ok(())
}

Running migrations from a dedicated binary rather than at start-up is the better fit when several instances share one database file, or when a human should see the status before anything changes.

Starting from the entities

The DDL builders and the entity schema generator produce the same statements, so the first migration of a project can be written in terms of the entities:

/// Creates the three tables and the index declared on `cake`, in dependency order.
///
/// Foreign keys are derived from the `belongs_to` relations and the DDL is
/// `STRICT`-compatible, so the generated statements are what a migration
/// would contain.
async fn create_schema(db: &Database) -> Result<(), DbErr> {
    let schema = Schema::new();
    for stmt in [
        schema.create_table_from_entity(Bakery).to_statement(),
        schema.create_table_from_entity(Cake).to_statement(),
        schema.create_table_from_entity(Fruit).to_statement(),
    ] {
        println!("{}", stmt.sql);
        db.execute(stmt).await?;
    }
    for idx in schema.create_index_from_entity(Cake) {
        let stmt = idx.to_statement();
        println!("{}", stmt.sql);
        db.execute(stmt).await?;
    }
    println!();
    Ok(())
}

In a migration, each to_statement() would be executed through manager.get_connection(). From the second migration on, write the change explicitly: the entity now describes the target state, not the step from the previous one.

Turso specifics

CREATE VIEW IF NOT EXISTS is not idempotent in Turso 0.8, and PRAGMA defer_foreign_keys and foreign_key_check are unsupported. Create referenced tables before the tables that reference them, and drop them in the reverse order in down.