wasm-dbms

What is wasm-dbms?

wasm-dbms is an embeddable relational database engine written in Rust, designed to run entirely inside WebAssembly runtimes. Unlike traditional databases that run as external services, wasm-dbms compiles into your WASM module and manages data directly in linear or stable memory — no network calls, no external dependencies.

You define your schema as Rust structs with derive macros, and wasm-dbms provides full CRUD operations, ACID transactions, foreign key integrity, validation, and sanitization — all running within the sandbox of your WASM module.

wasm-dbms supports any WASM runtime (Wasmtime, Wasmer, WasmEdge) and offers first-class integration with Internet Computer canisters through the ic-dbms adapter.

Documentation

Guides

Step-by-step guides for building databases with wasm-dbms:

• Getting Started - Set up your first wasm-dbms database
• CRUD Operations - Insert, select, update, and delete records
• Querying - Filters, ordering, pagination, and field selection
• Transactions - ACID transactions with commit/rollback
• Relationships - Foreign keys, delete behaviors, and eager loading
• Custom Data Types - Define your own data types (enums, structs)
• Wasmtime Example - Using wasm-dbms with the WIT Component Model and Wasmtime

Reference

API and type reference documentation:

• Data Types - All supported column types
• Schema Definition - Table attributes and generated types
• Validation - Built-in and custom validators
• Sanitization - Built-in and custom sanitizers
• JSON - JSON data type and filtering
• Errors - Error types and handling

IC Integration

For deploying wasm-dbms as an Internet Computer canister:

• IC Getting Started - Deploy a database canister on the IC
• Access Control - Managing the ACL
• Client API - Using the IC client library

WASI Integration

For deploying wasm-dbms on WASI runtimes (Wasmer, Wasmtime, WasmEdge):

• WASI Memory Provider - File-backed persistent storage for WASI

Technical Documentation

For advanced users and contributors:

• Architecture - Three-layer system overview
• Memory Management - Stable memory internals
• Join Engine - Cross-table join query internals

Quick Example

Define your schema:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[sanitizer(TrimSanitizer)]
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,
    #[validate(EmailValidator)]
    pub email: Text,
}
}

Use the Database trait for CRUD operations:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

// Insert
let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    email: "alice@example.com".into(),
};
database.insert::<User>(user)?;

// Query
let query = Query::builder()
    .filter(Filter::eq("name", Value::Text("Alice".into())))
    .build();
let users = database.select::<User>(query)?;
}

Features

• Schema-driven: Define tables as Rust structs with derive macros
• Runtime-agnostic: Works on any WASM runtime, not tied to a specific platform
• CRUD operations: Full insert, select, update, delete support
• ACID transactions: Commit/rollback with isolation
• Foreign keys: Referential integrity with cascade/restrict behaviors
• Validation & Sanitization: Built-in validators and sanitizers
• JSON support: Store and query semi-structured data
• IC Integration: First-class support for Internet Computer canisters via ic-dbms

Get Started

• Prerequisites
• Project Setup
  • Workspace Structure
  • Cargo Configuration
• Define Your Schema
  • Create the Schema Crate
  • Define Tables
• Define a Database Schema
• Using the Database
  • Create a DbmsContext
  • Perform CRUD Operations
• Quick Example: Complete Workflow
• Testing with HeapMemoryProvider
• Next Steps

This guide walks you through setting up a database using wasm-dbms. By the end, you’ll have a working database with CRUD operations and transactions.

Prerequisites

Before starting, ensure you have:

• Rust 1.91.1 or later
• wasm32-unknown-unknown target: rustup target add wasm32-unknown-unknown

Project Setup

Workspace Structure

We recommend organizing your project as a Cargo workspace with a schema crate:

my-dbms-project/
├── Cargo.toml          # Workspace manifest
├── schema/             # Schema definitions (reusable types)
│   ├── Cargo.toml
│   └── src/
│       └── lib.rs
└── app/                # Your application using the database
    ├── Cargo.toml
    └── src/
        └── lib.rs

Workspace Cargo.toml:

[workspace]
members = ["schema", "app"]
resolver = "2"

Cargo Configuration

Create .cargo/config.toml to configure the getrandom crate for WebAssembly:

[target.wasm32-unknown-unknown]
rustflags = ['--cfg', 'getrandom_backend="custom"']

This is required because the uuid crate depends on getrandom.

Define Your Schema

Create the Schema Crate

Create schema/Cargo.toml:

[package]
name = "my-schema"
version = "0.1.0"
edition = "2024"

[dependencies]
wasm-dbms-api = "0.6"

Define Tables

In schema/src/lib.rs, define your database tables using the Table derive macro:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[sanitizer(TrimSanitizer)]
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,
    #[validate(EmailValidator)]
    pub email: Text,
    pub created_at: DateTime,
}

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uint32,
    #[validate(MaxStrlenValidator(200))]
    pub title: Text,
    pub content: Text,
    pub published: Boolean,
    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,
}
}

Required derives: Table, Clone

The Table macro generates additional types for each table:

Define a Database Schema

Once you’ve defined your tables, create a schema struct with #[derive(DatabaseSchema)] to wire them together:

#![allow(unused)]
fn main() {
use wasm_dbms::prelude::DatabaseSchema;

#[derive(DatabaseSchema)]
#[tables(User = "users", Post = "posts")]
pub struct MySchema;
}

The DatabaseSchema derive macro auto-generates the DatabaseSchema<M> trait implementation and a register_tables method. This replaces what would otherwise be ~130+ lines of manual dispatch code.

Using the Database

Create a DbmsContext

The DbmsContext holds all database state. Create one using a MemoryProvider:

#![allow(unused)]
fn main() {
use wasm_dbms::prelude::*;
use wasm_dbms_api::prelude::*;

// For testing, use HeapMemoryProvider
let ctx = DbmsContext::new(HeapMemoryProvider::default());

// Register tables from the schema
MySchema::register_tables(&ctx).expect("failed to register tables");
}

Perform CRUD Operations

Create a WasmDbmsDatabase from the context to perform operations:

#![allow(unused)]
fn main() {
use wasm_dbms::prelude::*;
use my_schema::{User, UserInsertRequest};

// Create a one-shot (non-transactional) database
let database = WasmDbmsDatabase::oneshot(&ctx, MySchema);

// Insert a record
let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    email: "alice@example.com".into(),
    created_at: DateTime::now(),
};
database.insert::<User>(user)?;

// Query records
let query = Query::builder().all().build();
let users = database.select::<User>(query)?;
}

Quick Example: Complete Workflow

Here’s a complete example showing insert, query, update, and delete operations:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;
use my_schema::{User, UserInsertRequest, UserUpdateRequest};

fn example(database: &impl Database) -> Result<(), DbmsError> {
    // 1. INSERT a new user
    let insert_req = UserInsertRequest {
        id: 1.into(),
        name: "Alice".into(),
        email: "alice@example.com".into(),
        created_at: DateTime::now(),
    };
    database.insert::<User>(insert_req)?;

    // 2. SELECT users
    let query = Query::builder()
        .filter(Filter::eq("name", Value::Text("Alice".into())))
        .build();
    let users = database.select::<User>(query)?;
    println!("Found {} user(s)", users.len());

    // 3. UPDATE the user
    let update_req = UserUpdateRequest::builder()
        .set_email("alice.new@example.com".into())
        .filter(Filter::eq("id", Value::Uint32(1.into())))
        .build();
    let updated = database.update::<User>(update_req)?;
    println!("Updated {} record(s)", updated);

    // 4. DELETE the user
    let deleted = database.delete::<User>(
        DeleteBehavior::Restrict,
        Some(Filter::eq("id", Value::Uint32(1.into()))),
    )?;
    println!("Deleted {} record(s)", deleted);

    Ok(())
}
}

Testing with HeapMemoryProvider

For unit tests, use HeapMemoryProvider which stores data in heap memory:

#![allow(unused)]
fn main() {
use wasm_dbms::prelude::*;
use wasm_dbms_api::prelude::*;
use my_schema::{User, UserInsertRequest, MySchema};

#[test]
fn test_insert_and_select() {
    let ctx = DbmsContext::new(HeapMemoryProvider::default());
    MySchema::register_tables(&ctx).expect("register failed");
    let database = WasmDbmsDatabase::oneshot(&ctx, MySchema);

    let insert_req = UserInsertRequest {
        id: 1.into(),
        name: "Test User".into(),
        email: "test@example.com".into(),
        created_at: DateTime::now(),
    };

    database.insert::<User>(insert_req).expect("insert failed");

    let query = Query::builder().all().build();
    let users = database.select::<User>(query).expect("select failed");

    assert_eq!(users.len(), 1);
    assert_eq!(users[0].name.as_str(), "Test User");
}
}

For deploying on the Internet Computer as a canister, see the IC Getting Started Guide.

Next Steps

Now that you have a working database, explore these topics:

• CRUD Operations - Detailed guide on all database operations
• Querying - Filters, ordering, pagination, and field selection
• Transactions - ACID transactions with commit/rollback
• Relationships - Foreign keys and eager loading
• Custom Data Types - Define your own data types (enums, structs)
• Schema Definition - Complete schema reference
• Data Types - All supported field types

CRUD Operations

• CRUD Operations
  • Overview
  • Insert
    • Basic Insert
    • Handling Primary Keys
    • Nullable Fields
    • Insert with Transaction
  • Select
    • Select All Records
    • Select with Filter
    • Select Specific Columns
    • Select with Eager Loading
  • Update
    • Basic Update
    • Partial Updates
    • Update with Filter
    • Update Return Value
  • Delete
    • Delete with Filter
    • Delete Behaviors
    • Delete All Records
  • Operations with Transactions
  • Error Handling

Overview

wasm-dbms provides four fundamental database operations through the Database trait:

All operations:

• Support optional transaction IDs
• Validate and sanitize data according to schema rules
• Enforce foreign key constraints

Insert

Basic Insert

To insert a record, create an InsertRequest and call the insert method:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;
use my_schema::{User, UserInsertRequest};

let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    email: "alice@example.com".into(),
    created_at: DateTime::now(),
};

// Insert without transaction
database.insert::<User>(user)?;
}

Handling Primary Keys

Every table must have a primary key. Insert will fail if a record with the same primary key already exists:

#![allow(unused)]
fn main() {
// First insert succeeds
database.insert::<User>(user1)?;

// Second insert with same ID fails with PrimaryKeyConflict
let result = database.insert::<User>(user2_same_id);
assert!(matches!(result, Err(DbmsError::Query(QueryError::PrimaryKeyConflict))));
}

Nullable Fields

For fields wrapped in Nullable<T>, you can insert either a value or null:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "profiles"]
pub struct Profile {
    #[primary_key]
    pub id: Uint32,
    pub bio: Nullable<Text>,      // Optional field
    pub website: Nullable<Text>,  // Optional field
}

// Insert with value
let profile = ProfileInsertRequest {
    id: 1.into(),
    bio: Nullable::Value("Hello world".into()),
    website: Nullable::Null,  // No website
};

database.insert::<Profile>(profile)?;
}

Insert with Transaction

To insert within a transaction, use a transactional database instance:

#![allow(unused)]
fn main() {
// Begin transaction
let tx_id = ctx.begin_transaction();

// Create a transactional database
let database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

// Insert within transaction
database.insert::<User>(user)?;

// Commit or rollback
database.commit()?;
}

Select

Select All Records

Use Query::builder().all() to select all records:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

let query = Query::builder().all().build();
let users: Vec<UserRecord> = database.select::<User>(query)?;

for user in users {
    println!("User: {} ({})", user.name, user.email);
}
}

Select with Filter

Add filters to narrow down results:

#![allow(unused)]
fn main() {
// Select users with specific name
let query = Query::builder()
    .filter(Filter::eq("name", Value::Text("Alice".into())))
    .build();

let users = database.select::<User>(query)?;
}

See the Querying Guide for comprehensive filter documentation.

Select Specific Columns

Select only the columns you need:

#![allow(unused)]
fn main() {
let query = Query::builder()
    .columns(vec!["id".to_string(), "name".to_string()])
    .build();

let users = database.select::<User>(query)?;
// Only id and name are populated; other fields have default values
}

Select with Eager Loading

Load related records in a single query using with():

#![allow(unused)]
fn main() {
// Load posts with their authors
let query = Query::builder()
    .all()
    .with("users")  // Eager load the related users table
    .build();

let posts = database.select::<Post>(query)?;
}

See the Relationships Guide for more on eager loading.

Update

Basic Update

Create an UpdateRequest to modify records:

#![allow(unused)]
fn main() {
use my_schema::UserUpdateRequest;

let update = UserUpdateRequest::builder()
    .set_name("Alice Smith".into())
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

let affected_rows = database.update::<User>(update)?;

println!("Updated {} row(s)", affected_rows);
}

Partial Updates

Only specify the fields you want to change. Unspecified fields remain unchanged:

#![allow(unused)]
fn main() {
// Only update the email, keep everything else
let update = UserUpdateRequest::builder()
    .set_email("new.email@example.com".into())
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

database.update::<User>(update)?;
}

Update with Filter

The filter determines which records are updated:

#![allow(unused)]
fn main() {
// Update all users with a specific domain
let update = UserUpdateRequest::builder()
    .set_verified(true.into())
    .filter(Filter::like("email", "%@company.com"))
    .build();

let affected = database.update::<User>(update)?;
println!("Verified {} company users", affected);
}

Update Return Value

Update returns the number of affected rows:

#![allow(unused)]
fn main() {
let affected = database.update::<User>(update)?;

if affected == 0 {
    println!("No records matched the filter");
} else {
    println!("Updated {} record(s)", affected);
}
}

Delete

Delete with Filter

Delete records matching a filter:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::DeleteBehavior;

let filter = Filter::eq("id", Value::Uint32(1.into()));

let deleted = database.delete::<User>(
    DeleteBehavior::Restrict,
    Some(filter),
)?;

println!("Deleted {} record(s)", deleted);
}

Delete Behaviors

When deleting records that are referenced by foreign keys, you must specify a behavior:

Restrict Example:

#![allow(unused)]
fn main() {
// Will fail if any posts reference this user
let result = database.delete::<User>(
    DeleteBehavior::Restrict,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
);

match result {
    Ok(count) => println!("Deleted {} user(s)", count),
    Err(DbmsError::Query(QueryError::ForeignKeyConstraintViolation)) => {
        println!("Cannot delete: user has posts");
    }
    Err(e) => return Err(e),
}
}

Cascade Example:

#![allow(unused)]
fn main() {
// Deletes the user AND all their posts
database.delete::<User>(
    DeleteBehavior::Cascade,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
)?;
}

Delete All Records

Pass None as the filter to delete all records (use with caution):

#![allow(unused)]
fn main() {
// Delete ALL users (respecting foreign key behavior)
let deleted = database.delete::<User>(
    DeleteBehavior::Cascade,
    None,  // No filter = all records
)?;

println!("Deleted all {} users and their related records", deleted);
}

Operations with Transactions

All CRUD operations can be performed within a transaction. When using a transactional database instance, operations won’t be visible to other callers until committed:

#![allow(unused)]
fn main() {
// Begin transaction
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

// Perform operations within transaction
database.insert::<User>(user1)?;
database.insert::<User>(user2)?;

// Update within same transaction
let update = UserUpdateRequest::builder()
    .set_verified(true.into())
    .filter(Filter::all())
    .build();
database.update::<User>(update)?;

// Commit all changes atomically
database.commit()?;
}

See the Transactions Guide for comprehensive transaction documentation.

Error Handling

CRUD operations can fail for various reasons. Here are common errors:

Example error handling:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::{DbmsError, QueryError};

let result = database.insert::<User>(user);

match result {
    Ok(()) => println!("Insert successful"),
    Err(DbmsError::Query(QueryError::PrimaryKeyConflict)) => {
        println!("User with this ID already exists");
    }
    Err(DbmsError::Query(QueryError::BrokenForeignKeyReference)) => {
        println!("Referenced record does not exist");
    }
    Err(DbmsError::Validation(msg)) => {
        println!("Validation failed: {}", msg);
    }
    Err(e) => {
        println!("Unexpected error: {:?}", e);
    }
}
}

See the Errors Reference for complete error documentation.

For IC canister client usage with the IcDbmsCanisterClient, see the IC CRUD Guide.

Querying

• Querying
  • Overview
  • Query Builder
    • Basic Queries
    • Query Structure
  • Filters
    • Comparison Filters
    • List Membership
    • Pattern Matching
    • Null Checks
    • Combining Filters
  • JSON Filters
  • Ordering
    • Single Column Ordering
    • Multiple Column Ordering
  • Pagination
    • Limit
    • Offset
    • Pagination Pattern
  • Field Selection
    • Select All Fields
    • Select Specific Fields
  • Eager Loading
  • Joins
    • Join Types
    • Basic Join
    • Left, Right, and Full Joins
    • Chaining Multiple Joins
    • Qualified Column Names
    • Joins vs Eager Loading
• Index-Accelerated Queries
  • How Indexes Improve Queries
  • Which Filters Use Indexes
  • Residual Filters
  • Transaction-Aware Lookups

Overview

wasm-dbms provides a powerful query API for retrieving data from your tables. Queries are built using the QueryBuilder and can include:

• Filters - Narrow down which records to return
• Ordering - Sort results by one or more columns
• Pagination - Limit results and implement pagination
• Field Selection - Choose which columns to return
• Eager Loading - Load related records in a single query
• Joins - Combine rows from multiple tables

Query Builder

Basic Queries

Use Query::builder() to construct queries:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

// Select all records
let query = Query::builder().all().build();

// Select with filter
let query = Query::builder()
.filter(Filter::eq("status", Value::Text("active".into())))
.build();

// Complex query with multiple options
let query = Query::builder()
.filter(Filter::gt("age", Value::Int32(18.into())))
.order_by("created_at", OrderDirection::Descending)
.limit(10)
.offset(20)
.build();
}

Query Structure

A query consists of these optional components:

Filters

Filters determine which records match your query. All filters are created using the Filter struct.

Comparison Filters

Examples:

#![allow(unused)]
fn main() {
// Find users older than 21
let filter = Filter::gt("age", Value::Int32(21.into()));

// Find products under $50
let filter = Filter::lt("price", Value::Decimal(50.0.into()));

// Find orders from a specific date
let filter = Filter::ge("created_at", Value::DateTime(some_datetime));
}

List Membership

Check if a value is in a list of values:

#![allow(unused)]
fn main() {
// Find users with specific roles
let filter = Filter::in_list("role", vec![
    Value::Text("admin".into()),
    Value::Text("moderator".into()),
    Value::Text("editor".into()),
]);

// Find products in certain categories
let filter = Filter::in_list("category_id", vec![
    Value::Uint32(1.into()),
    Value::Uint32(2.into()),
    Value::Uint32(5.into()),
]);
}

Pattern Matching

Use like for pattern matching with wildcards:

#![allow(unused)]
fn main() {
// Find users whose email ends with @company.com
let filter = Filter::like("email", "%@company.com");

// Find products starting with "Pro"
let filter = Filter::like("name", "Pro%");

// Find codes with pattern XX-###
let filter = Filter::like("code", "__-___");

// Find text containing literal %
let filter = Filter::like("description", "%%25%% off");
}

Null Checks

Check for null or non-null values:

#![allow(unused)]
fn main() {
// Find users without a phone number
let filter = Filter::is_null("phone");

// Find users with a profile picture
let filter = Filter::not_null("avatar_url");
}

Combining Filters

Filters can be combined using logical operators:

AND - Both conditions must match:

#![allow(unused)]
fn main() {
// Active users over 18
let filter = Filter::eq("status", Value::Text("active".into()))
.and(Filter::gt("age", Value::Int32(18.into())));
}

OR - Either condition matches:

#![allow(unused)]
fn main() {
// Admins or moderators
let filter = Filter::eq("role", Value::Text("admin".into()))
.or(Filter::eq("role", Value::Text("moderator".into())));
}

NOT - Negate a condition:

#![allow(unused)]
fn main() {
// Users who are not banned
let filter = Filter::eq("status", Value::Text("banned".into())).not();
}

Complex combinations:

#![allow(unused)]
fn main() {
// (active AND age > 18) OR role = "admin"
let filter = Filter::eq("status", Value::Text("active".into()))
.and(Filter::gt("age", Value::Int32(18.into())))
.or(Filter::eq("role", Value::Text("admin".into())));

// NOT (deleted OR archived)
let filter = Filter::eq("status", Value::Text("deleted".into()))
.or(Filter::eq("status", Value::Text("archived".into())))
.not();
}

JSON Filters

For columns with Json type, use specialized JSON filters. See the JSON Reference for comprehensive documentation.

Quick examples:

#![allow(unused)]
fn main() {
// Check if JSON contains a pattern
let pattern = Json::from_str(r#"{"active": true}"#).unwrap();
let filter = Filter::json("metadata", JsonFilter::contains(pattern));

// Extract and compare a value
let filter = Filter::json(
"settings",
JsonFilter::extract_eq("theme", Value::Text("dark".into()))
);

// Check if a path exists
let filter = Filter::json("data", JsonFilter::has_key("user.email"));
}

Ordering

Single Column Ordering

Sort results by a single column:

#![allow(unused)]
fn main() {
// Sort by name ascending (A-Z)
let query = Query::builder()
.all()
.order_by("name", OrderDirection::Ascending)
.build();

// Sort by created_at descending (newest first)
let query = Query::builder()
.all()
.order_by("created_at", OrderDirection::Descending)
.build();
}

Multiple Column Ordering

Chain multiple order_by calls for secondary sorting:

#![allow(unused)]
fn main() {
// Sort by category, then by price within each category
let query = Query::builder()
.all()
.order_by("category", OrderDirection::Ascending)
.order_by("price", OrderDirection::Descending)
.build();

// Sort by status, then by priority, then by created_at
let query = Query::builder()
.all()
.order_by("status", OrderDirection::Ascending)
.order_by("priority", OrderDirection::Descending)
.order_by("created_at", OrderDirection::Ascending)
.build();
}

Pagination

Limit

Restrict the number of records returned:

#![allow(unused)]
fn main() {
// Get only the first 10 records
let query = Query::builder()
.all()
.limit(10)
.build();
}

Offset

Skip a number of records before returning results:

#![allow(unused)]
fn main() {
// Skip the first 20 records
let query = Query::builder()
.all()
.offset(20)
.build();
}

Pagination Pattern

Combine limit and offset for pagination:

#![allow(unused)]
fn main() {
const PAGE_SIZE: u64 = 20;

fn get_page_query(page: u64) -> Query {
    Query::builder()
        .all()
        .order_by("id", OrderDirection::Ascending)  // Consistent ordering is important
        .limit(PAGE_SIZE)
        .offset(page * PAGE_SIZE)
        .build()
}

// Page 0: records 0-19
let page_0 = get_page_query(0);

// Page 1: records 20-39
let page_1 = get_page_query(1);

// Page 2: records 40-59
let page_2 = get_page_query(2);
}

Tip: Always use order_by with pagination to ensure consistent ordering across pages.

Field Selection

Select All Fields

Use .all() to select all columns:

#![allow(unused)]
fn main() {
let query = Query::builder()
.all()
.build();

let users = database.select::<User>(query)?;
// All fields are populated
}

Select Specific Fields

Use .columns() to select only specific columns:

#![allow(unused)]
fn main() {
let query = Query::builder()
.columns(vec!["id".to_string(), "name".to_string(), "email".to_string()])
.build();

let users = database.select::<User>(query)?;
// Only id, name, and email are populated
// Other fields will have default values
}

Note: The primary key is always included, even if not specified.

Eager Loading

Load related records in a single query using .with():

#![allow(unused)]
fn main() {
// Define tables with foreign key
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,
}

// Query posts with authors eagerly loaded
let query = Query::builder()
.all()
.with("users")
.build();

let posts = database.select::<Post>(query)?;
}

See the Relationships Guide for more on eager loading.

Joins

Joins combine rows from two or more tables based on a related column, producing a single result set with columns from all joined tables. Use joins when you need to correlate data across tables in a single flat result – for example, listing posts alongside their author names.

Note: Joins require the select_join method, which returns rows with [JoinColumnDef] that include the source table name. Typed select::<T> rejects queries that contain joins with a JoinInsideTypedSelect error.

Join Types

Basic Join

Use .inner_join(table, left_column, right_column) to join two tables:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

// Join users with their posts (INNER JOIN)
let query = Query::builder()
    .all()
    .inner_join("posts", "id", "user_id")
    .build();

// Use select_join since joins return rows with table provenance
let rows = database.select_join("users", query)?;

// Each row contains columns from both "users" and "posts"
for row in &rows {
    for (col_def, value) in row {
        // col_def.table tells you which table the column came from
        println!("{}.{} = {:?}", col_def.table.as_deref().unwrap_or("?"), col_def.name, value);
    }
}
}

Left, Right, and Full Joins

#![allow(unused)]
fn main() {
// LEFT JOIN: all users, even those without posts
let query = Query::builder()
    .all()
    .left_join("posts", "id", "user_id")
    .build();

// RIGHT JOIN: all posts, even those with missing/deleted authors
let query = Query::builder()
    .all()
    .right_join("posts", "id", "user_id")
    .build();

// FULL JOIN: all users and all posts, matched where possible
let query = Query::builder()
    .all()
    .full_join("posts", "id", "user_id")
    .build();
}

For LEFT, RIGHT, and FULL joins, columns from the unmatched side are filled with Value::Null.

Chaining Multiple Joins

Chain multiple joins to combine more than two tables:

#![allow(unused)]
fn main() {
// Users -> Posts -> Comments
let query = Query::builder()
    .all()
    .inner_join("posts", "id", "user_id")
    .left_join("comments", "posts.id", "post_id")
    .build();

let rows = database.select_join("users", query)?;
}

Joins are processed left-to-right. The second join operates on the result of the first.

Qualified Column Names

When joining tables that share column names, use table.column syntax to disambiguate:

#![allow(unused)]
fn main() {
// Both "users" and "posts" have an "id" column
let query = Query::builder()
    .field("users.id")
    .field("users.name")
    .field("posts.title")
    .inner_join("posts", "users.id", "user_id")
    .and_where(Filter::eq("users.name", Value::Text("Alice".into())))
    .order_by_asc("posts.title")
    .build();
}

Qualified names (table.column) work in:

• Field selection (.field())
• Filters (.and_where(), .or_where())
• Ordering (.order_by_asc(), .order_by_desc())
• Join ON conditions

Unqualified names default to the FROM table (the table passed to select_join).

Joins vs Eager Loading

Use eager loading when you want typed results with related records attached. Use joins when you need a flat, cross-table result set – for example, for reporting, search, or when you need columns from multiple tables in a single row.

Index-Accelerated Queries

When a table has indexes defined (via #[index] or the automatic primary key index), the query engine can use them to avoid full table scans. This happens transparently — you write the same filters as before, and the engine picks the best available index.

How Indexes Improve Queries

Without indexes, every SELECT, UPDATE, and DELETE scans all records in the table. With an index on the filtered column, the engine navigates the B-tree to locate matching records directly, then loads only those records from memory.

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[index]
    pub email: Text,
    pub name: Text,
}

// This uses the index on `email` — no full table scan
let query = Query::builder()
    .filter(Filter::eq("email", Value::Text("alice@example.com".into())))
    .build();

let users = database.select::<User>(query)?;
}

Which Filters Use Indexes

The filter analyzer extracts an index plan from the leftmost AND-chain of conditions on indexed columns:

Filters that fall back to full scan:

• OR at the top level
• NOT wrapping an indexable condition
• Filters on non-indexed columns
• Complex nested expressions

Residual Filters

When the index narrows down the candidate set but doesn’t fully satisfy the filter, the remaining conditions are applied as a residual check on each loaded record:

#![allow(unused)]
fn main() {
// Index on `email` handles the equality check.
// `name LIKE 'A%'` is applied as a residual filter on the results.
let filter = Filter::eq("email", Value::Text("alice@example.com".into()))
    .and(Filter::like("name", "A%"));
}

Transaction-Aware Lookups

Inside a transaction, index lookups are merged with the transaction overlay. Records added in the current transaction appear in index results, and deleted records are excluded — even though the on-disk B-tree has not been modified yet. On commit, overlay changes are flushed to the persistent B-tree. On rollback, the overlay is discarded and the B-tree remains unchanged.

Relationships

• Relationships
  • Overview
  • Defining Foreign Keys
    • Foreign Key Syntax
    • Foreign Key Constraints
  • Referential Integrity
    • Insert Validation
    • Update Validation
  • Delete Behaviors
    • Restrict
    • Cascade
    • Choosing a Delete Behavior
  • Eager Loading
    • Basic Eager Loading
    • Multiple Relations
    • Eager Loading with Filters
    • Cross-Table Queries with Joins
  • Common Patterns
    • One-to-Many
    • Many-to-Many
    • Self-Referential

Overview

wasm-dbms supports foreign key relationships between tables, providing:

• Referential integrity: Ensures foreign keys point to valid records
• Delete behaviors: Control what happens when referenced records are deleted
• Eager loading: Load related records in a single query

Defining Foreign Keys

Foreign Key Syntax

Use the #[foreign_key] attribute to define relationships:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    pub content: Text,

    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,
}
}

Attribute parameters:

Foreign Key Constraints

When you define a foreign key:

1. The field type must match the referenced column type
2. The referenced table must be registered in your database schema
3. Foreign key values must reference existing records (enforced on insert/update)

Referential Integrity

wasm-dbms enforces referential integrity automatically.

Insert Validation

When inserting a record with a foreign key, the referenced record must exist:

#![allow(unused)]
fn main() {
// This user exists
database.insert::<User>(UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    ..
})?;

// Insert post referencing existing user - OK
database.insert::<Post>(PostInsertRequest {
    id: 1.into(),
    title: "My Post".into(),
    author_id: 1.into(),  // User 1 exists
    ..
})?;

// Insert post referencing non-existent user - FAILS
let result = database.insert::<Post>(PostInsertRequest {
    id: 2.into(),
    title: "Another Post".into(),
    author_id: 999.into(),  // User 999 doesn't exist
    ..
});

assert!(matches!(
    result,
    Err(DbmsError::Query(QueryError::BrokenForeignKeyReference))
));
}

Update Validation

Updates are also validated:

#![allow(unused)]
fn main() {
// Changing author_id to non-existent user fails
let update = PostUpdateRequest::builder()
    .set_author_id(999.into())  // User 999 doesn't exist
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

let result = database.update::<Post>(update);
assert!(matches!(
    result,
    Err(DbmsError::Query(QueryError::BrokenForeignKeyReference))
));
}

Delete Behaviors

When deleting a record that is referenced by other records, you must specify how to handle the references.

Restrict

Behavior: Fail if any records reference this one.

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::DeleteBehavior;

// User has posts - delete fails
let result = database.delete::<User>(
    DeleteBehavior::Restrict,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
);

match result {
    Err(DbmsError::Query(QueryError::ForeignKeyConstraintViolation)) => {
        println!("Cannot delete: user has posts");
    }
    _ => {}
}

// Delete posts first, then user
database.delete::<Post>(
    DeleteBehavior::Restrict,
    Some(Filter::eq("author_id", Value::Uint32(1.into()))),
)?;

// Now user can be deleted
database.delete::<User>(
    DeleteBehavior::Restrict,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
)?;
}

Use when: You want to prevent accidental data loss. The caller must explicitly handle related records.

Cascade

Behavior: Delete all records that reference this one (recursively).

#![allow(unused)]
fn main() {
// Deletes user AND all their posts
database.delete::<User>(
    DeleteBehavior::Cascade,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
)?;
}

Cascade is recursive:

#![allow(unused)]
fn main() {
// Schema:
// User -> Posts -> Comments
// Deleting a user cascades to posts, which cascades to comments

database.delete::<User>(
    DeleteBehavior::Cascade,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
)?;
// User deleted
// All user's posts deleted
// All comments on those posts deleted
}

Use when: Related records have no meaning without the parent (e.g., comments on a deleted post).

Choosing a Delete Behavior

Eager Loading

Eager loading fetches related records in a single query, avoiding N+1 query problems.

Basic Eager Loading

Use .with() to eager load a related table:

#![allow(unused)]
fn main() {
// Load posts with their authors
let query = Query::builder()
    .all()
    .with("users")  // Name of the related table
    .build();

let posts = database.select::<Post>(query)?;

// Each post now has author data available
for post in posts {
    println!("Post '{}' by author_id {}", post.title, post.author_id);
}
}

Multiple Relations

Load multiple related tables:

#![allow(unused)]
fn main() {
// Schema:
// Post -> User (author)
// Post -> Category

let query = Query::builder()
    .all()
    .with("users")
    .with("categories")
    .build();

let posts = database.select::<Post>(query)?;
}

Eager Loading with Filters

Combine eager loading with filters:

#![allow(unused)]
fn main() {
// Load published posts with their authors
let query = Query::builder()
    .filter(Filter::eq("published", Value::Boolean(true)))
    .order_by("created_at", OrderDirection::Descending)
    .limit(10)
    .with("users")
    .build();

let posts = database.select::<Post>(query)?;
}

Cross-Table Queries with Joins

In addition to eager loading, wasm-dbms supports SQL-style joins (INNER, LEFT, RIGHT, FULL) for combining rows from multiple tables into a flat result set. Joins are useful when you need columns from several tables in a single row – for example, listing post titles alongside author names. Unlike eager loading, joins return untyped results via the select_raw path.

See the Querying Guide – Joins section for full details and examples.

Common Patterns

One-to-Many

A user has many posts:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
}

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,
}

// Query all posts by a user
let query = Query::builder()
    .filter(Filter::eq("author_id", Value::Uint32(user_id.into())))
    .build();
let user_posts = database.select::<Post>(query)?;
}

Many-to-Many

Use a junction table for many-to-many relationships:

#![allow(unused)]
fn main() {
// Students and Courses (many-to-many)

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "students"]
pub struct Student {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
}

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "courses"]
pub struct Course {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
}

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "enrollments"]
pub struct Enrollment {
    #[primary_key]
    pub id: Uint32,
    #[foreign_key(entity = "Student", table = "students", column = "id")]
    pub student_id: Uint32,
    #[foreign_key(entity = "Course", table = "courses", column = "id")]
    pub course_id: Uint32,
    pub enrolled_at: DateTime,
}

// Find all courses for a student
let query = Query::builder()
    .filter(Filter::eq("student_id", Value::Uint32(student_id.into())))
    .with("courses")
    .build();
let enrollments = database.select::<Enrollment>(query)?;
}

Self-Referential

A table can reference itself (e.g., categories with parent categories, employees with managers):

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "employees"]
pub struct Employee {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    #[foreign_key(entity = "Employee", table = "employees", column = "id")]
    pub manager_id: Nullable<Uint32>,  // Nullable for top-level employees
}

// Find all employees under a manager
let query = Query::builder()
    .filter(Filter::eq("manager_id", Value::Uint32(manager_id.into())))
    .build();
let direct_reports = database.select::<Employee>(query)?;
}

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "categories"]
pub struct Category {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    #[foreign_key(entity = "Category", table = "categories", column = "id")]
    pub parent_id: Nullable<Uint32>,  // Nullable for root categories
}

// Find root categories
let query = Query::builder()
    .filter(Filter::is_null("parent_id"))
    .build();
let root_categories = database.select::<Category>(query)?;

// Find children of a category
let query = Query::builder()
    .filter(Filter::eq("parent_id", Value::Uint32(parent_id.into())))
    .build();
let children = database.select::<Category>(query)?;
}

Transactions

• Transactions
  • Overview
  • Transaction Lifecycle
    • Begin Transaction
    • Perform Operations
    • Commit
    • Rollback
  • ACID Properties
    • Atomicity
    • Consistency
    • Isolation
    • Durability
  • Error Handling
    • Handling Failures
    • Transaction Errors
  • Best Practices
    • 1. Keep transactions short
    • 2. Always handle rollback
    • 3. Use transactions for related operations
    • 4. Don’t mix transactional and non-transactional operations
  • Examples
    • Bank Transfer
    • Order Processing

Overview

wasm-dbms supports ACID transactions, allowing you to group multiple database operations into a single atomic unit. Either all operations succeed and are committed together, or none of them take effect.

Key features:

• Atomicity: All operations in a transaction succeed or fail together
• Consistency: Data integrity constraints are maintained
• Isolation: Transactions are isolated from each other
• Durability: Committed changes persist

Transaction Lifecycle

Begin Transaction

Start a new transaction using DbmsContext::begin_transaction():

#![allow(unused)]
fn main() {
use wasm_dbms::prelude::*;

// Begin a new transaction
let tx_id = ctx.begin_transaction();
println!("Started transaction: {}", tx_id);
}

The returned transaction ID is used to create a transactional database instance.

Perform Operations

Create a WasmDbmsDatabase with the transaction ID and perform operations:

#![allow(unused)]
fn main() {
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

// Insert within transaction
database.insert::<User>(user)?;

// Update within transaction
database.update::<User>(update)?;

// Delete within transaction
database.delete::<User>(DeleteBehavior::Restrict, Some(filter))?;

// Select within transaction (sees uncommitted changes)
let users = database.select::<User>(query)?;
}

Note: Operations within a transaction are visible to subsequent operations in the same transaction, but not to other callers until committed.

Commit

Commit the transaction to make all changes permanent:

#![allow(unused)]
fn main() {
// Commit the transaction
database.commit()?;
println!("Transaction committed successfully");
}

After commit:

• All changes become visible to other callers
• The transaction ID becomes invalid
• Changes persist in storage

Rollback

Rollback the transaction to discard all changes:

#![allow(unused)]
fn main() {
// Rollback the transaction
database.rollback()?;
println!("Transaction rolled back");
}

After rollback:

• All changes within the transaction are discarded
• The transaction ID becomes invalid
• The database state is as if the transaction never happened

ACID Properties

Atomicity

All operations in a transaction are treated as a single unit. If any operation fails, the entire transaction can be rolled back:

#![allow(unused)]
fn main() {
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

// First operation succeeds
database.insert::<User>(user1)?;

// Second operation fails (e.g., primary key conflict)
let result = database.insert::<User>(user2_duplicate);

if result.is_err() {
    // Rollback everything - user1 is also discarded
    database.rollback()?;
}
}

Consistency

Transactions maintain data integrity:

• Primary key uniqueness is enforced
• Foreign key constraints are checked
• Validators run on all data
• Sanitizers are applied

#![allow(unused)]
fn main() {
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

// This will fail if referenced user doesn't exist
let post = PostInsertRequest {
    id: 1.into(),
    title: "My Post".into(),
    author_id: 999.into(),  // Non-existent user
};

let result = database.insert::<Post>(post);
// Returns Err(DbmsError::Query(QueryError::BrokenForeignKeyReference))
}

Isolation

Changes made within a transaction are not visible to other callers until committed:

#![allow(unused)]
fn main() {
// Database A starts a transaction
let tx_id = ctx.begin_transaction();
let mut db_a = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);
db_a.insert::<User>(new_user)?;

// Database B queries - does NOT see the new user
let db_b = WasmDbmsDatabase::oneshot(&ctx, my_schema);
let users = db_b.select::<User>(query)?;
assert!(!users.iter().any(|u| u.id == new_user.id));

// Database A commits
db_a.commit()?;

// Now Database B can see the user
let users = db_b.select::<User>(query)?;
assert!(users.iter().any(|u| u.id == new_user.id));
}

Durability

Committed transactions persist in storage. When using stable memory providers (e.g., on the Internet Computer), data survives across upgrades.

Error Handling

Handling Failures

When an operation fails within a transaction, you should typically rollback:

#![allow(unused)]
fn main() {
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

fn process_order(database: &impl Database) -> Result<(), DbmsError> {
    // Multiple operations that should succeed together
    database.insert::<Order>(order)?;
    database.update::<Inventory>(update)?;
    database.insert::<OrderItem>(item)?;
    Ok(())
}

match process_order(&database) {
    Ok(()) => {
        database.commit()?;
        println!("Order processed successfully");
    }
    Err(e) => {
        database.rollback()?;
        println!("Order failed, rolled back: {:?}", e);
    }
}
}

Transaction Errors

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::{DbmsError, TransactionError};

match database.commit() {
    Ok(()) => println!("Committed"),
    Err(DbmsError::Transaction(TransactionError::NoActiveTransaction)) => {
        println!("No active transaction to commit");
    }
    Err(e) => println!("Other error: {:?}", e),
}
}

Best Practices

1. Keep transactions short

Long-running transactions hold resources and block other operations:

#![allow(unused)]
fn main() {
// GOOD: Prepare data outside transaction
let users_to_insert = prepare_users();

let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);
for user in users_to_insert {
    database.insert::<User>(user)?;
}
database.commit()?;

// BAD: Doing expensive work inside transaction
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);
for raw_data in large_dataset {
    let user = expensive_parsing(raw_data);  // Don't do this in transaction
    database.insert::<User>(user)?;
}
database.commit()?;
}

2. Always handle rollback

Ensure transactions are either committed or rolled back:

#![allow(unused)]
fn main() {
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

let result = (|| -> Result<(), DbmsError> {
    database.insert::<User>(user1)?;
    database.insert::<User>(user2)?;
    Ok(())
})();

match result {
    Ok(()) => database.commit()?,
    Err(_) => database.rollback()?,
}
}

3. Use transactions for related operations

Group operations that should succeed or fail together:

#![allow(unused)]
fn main() {
// GOOD: Related operations in transaction
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);
database.insert::<Order>(order)?;
database.insert::<Payment>(payment)?;
database.update::<Inventory>(inv_update)?;
database.commit()?;

// BAD: Unrelated operations in transaction (unnecessary)
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);
database.insert::<UserPreferences>(prefs)?;
database.insert::<AuditLog>(log)?;  // Unrelated
database.commit()?;
}

4. Don’t mix transactional and non-transactional operations

#![allow(unused)]
fn main() {
let tx_id = ctx.begin_transaction();
let mut database = WasmDbmsDatabase::from_transaction(&ctx, my_schema, tx_id);

// GOOD: All operations use the transaction
database.insert::<Order>(order)?;
database.insert::<OrderItem>(item)?;

// BAD: Mixing transaction and non-transaction
let oneshot = WasmDbmsDatabase::oneshot(&ctx, my_schema);
database.insert::<Order>(order)?;
oneshot.insert::<AuditLog>(log)?;  // Not in transaction!
}

Examples

Bank Transfer

Transfer money between accounts atomically:

#![allow(unused)]
fn main() {
fn transfer(
    ctx: &DbmsContext<impl MemoryProvider>,
    from_account: u32,
    to_account: u32,
    amount: Decimal,
) -> Result<(), DbmsError> {
    let tx_id = ctx.begin_transaction();
    let mut database = WasmDbmsDatabase::from_transaction(ctx, my_schema, tx_id);

    // Deduct from source account
    let deduct = AccountUpdateRequest::builder()
        .decrease_balance(amount)
        .filter(Filter::eq("id", Value::Uint32(from_account.into())))
        .build();
    database.update::<Account>(deduct)?;

    // Add to destination account
    let add = AccountUpdateRequest::builder()
        .increase_balance(amount)
        .filter(Filter::eq("id", Value::Uint32(to_account.into())))
        .build();
    database.update::<Account>(add)?;

    // Record the transfer
    let transfer_record = TransferInsertRequest {
        id: Uuid::new_v4().into(),
        from_account: from_account.into(),
        to_account: to_account.into(),
        amount,
        timestamp: DateTime::now(),
    };
    database.insert::<Transfer>(transfer_record)?;

    // Commit atomically
    database.commit()?;
    Ok(())
}
}

Order Processing

Process an order with inventory update:

#![allow(unused)]
fn main() {
fn process_order(
    ctx: &DbmsContext<impl MemoryProvider>,
    order: OrderInsertRequest,
    items: Vec<OrderItemInsertRequest>,
) -> Result<u32, Box<dyn std::error::Error>> {
    let tx_id = ctx.begin_transaction();
    let mut database = WasmDbmsDatabase::from_transaction(ctx, my_schema, tx_id);

    // Insert the order
    database.insert::<Order>(order.clone())?;

    // Insert order items and update inventory
    for item in items {
        // Insert order item
        database.insert::<OrderItem>(item.clone())?;

        // Decrease inventory
        let inv_update = InventoryUpdateRequest::builder()
            .decrease_quantity(item.quantity)
            .filter(Filter::eq("product_id", Value::Uint32(item.product_id)))
            .build();

        let updated = database.update::<Inventory>(inv_update)?;

        if updated == 0 {
            // Product not in inventory, rollback
            database.rollback()?;
            return Err("Product not found in inventory".into());
        }
    }

    // All successful, commit
    database.commit()?;
    Ok(order.id.into())
}
}

Custom Data Types

• Custom Data Types
  • Overview
  • Defining a Custom Type
    • Step 1: Define the Type
    • Step 2: Implement Display
    • Step 3: Implement Encode
    • Step 4: Implement DataType and Derive CustomDataType
  • Using Custom Types in Tables
    • The custom_type Attribute
    • Nullable Custom Types
  • Filtering and Querying
  • Ordering Contract
  • Examples
    • Enum: Priority
    • Struct: Address

Overview

wasm-dbms ships with a set of built-in data types that cover the most common use cases. When your domain requires types that go beyond those built-ins, you can define custom data types.

Custom data types let you store any Rust type – enums, newtypes, structs – inside your tables. The DBMS engine stores them as opaque bytes internally and uses a type tag string to identify each custom type.

When to use custom types:

• Domain-specific enums (e.g., Priority, Status, Role)
• Composite value objects (e.g., Address, Coordinates)
• Newtypes that wrap primitives with domain meaning (e.g., Email(String))

Defining a Custom Type

Creating a custom type requires four steps:

1. Define the type with the required derives
2. Implement Display
3. Implement Encode (binary serialization)
4. Implement DataType and derive CustomDataType

Step 1: Define the Type

Your type must derive or implement several traits. For enums, all must be implemented manually or derived:

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};

#[derive(
    Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord,
    Hash, Default, Serialize, Deserialize,
)]
pub enum Priority {
    #[default]
    Low,
    Medium,
    High,
}
}

For structs, the same traits are required:

#![allow(unused)]
fn main() {
#[derive(
    Debug, Clone, PartialEq, Eq, PartialOrd, Ord,
    Hash, Default, Serialize, Deserialize,
)]
pub struct Address {
    pub street: String,
    pub city: String,
    pub zip: String,
}
}

Required traits:

Note: For IC canister usage, also derive CandidType and Deserialize from the candid crate.

Step 2: Implement Display

The Display implementation provides a human-readable representation used for logging and diagnostics:

#![allow(unused)]
fn main() {
use std::fmt;

impl fmt::Display for Priority {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            Priority::Low => write!(f, "low"),
            Priority::Medium => write!(f, "medium"),
            Priority::High => write!(f, "high"),
        }
    }
}

impl fmt::Display for Address {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "{}, {} {}", self.street, self.city, self.zip)
    }
}
}

Step 3: Implement Encode

The Encode trait defines how your type is serialized to and from bytes for memory storage. Enums require a manual implementation; for structs, you can use #[derive(Encode)].

Enum (manual implementation):

#![allow(unused)]
fn main() {
use std::borrow::Cow;
use wasm_dbms_api::prelude::*;

impl Encode for Priority {
    const SIZE: DataSize = DataSize::Fixed(1);
    const ALIGNMENT: PageOffset = DEFAULT_ALIGNMENT;

    fn encode(&self) -> Cow<'_, [u8]> {
        Cow::Owned(vec![match self {
            Priority::Low => 0,
            Priority::Medium => 1,
            Priority::High => 2,
        }])
    }

    fn decode(data: Cow<[u8]>) -> MemoryResult<Self> {
        match data[0] {
            0 => Ok(Priority::Low),
            1 => Ok(Priority::Medium),
            2 => Ok(Priority::High),
            other => Err(MemoryError::DecodeError(
                DecodeError::TryFromSliceError(
                    format!("invalid Priority byte: {other}"),
                ),
            )),
        }
    }

    fn size(&self) -> MSize {
        1
    }
}
}

Struct (derive macro):

The #[derive(Encode)] macro works for structs whose fields all implement Encode. Since String does not implement Encode but Text does, use wasm-dbms types for the struct fields:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(
    Debug, Clone, PartialEq, Eq, PartialOrd, Ord,
    Hash, Default, Serialize, Deserialize,
    Encode,
)]
pub struct Address {
    pub street: Text,
    pub city: Text,
    pub zip: Text,
}
}

Key Encode concepts:

Step 4: Implement DataType and Derive CustomDataType

Finally, implement the DataType marker trait and derive CustomDataType with a unique type tag:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

impl DataType for Priority {}

// Manual CustomDataType implementation for enums
impl CustomDataType for Priority {
    const TYPE_TAG: &'static str = "priority";
}

// Manual From<Priority> for Value implementation
impl From<Priority> for Value {
    fn from(val: Priority) -> Value {
        Value::Custom(CustomValue {
            type_tag: <Priority as CustomDataType>::TYPE_TAG.to_string(),
            encoded: Encode::encode(&val).into_owned(),
            display: val.to_string(),
        })
    }
}
}

For structs, you can use the CustomDataType derive macro instead of the manual implementation above:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

impl DataType for Address {}

#[derive(CustomDataType)]
#[type_tag = "address"]
pub struct Address {
    // ...
}
}

The #[derive(CustomDataType)] macro generates both the CustomDataType trait implementation and the From<T> for Value conversion. For enums, you must write these implementations manually.

Type tag rules:

• Must be unique across all custom types in your database
• Must be stable across upgrades (changing it makes existing data unreadable)
• Use lowercase, descriptive names (e.g., "priority", "address", "role")

Using Custom Types in Tables

The custom_type Attribute

To use a custom type in a table, annotate the field with #[custom_type]:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "tasks"]
pub struct Task {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    #[custom_type]
    pub priority: Priority,
    #[custom_type]
    pub address: Address,
}
}

Without the #[custom_type] attribute, the Table macro won’t know how to handle your type and compilation will fail.

Nullable Custom Types

Custom types can be wrapped in Nullable<T> for optional fields:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "tasks"]
pub struct Task {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    #[custom_type]
    pub priority: Nullable<Priority>,
}
}

When Nullable::Null, the value is stored as Value::Null. When Nullable::Value(v), it is stored as Value::Custom(...).

Filtering and Querying

To filter on custom type fields, construct a Value::Custom with the appropriate CustomValue:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

// Create a filter for Priority::High
let high_priority = Priority::High;
let filter = Filter::eq("priority", high_priority.into());

// You can also construct the Value manually
let filter = Filter::eq("priority", Value::Custom(CustomValue {
    type_tag: "priority".to_string(),
    encoded: Encode::encode(&Priority::High).into_owned(),
    display: "high".to_string(),
}));
}

To extract a custom type from a Value:

#![allow(unused)]
fn main() {
let value: Value = Priority::High.into();

// Get the raw CustomValue
if let Some(cv) = value.as_custom() {
    println!("type: {}, display: {}", cv.type_tag, cv.display);
}

// Decode into the concrete type
if let Some(priority) = value.as_custom_type::<Priority>() {
    println!("Priority: {priority}");
}
}

Ordering Contract

Custom types support all filter operations: Eq, Ne, In, Gt, Lt, Ge, Le.

For equality filters (Eq, Ne, In), the only requirement is that the Encode implementation produces canonical output – the same value always encodes to the same bytes.

For range filters (Gt, Lt, Ge, Le) and ORDER BY, the encoding must be order-preserving: if a < b according to Ord, then a.encode() < b.encode() lexicographically. This is because the DBMS compares custom values by their encoded bytes.

Example of order-preserving encoding:

The Priority enum above encodes Low = 0, Medium = 1, High = 2. Since Low < Medium < High in the Ord implementation and [0] < [1] < [2] lexicographically, range filters work correctly.

Warning: If your encoding is not order-preserving, equality filters will still work, but range filters and sorting will produce incorrect results.

Examples

Enum: Priority

A complete example of a custom enum type used in a table:

#![allow(unused)]
fn main() {
use std::borrow::Cow;
use std::fmt;

use serde::{Deserialize, Serialize};
use wasm_dbms_api::prelude::*;

// 1. Define the type
#[derive(
    Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord,
    Hash, Default, Serialize, Deserialize,
)]
pub enum Priority {
    #[default]
    Low,
    Medium,
    High,
}

// 2. Implement Display
impl fmt::Display for Priority {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            Priority::Low => write!(f, "low"),
            Priority::Medium => write!(f, "medium"),
            Priority::High => write!(f, "high"),
        }
    }
}

// 3. Implement Encode (manual for enums)
impl Encode for Priority {
    const SIZE: DataSize = DataSize::Fixed(1);
    const ALIGNMENT: PageOffset = DEFAULT_ALIGNMENT;

    fn encode(&self) -> Cow<'_, [u8]> {
        Cow::Owned(vec![match self {
            Priority::Low => 0,
            Priority::Medium => 1,
            Priority::High => 2,
        }])
    }

    fn decode(data: Cow<[u8]>) -> MemoryResult<Self> {
        match data[0] {
            0 => Ok(Priority::Low),
            1 => Ok(Priority::Medium),
            2 => Ok(Priority::High),
            other => Err(MemoryError::DecodeError(
                DecodeError::TryFromSliceError(
                    format!("invalid Priority byte: {other}"),
                ),
            )),
        }
    }

    fn size(&self) -> MSize {
        1
    }
}

// 4. Implement DataType + CustomDataType + From<Priority> for Value
impl DataType for Priority {}

impl CustomDataType for Priority {
    const TYPE_TAG: &'static str = "priority";
}

impl From<Priority> for Value {
    fn from(val: Priority) -> Value {
        Value::Custom(CustomValue {
            type_tag: <Priority as CustomDataType>::TYPE_TAG.to_string(),
            encoded: Encode::encode(&val).into_owned(),
            display: val.to_string(),
        })
    }
}

// Use in a table
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "tasks"]
pub struct Task {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    #[custom_type]
    pub priority: Priority,
}
}

Struct: Address

A complete example of a custom struct type. Structs can use #[derive(Encode)] and #[derive(CustomDataType)]:

#![allow(unused)]
fn main() {
use std::fmt;

use serde::{Deserialize, Serialize};
use wasm_dbms_api::prelude::*;

// 1. Define the type with Encode and CustomDataType derives
#[derive(
    Debug, Clone, PartialEq, Eq, PartialOrd, Ord,
    Hash, Default, Serialize, Deserialize,
    Encode, CustomDataType,
)]
#[type_tag = "address"]
pub struct Address {
    pub street: Text,
    pub city: Text,
    pub zip: Text,
}

// 2. Implement Display
impl fmt::Display for Address {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(
            f, "{}, {} {}",
            self.street.as_str(),
            self.city.as_str(),
            self.zip.as_str(),
        )
    }
}

// 3. Implement DataType
impl DataType for Address {}

// Use in a table
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "customers"]
pub struct Customer {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    #[custom_type]
    pub address: Address,
}
}

Wasmtime Example

• Overview
• How It Works
  • WIT Interface
  • Guest Component
  • Host Binary
• FileMemoryProvider
• Building and Running
  • Prerequisites
  • Build
  • Run
• Extending with Custom Tables
• Key Concepts

This guide explains how to use wasm-dbms with the WebAssembly Component Model (WIT) and Wasmtime. It walks through the example in crates/wasm-dbms/example/.

Overview

The WebAssembly Component Model defines a standard way for WASM modules to expose typed interfaces using WIT (WebAssembly Interface Types). This example shows how to:

1. Define a WIT interface for the wasm-dbms CRUD and transaction API
2. Build a guest WASM component that wraps wasm-dbms behind the WIT interface
3. Run the guest inside a native Wasmtime host

This approach makes wasm-dbms usable from any Component Model host, not just Rust. The WIT contract at /wit/dbms.wit can be consumed by hosts written in Go, Python, JavaScript, or any language with Component Model tooling.

How It Works

WIT Interface

The WIT definition (/wit/dbms.wit) exposes a database interface with these operations:

• select — query rows from a table with optional filter, ordering, limit, and offset
• insert — insert a row into a table, optionally within a transaction
• update — update rows matching a filter, optionally within a transaction
• delete — delete rows matching a filter, optionally within a transaction
• begin-transaction — start a new ACID transaction
• commit / rollback — finalize or abort a transaction

Values are passed as a value variant type that covers booleans, integers, floats, strings, blobs, and null. Filters are JSON-serialized strings matching the wasm_dbms_api::Filter type.

This raw/dynamic API is intentional: WIT cannot express Rust generics or user-defined table schemas, so type safety is enforced inside the guest by the wasm-dbms engine.

Guest Component

The guest (crates/wasm-dbms/example/guest/) compiles to wasm32-wasip2 and exports the WIT database interface. Internally it:

1. Initializes a DbmsContext<FileMemoryProvider> lazily on first call
2. Registers example tables (users, posts) using #[derive(Table)]
3. Converts between WIT variant values and wasm-dbms Value types
4. Dispatches operations through a DatabaseSchema implementation

The bridge layer in lib.rs handles all the type conversions between the WIT boundary and the typed wasm-dbms internals.

Host Binary

The host (crates/wasm-dbms/example/host/) is a native Rust binary using Wasmtime. It:

1. Creates a Wasmtime engine with Component Model enabled
2. Sets up a WASI context with a preopened directory for the database file
3. Loads the guest .wasm component and instantiates it
4. Calls the exported database functions to demonstrate all operations

FileMemoryProvider

The FileMemoryProvider implements the MemoryProvider trait using std::fs file I/O. It provides persistent, file-backed storage so that data survives across invocations.

#![allow(unused)]
fn main() {
use wasm_dbms_memory::prelude::MemoryProvider;

pub struct FileMemoryProvider {
    file: File,       // open file handle
    size: u64,        // current size in bytes
    pages: u64,       // allocated pages (size / PAGE_SIZE)
}
}

Operations:

• grow(n) — extends the file by n × 65536 bytes
• read(offset, buf) — seeks to offset and reads into buffer
• write(offset, buf) — seeks to offset, writes buffer, and flushes

The provider is initialized with a file path relative to the WASI preopened directory (defaults to wasm-dbms.db).

Note: FileMemoryProvider does not handle concurrent access. It assumes single-writer usage.

Building and Running

Prerequisites

• Rust 1.91.1+

• wasm32-wasip2 target:

  rustup target add wasm32-wasip2
  

• just command runner

Build

# Build guest + host
just build_wasm_dbms_example

This compiles the guest to wasm32-wasip2 (producing a WASM component at .artifact/wasm-dbms-example-guest.wasm) and builds the native host binary.

Run

just test_wasm_dbms_example

Or run manually:

cargo run --release -p wasm-dbms-example-host -- .artifact/wasm-dbms-example-guest.wasm

The demo inserts users and posts, queries them with filters and ordering, demonstrates transaction commit (data persists) and rollback (data discarded), then cleans up.

Extending with Custom Tables

To add your own tables to the example:

1. Define the table in guest/src/schema.rs:

   #![allow(unused)]
   fn main() {
   #[derive(Debug, Table, Clone, PartialEq, Eq)]
   #[table = "comments"]
   pub struct Comment {
       #[primary_key]
       pub id: Uint32,
       pub body: Text,
       #[foreign_key(entity = "Post", table = "posts", column = "id")]
       pub post_id: Uint32,
   }
   }

2. Add dispatch arms for "comments" in every method of ExampleDatabaseSchema in guest/src/schema.rs (select, insert, update, delete, validate_insert, validate_update, referenced_tables).

3. Register the table in register_tables():

   #![allow(unused)]
   fn main() {
   ctx.register_table::<Comment>()?;
   }

4. Update the column lookup in table_columns() (guest/src/lib.rs):

   #![allow(unused)]
   fn main() {
   "comments" => Ok(schema::Comment::columns()),
   }

5. Rebuild with just build_wasm_dbms_example.

Key Concepts

Next Steps

• Getting Started — Set up wasm-dbms from scratch with the Database trait
• CRUD Operations — Detailed guide on all database operations
• Transactions — ACID transactions with commit/rollback
• Schema Definition — Complete schema reference

Schema Definition

• Schema Definition
  • Overview
  • Table Definition
    • Required Derives
    • Table Attribute
  • Column Attributes
    • Primary Key
    • Autoincrement
    • Unique
    • Index
    • Foreign Key
    • Custom Type
    • Sanitizer
    • Validate
    • Candid
    • Alignment
  • Generated Types
    • Record Type
    • InsertRequest Type
    • UpdateRequest Type
    • ForeignFetcher Type
  • Complete Example
  • Best Practices

Overview

wasm-dbms schemas are defined entirely in Rust using derive macros and attributes. Each struct represents a database table, and each field represents a column.

Key concepts:

• Structs with #[derive(Table)] become database tables
• Fields become columns with their types
• Attributes configure primary keys, foreign keys, validation, and more

Table Definition

Required Derives

Every table struct must have these derives:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
}
}

Note: For IC canister usage, also add CandidType and Deserialize derives plus the #[candid] attribute. See the IC Schema Reference.

Table Attribute

The #[table = "name"] attribute specifies the table name in the database:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "user_accounts"]  // Table name in database
pub struct UserAccount {    // Rust struct name (can differ)
    // ...
}
}

Naming conventions:

• Use snake_case for table names
• Table names should be plural (e.g., users, posts, order_items)
• Keep names short but descriptive

Column Attributes

Primary Key

Every table must have exactly one primary key:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,  // Primary key
    pub name: Text,
}
}

Primary key rules:

• Exactly one field must be marked with #[primary_key]
• Primary keys must be unique across all records
• Primary keys cannot be null
• Common types: Uint32, Uint64, Uuid, Text

UUID as primary key:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "orders"]
pub struct Order {
    #[primary_key]
    pub id: Uuid,  // UUID primary key
    pub total: Decimal,
}
}

Autoincrement

Automatically generate sequential values for a column on insert:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    #[autoincrement]
    pub id: Uint32,  // Automatically assigned 1, 2, 3, ...
    pub name: Text,
}
}

Autoincrement rules:

• Only integer types are supported: Int8, Int16, Int32, Int64, Uint8, Uint16, Uint32, Uint64
• The counter starts at zero and increments by one on each insert
• Each autoincrement column has an independent counter
• Counters persist across canister upgrades (stored in stable memory)
• When the counter reaches the type’s maximum value, inserts return an AutoincrementOverflow error
• Deleted records do not recycle their autoincrement values
• A table can have multiple #[autoincrement] columns

Choosing the right type:

Tip: Uint64 is recommended for most use cases. Only use smaller types when storage space is critical and you are certain the record count will stay within bounds.

Combining with other attributes:

#![allow(unused)]
fn main() {
#[primary_key]
#[autoincrement]
pub id: Uint64,  // Auto-generated unique primary key
}

Unique

Enforce uniqueness on a column:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[unique]
    pub email: Text,  // Must be unique across all rows

    pub name: Text,
}
}

Unique constraint rules:

• Insert and update operations that would create a duplicate value return a UniqueConstraintViolation error
• Multiple fields in the same table can each be marked #[unique] independently
• A #[unique] field automatically gets a B+ tree index – no separate #[index] annotation is needed
• Primary keys are always unique by definition; you don’t need #[unique] on a #[primary_key] field

Combining with other attributes:

#![allow(unused)]
fn main() {
#[unique]
#[sanitizer(TrimSanitizer)]
#[sanitizer(LowerCaseSanitizer)]
#[validate(EmailValidator)]
pub email: Text,  // Sanitized, validated, then checked for uniqueness
}

Note: Sanitization and validation run before the uniqueness check, so the sanitized value is what gets compared.

Index

Define indexes on columns for faster lookups:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[index]
    pub email: Text,  // Single-column index

    pub name: Text,
}
}

The primary key is always an implicit index – you don’t need to add #[index] to it.

Composite indexes:

Use group to group multiple fields into a single composite index:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "products"]
pub struct Product {
    #[primary_key]
    pub id: Uint32,

    #[index(group = "category_brand")]
    pub category: Text,

    #[index(group = "category_brand")]
    pub brand: Text,

    pub name: Text,
}
}

Fields sharing the same group name form a composite index, with columns ordered by field declaration order. In the example above, the composite index covers (category, brand).

Syntax variants:

#![allow(unused)]
fn main() {
// Single-column index
#[index]

// Composite index (group multiple fields by name)
#[index(group = "group_name")]
}

Foreign Key

Define relationships between tables:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,

    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,
}
}

Attribute parameters:

Nullable foreign key:

#![allow(unused)]
fn main() {
#[foreign_key(entity = "User", table = "users", column = "id")]
pub manager_id: Nullable<Uint32>,  // Can be null
}

Self-referential foreign key:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "categories"]
pub struct Category {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,

    #[foreign_key(entity = "Category", table = "categories", column = "id")]
    pub parent_id: Nullable<Uint32>,
}
}

Custom Type

Mark a field as a user-defined custom data type:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "tasks"]
pub struct Task {
    #[primary_key]
    pub id: Uint32,
    #[custom_type]
    pub priority: Priority,  // User-defined type
}
}

The #[custom_type] attribute tells the Table macro that this field implements the CustomDataType trait. Without it, the macro won’t know how to serialize and deserialize the field.

Nullable custom types:

#![allow(unused)]
fn main() {
#[custom_type]
pub priority: Nullable<Priority>,  // Optional custom type
}

See the Custom Data Types Guide for how to define custom types.

Sanitizer

Apply data transformations before storage:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[sanitizer(TrimSanitizer)]
    pub name: Text,

    #[sanitizer(LowerCaseSanitizer)]
    #[sanitizer(TrimSanitizer)]
    pub email: Text,

    #[sanitizer(RoundToScaleSanitizer(2))]
    pub balance: Decimal,

    #[sanitizer(ClampSanitizer, min = 0, max = 120)]
    pub age: Uint8,
}
}

Syntax variants:

#![allow(unused)]
fn main() {
// Unit struct (no parameters)
#[sanitizer(TrimSanitizer)]

// Tuple struct (positional parameter)
#[sanitizer(RoundToScaleSanitizer(2))]

// Named fields struct
#[sanitizer(ClampSanitizer, min = 0, max = 100)]
}

See Sanitization Reference for all available sanitizers.

Validate

Add validation rules:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[validate(MaxStrlenValidator(100))]
    pub name: Text,

    #[validate(EmailValidator)]
    pub email: Text,

    #[validate(UrlValidator)]
    pub website: Nullable<Text>,
}
}

Validation happens after sanitization:

#![allow(unused)]
fn main() {
#[sanitizer(TrimSanitizer)]           // 1. First: trim whitespace
#[validate(MaxStrlenValidator(100))]  // 2. Then: check length
pub name: Text,
}

See Validation Reference for all available validators.

Candid

Enable CandidType and Deserialize derives on generated types:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
}
}

When the #[candid] attribute is present, the Table macro adds candid::CandidType, serde::Serialize, and serde::Deserialize derives to the generated Record, InsertRequest, and UpdateRequest types.

When to use:

• Required for IC canister deployment where types must cross canister boundaries via Candid
• Any context where generated types need Candid serialization

Note: The #[candid] attribute only affects the types generated by the Table macro. You still need to derive CandidType and Deserialize on the table struct itself.

See the IC Schema Reference for full IC integration details.

Alignment

Advanced: Configure memory alignment for dynamic-size tables:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "large_records"]
#[alignment = 64]  // 64-byte alignment
pub struct LargeRecord {
    #[primary_key]
    pub id: Uint32,
    pub data: Text,  // Variable-size field
}
}

When to use:

• Performance tuning for specific access patterns
• Optimizing memory layout for large records

Rules:

• Minimum alignment is 8 bytes for dynamic types
• Default alignment is 32 bytes
• Fixed-size tables ignore this attribute (alignment equals record size)

Caution: Only change alignment if you understand the performance implications.

Generated Types

The Table macro generates several types for each table.

Record Type

{StructName}Record - The full record type returned from queries:

#![allow(unused)]
fn main() {
// Generated from User struct
pub struct UserRecord {
    pub id: Uint32,
    pub name: Text,
    pub email: Text,
}

// Usage
let users: Vec<UserRecord> = database.select::<User>(query)?;

for user in users {
    println!("{}: {}", user.id, user.name);
}
}

InsertRequest Type

{StructName}InsertRequest - Request type for inserting records:

#![allow(unused)]
fn main() {
// Generated from User struct
pub struct UserInsertRequest {
    pub id: Uint32,
    pub name: Text,
    pub email: Text,
}

// Usage
let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    email: "alice@example.com".into(),
};

database.insert::<User>(user)?;
}

UpdateRequest Type

{StructName}UpdateRequest - Request type for updating records:

#![allow(unused)]
fn main() {
// Generated from User struct (with builder pattern)
let update = UserUpdateRequest::builder()
    .set_name("New Name".into())
    .set_email("new@example.com".into())
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

// Usage
database.update::<User>(update)?;
}

Builder methods:

• set_{field_name}(value) - Set a field value
• filter(Filter) - WHERE clause (required)
• build() - Build the update request

ForeignFetcher Type

{StructName}ForeignFetcher - Internal type for eager loading:

#![allow(unused)]
fn main() {
// Generated automatically, used internally
// You typically don't interact with this directly
}

Complete Example

#![allow(unused)]
fn main() {
// schema/src/lib.rs
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[sanitizer(TrimSanitizer)]
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,

    #[unique]
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(LowerCaseSanitizer)]
    #[validate(EmailValidator)]
    pub email: Text,

    pub created_at: DateTime,

    pub is_active: Boolean,
}

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uuid,

    #[validate(MaxStrlenValidator(200))]
    pub title: Text,

    pub content: Text,

    pub published: Boolean,

    #[index(group = "author_date")]
    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,

    pub metadata: Nullable<Json>,

    #[index(group = "author_date")]
    pub created_at: DateTime,
}

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "comments"]
pub struct Comment {
    #[primary_key]
    pub id: Uuid,

    #[validate(MaxStrlenValidator(1000))]
    pub content: Text,

    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,

    #[foreign_key(entity = "Post", table = "posts", column = "id")]
    pub post_id: Uuid,

    pub created_at: DateTime,
}
}

For generating a complete IC canister API from this schema, see the IC Schema Reference.

Best Practices

1. Keep schema in a separate crate

my-project/
├── schema/           # Reusable types
│   ├── Cargo.toml
│   └── src/lib.rs
└── app/              # Application using the database
    ├── Cargo.toml
    └── src/lib.rs

2. Use appropriate primary key types

#![allow(unused)]
fn main() {
// Sequential IDs - simple, good for internal use
pub id: Uint32,

// UUIDs - better for distributed systems, no guessing
pub id: Uuid,
}

3. Always validate user input

#![allow(unused)]
fn main() {
#[validate(MaxStrlenValidator(1000))]  // Prevent huge strings
pub content: Text,

#[validate(EmailValidator)]  // Validate format
pub email: Text,
}

4. Use nullable for optional fields

#![allow(unused)]
fn main() {
pub phone: Nullable<Text>,  // Clearly optional
pub bio: Nullable<Text>,
}

5. Consider sanitization for consistency

#![allow(unused)]
fn main() {
#[sanitizer(TrimSanitizer)]
#[sanitizer(LowerCaseSanitizer)]
pub email: Text,  // Always lowercase, no whitespace
}

6. Document your schema

#![allow(unused)]
fn main() {
/// User account information
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    /// Unique user identifier
    #[primary_key]
    pub id: Uint32,

    /// User's display name (max 100 chars)
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,
}
}

Data Types

• Data Types
  • Overview
  • Integer Types
    • Unsigned Integers
    • Signed Integers
  • Decimal
  • Text
  • Boolean
  • Date and Time
    • Date
    • DateTime
  • Binary Data
    • Blob
  • Identifiers
    • Uuid
  • Semi-Structured Data
    • Json
  • Nullable
  • Custom Types
  • Type Conversion Reference

Overview

wasm-dbms provides a rich set of data types for defining table schemas. Each type maps to standard Rust types for seamless integration.

Type categories:

Note: The Principal type is available in ic-dbms-api for Internet Computer integration. See the IC Data Types reference for details.

Integer Types

Unsigned Integers

Uint8 - 8-bit unsigned integer (0 to 255)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Uint8;

#[derive(Table, ...)]
#[table = "settings"]
pub struct Setting {
    #[primary_key]
    pub id: Uint32,
    pub priority: Uint8,  // 0-255
}

// Usage
let setting = SettingInsertRequest {
    id: 1.into(),
    priority: 10.into(),  // or Uint8::from(10)
};
}

Uint16 - 16-bit unsigned integer (0 to 65,535)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Uint16;

pub struct Product {
    pub stock_count: Uint16,  // 0-65,535
}

let count: Uint16 = 1000.into();
}

Uint32 - 32-bit unsigned integer (0 to 4,294,967,295)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Uint32;

pub struct User {
    #[primary_key]
    pub id: Uint32,  // Common for primary keys
}

let id: Uint32 = 12345.into();
}

Uint64 - 64-bit unsigned integer (0 to 18,446,744,073,709,551,615)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Uint64;

pub struct Transaction {
    pub amount_e8s: Uint64,  // For large numbers like token amounts
}

let amount: Uint64 = 1_000_000_000u64.into();
}

Signed Integers

Int8 - 8-bit signed integer (-128 to 127)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Int8;

pub struct Temperature {
    pub celsius: Int8,  // -128 to 127
}

let temp: Int8 = (-10).into();
}

Int16 - 16-bit signed integer (-32,768 to 32,767)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Int16;

pub struct Altitude {
    pub meters: Int16,  // Can be negative (below sea level)
}

let altitude: Int16 = (-100).into();
}

Int32 - 32-bit signed integer (-2,147,483,648 to 2,147,483,647)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Int32;

pub struct Account {
    pub balance_cents: Int32,  // Can be negative (debt)
}

let balance: Int32 = (-5000).into();
}

Int64 - 64-bit signed integer

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Int64;

pub struct Statistics {
    pub total_change: Int64,  // Large signed values
}

let change: Int64 = (-1_000_000_000i64).into();
}

Decimal

Decimal - Arbitrary-precision decimal number

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Decimal;

pub struct Product {
    pub price: Decimal,      // $19.99
    pub weight_kg: Decimal,  // 2.5
}

// From f64
let price: Decimal = 19.99.into();

// From string (more precise)
let price: Decimal = "19.99".parse().unwrap();

// With sanitizer for rounding
#[derive(Table, ...)]
#[table = "products"]
pub struct Product {
    #[primary_key]
    pub id: Uint32,
    #[sanitizer(RoundToScaleSanitizer(2))]  // Round to 2 decimal places
    pub price: Decimal,
}
}

Note: Use RoundToScaleSanitizer to ensure consistent decimal precision.

Text

Text - UTF-8 string

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Text;

pub struct User {
    pub name: Text,
    pub email: Text,
    pub bio: Text,
}

// From &str
let name: Text = "Alice".into();

// From String
let email: Text = String::from("alice@example.com").into();

// Access the string
let text: Text = "Hello".into();
assert_eq!(text.as_str(), "Hello");
}

With validation:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,
    #[validate(EmailValidator)]
    pub email: Text,
}
}

With sanitization:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(CollapseWhitespaceSanitizer)]
    pub name: Text,
    #[sanitizer(LowerCaseSanitizer)]
    pub email: Text,
}
}

Boolean

Boolean - True or false value

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Boolean;

pub struct User {
    pub is_active: Boolean,
    pub email_verified: Boolean,
}

let active: Boolean = true.into();
let verified: Boolean = false.into();

// Convert back
let value: bool = active.into();
}

Filtering by boolean:

#![allow(unused)]
fn main() {
// Find active users
let filter = Filter::eq("is_active", Value::Boolean(true));

// Find unverified users
let filter = Filter::eq("email_verified", Value::Boolean(false));
}

Date and Time

Date

Date - Calendar date (year, month, day)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Date;

pub struct Event {
    pub event_date: Date,
}

// Create from components
let date = Date::new(2024, 6, 15);  // June 15, 2024

// From chrono NaiveDate (if using chrono)
use chrono::NaiveDate;
let naive = NaiveDate::from_ymd_opt(2024, 6, 15).unwrap();
let date: Date = naive.into();
}

DateTime

DateTime - Date and time with timezone

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::DateTime;

pub struct User {
    pub created_at: DateTime,
    pub last_login: DateTime,
}

// Current time
let now = DateTime::now();

// From chrono DateTime<Utc>
use chrono::{DateTime as ChronoDateTime, Utc};
let chrono_dt: ChronoDateTime<Utc> = Utc::now();
let dt: DateTime = chrono_dt.into();
}

With sanitization:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "events"]
pub struct Event {
    #[primary_key]
    pub id: Uint32,
    #[sanitizer(UtcSanitizer)]  // Convert to UTC
    pub scheduled_at: DateTime,
}
}

Binary Data

Blob

Blob - Binary large object (byte array)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Blob;

pub struct Document {
    pub content: Blob,      // File content
    pub thumbnail: Blob,    // Image data
}

// From Vec<u8>
let data: Vec<u8> = vec![0x89, 0x50, 0x4E, 0x47];  // PNG header
let blob: Blob = data.into();

// From slice
let blob: Blob = Blob::from(&[1, 2, 3, 4][..]);

// Access bytes
let bytes: &[u8] = blob.as_slice();
}

Note: Be mindful of storage costs when storing large blobs. Consider storing only references (hashes, URLs) for very large files.

Identifiers

Uuid

Uuid - Universally unique identifier (128-bit)

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Uuid;

pub struct Order {
    #[primary_key]
    pub id: Uuid,  // UUID as primary key
}

// Generate new UUID
let id = Uuid::new_v4();

// From string
let id: Uuid = "550e8400-e29b-41d4-a716-446655440000".parse().unwrap();

// From bytes
let bytes: [u8; 16] = [/* 16 bytes */];
let id = Uuid::from_bytes(bytes);
}

Benefits over sequential IDs:

• Globally unique without coordination
• No sequential guessing
• Safe for distributed systems

Semi-Structured Data

Json

Json - JSON object or array

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::Json;
use std::str::FromStr;

pub struct User {
    pub metadata: Json,    // Flexible schema
    pub preferences: Json, // User settings
}

// From string
let json = Json::from_str(r#"{"theme": "dark", "language": "en"}"#).unwrap();

// From serde_json::Value
use serde_json::json;
let json: Json = json!({
    "notifications": true,
    "timezone": "UTC"
}).into();
}

Querying JSON:

#![allow(unused)]
fn main() {
// Check if JSON contains pattern
let filter = Filter::json("metadata", JsonFilter::contains(
    Json::from_str(r#"{"active": true}"#).unwrap()
));

// Extract and compare
let filter = Filter::json("preferences",
    JsonFilter::extract_eq("theme", Value::Text("dark".into()))
);

// Check path exists
let filter = Filter::json("metadata", JsonFilter::has_key("email"));
}

See the JSON Reference for comprehensive JSON documentation.

Nullable

Nullable<T> - Optional value wrapper

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::{Nullable, Text, Uint32};

pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    pub phone: Nullable<Text>,      // Optional phone number
    pub age: Nullable<Uint32>,      // Optional age
    pub bio: Nullable<Text>,        // Optional biography
}

// Insert with value
let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    phone: Nullable::Value("555-1234".into()),
    age: Nullable::Null,
    bio: Nullable::Null,
};

// Check if null
let phone = user.phone;
match phone {
    Nullable::Value(p) => println!("Phone: {}", p.as_str()),
    Nullable::Null => println!("No phone number"),
}
}

Filtering nullable fields:

#![allow(unused)]
fn main() {
// Find users with phone numbers
let filter = Filter::not_null("phone");

// Find users without phone numbers
let filter = Filter::is_null("phone");

// Find users with specific phone
let filter = Filter::eq("phone", Value::Text("555-1234".into()));
}

Nullable foreign keys:

#![allow(unused)]
fn main() {
pub struct Employee {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    #[foreign_key(entity = "Employee", table = "employees", column = "id")]
    pub manager_id: Nullable<Uint32>,  // Top-level employees have no manager
}
}

Custom Types

Beyond the built-in types listed above, wasm-dbms supports user-defined custom data types. Custom types let you store enums, structs, and newtypes in your tables by implementing the CustomDataType trait.

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "tasks"]
pub struct Task {
    #[primary_key]
    pub id: Uint32,
    #[custom_type]
    pub priority: Priority,  // User-defined custom type
}
}

See the Custom Data Types Guide for step-by-step instructions on defining and using custom types.

Type Conversion Reference

Note: For IC canister usage, these types also map to Candid types. See the IC Data Types reference for the Candid mapping.

Conversion examples:

#![allow(unused)]
fn main() {
// Rust primitive to wasm-dbms type
let uint: Uint32 = 42u32.into();
let text: Text = "hello".into();
let boolean: Boolean = true.into();

// wasm-dbms type to Rust primitive
let num: u32 = uint.into();
let s: String = text.into();
let b: bool = boolean.into();
}

Validation Reference

• Validation Reference
  • Overview
  • Syntax
  • Built-in Validators
    • String Length Validators
    • Format Validators
    • Case Validators
    • Locale Validators
  • Implementing Custom Validators
  • Validation Errors
  • Examples

Overview

Validators enforce constraints on data being inserted or updated. If validation fails, the operation is rejected with a Validation error.

Key points:

• Validators run after sanitizers
• Validation failure rejects the entire operation
• Multiple validators can be applied to a single field
• Validators are applied on both insert and update

Syntax

The #[validate(...)] attribute adds validation rules to fields:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    // Unit struct validator (no parameters)
    #[validate(EmailValidator)]
    pub email: Text,

    // Tuple struct validator (positional parameter)
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,
}
}

Built-in Validators

All validators are available in wasm_dbms_api::prelude.

String Length Validators

MaxStrlenValidator - Maximum string length

#![allow(unused)]
fn main() {
#[validate(MaxStrlenValidator(255))]
pub description: Text,  // Max 255 characters
}

MinStrlenValidator - Minimum string length

#![allow(unused)]
fn main() {
#[validate(MinStrlenValidator(8))]
pub password: Text,  // At least 8 characters
}

RangeStrlenValidator - String length within range

#![allow(unused)]
fn main() {
#[validate(RangeStrlenValidator(3, 50))]
pub username: Text,  // Between 3 and 50 characters
}

Format Validators

EmailValidator - Valid email format

#![allow(unused)]
fn main() {
#[validate(EmailValidator)]
pub email: Text,  // Must be valid email
}

UrlValidator - Valid URL format

#![allow(unused)]
fn main() {
#[validate(UrlValidator)]
pub website: Text,  // Must be valid URL
}

PhoneNumberValidator - Valid phone number format

#![allow(unused)]
fn main() {
#[validate(PhoneNumberValidator)]
pub phone: Text,  // Must be valid phone number
}

MimeTypeValidator - Valid MIME type format

#![allow(unused)]
fn main() {
#[validate(MimeTypeValidator)]
pub content_type: Text,  // e.g., "application/json", "image/png"
}

RgbColorValidator - Valid RGB color format

#![allow(unused)]
fn main() {
#[validate(RgbColorValidator)]
pub color: Text,  // e.g., "#FF5733", "rgb(255, 87, 51)"
}

Case Validators

CamelCaseValidator - Must be camelCase

#![allow(unused)]
fn main() {
#[validate(CamelCaseValidator)]
pub identifier: Text,  // e.g., "myVariableName"
}

KebabCaseValidator - Must be kebab-case

#![allow(unused)]
fn main() {
#[validate(KebabCaseValidator)]
pub slug: Text,  // e.g., "my-page-slug"
}

SnakeCaseValidator - Must be snake_case

#![allow(unused)]
fn main() {
#[validate(SnakeCaseValidator)]
pub code: Text,  // e.g., "my_constant_name"
}

Locale Validators

CountryIso639Validator - ISO 639 language code

#![allow(unused)]
fn main() {
#[validate(CountryIso639Validator)]
pub language: Text,  // e.g., "en", "es", "fr"
}

CountryIso3166Validator - ISO 3166 country code

#![allow(unused)]
fn main() {
#[validate(CountryIso3166Validator)]
pub country: Text,  // e.g., "US", "GB", "DE"
}

Implementing Custom Validators

Create a struct implementing the Validate trait:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::{Validate, Value, DbmsResult, DbmsError};

/// Validates that a number is positive
pub struct PositiveValidator;

impl Validate for PositiveValidator {
    fn validate(&self, value: &Value) -> DbmsResult<()> {
        match value {
            Value::Int32(n) if n.0 > 0 => Ok(()),
            Value::Int64(n) if n.0 > 0 => Ok(()),
            Value::Decimal(d) if d.0 > rust_decimal::Decimal::ZERO => Ok(()),
            Value::Int32(_) | Value::Int64(_) | Value::Decimal(_) => {
                Err(DbmsError::Validation("Value must be positive".to_string()))
            }
            _ => Err(DbmsError::Validation("PositiveValidator only applies to numeric types".to_string()))
        }
    }
}

// Usage
#[derive(Table, ...)]
#[table = "products"]
pub struct Product {
    #[primary_key]
    pub id: Uint32,
    #[validate(PositiveValidator)]
    pub price: Decimal,
}
}

Custom validator with parameters (tuple struct):

#![allow(unused)]
fn main() {
/// Validates that a string matches a regex pattern
pub struct RegexValidator(pub &'static str);

impl Validate for RegexValidator {
    fn validate(&self, value: &Value) -> DbmsResult<()> {
        if let Value::Text(text) = value {
            let re = regex::Regex::new(self.0).unwrap();
            if re.is_match(text.as_str()) {
                return Ok(());
            }
        }
        Err(DbmsError::Validation(
            format!("Value does not match pattern: {}", self.0)
        ))
    }
}

// Usage
#[validate(RegexValidator(r"^[A-Z]{2}-\d{4}$"))]
pub product_code: Text,  // Must match "XX-1234" format
}

Custom validator with named parameters:

#![allow(unused)]
fn main() {
/// Validates a number is within a range
pub struct RangeValidator {
    pub min: i64,
    pub max: i64,
}

impl Validate for RangeValidator {
    fn validate(&self, value: &Value) -> DbmsResult<()> {
        let num = match value {
            Value::Int32(n) => n.0 as i64,
            Value::Int64(n) => n.0,
            _ => return Err(DbmsError::Validation("RangeValidator requires integer".to_string())),
        };

        if num >= self.min && num <= self.max {
            Ok(())
        } else {
            Err(DbmsError::Validation(
                format!("Value must be between {} and {}", self.min, self.max)
            ))
        }
    }
}

// Usage
#[validate(RangeValidator, min = 1, max = 100)]
pub percentage: Int32,
}

Validation Errors

When validation fails, a DbmsError::Validation(String) is returned:

#![allow(unused)]
fn main() {
let result = database.insert::<User>(user);

match result {
    Ok(()) => println!("Insert successful"),
    Err(DbmsError::Validation(msg)) => {
        println!("Validation failed: {}", msg);
        // e.g., "Invalid email format"
        // e.g., "String length exceeds maximum of 100"
    }
    Err(e) => println!("Other error: {:?}", e),
}
}

Examples

Comprehensive user validation:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[validate(RangeStrlenValidator(2, 50))]
    pub name: Text,

    #[validate(EmailValidator)]
    pub email: Text,

    #[validate(MinStrlenValidator(8))]
    pub password_hash: Text,

    #[validate(PhoneNumberValidator)]
    pub phone: Nullable<Text>,

    #[validate(UrlValidator)]
    pub website: Nullable<Text>,

    #[validate(CountryIso3166Validator)]
    pub country: Nullable<Text>,

    #[validate(CountryIso639Validator)]
    pub language: Text,
}
}

Product validation:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "products"]
pub struct Product {
    #[primary_key]
    pub id: Uuid,

    #[validate(RangeStrlenValidator(1, 200))]
    pub name: Text,

    #[validate(MaxStrlenValidator(2000))]
    pub description: Text,

    #[validate(KebabCaseValidator)]
    pub slug: Text,

    #[validate(MimeTypeValidator)]
    pub image_type: Nullable<Text>,

    #[validate(RgbColorValidator)]
    pub accent_color: Nullable<Text>,
}
}

Combined with sanitizers:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "articles"]
pub struct Article {
    #[primary_key]
    pub id: Uuid,

    // Sanitize first, then validate
    #[sanitizer(TrimSanitizer)]
    #[validate(MaxStrlenValidator(200))]
    pub title: Text,

    // Convert to slug format, then validate
    #[sanitizer(SlugSanitizer)]
    #[validate(KebabCaseValidator)]
    pub slug: Text,

    #[sanitizer(TrimSanitizer)]
    pub content: Text,
}
}

Sanitization Reference

• Overview
• Syntax
• Built-in Sanitizers
  • String Sanitizers
  • Numeric Sanitizers
  • DateTime Sanitizers
  • Null Sanitizers
• Implementing Custom Sanitizers
• Sanitization Order
• Examples

Overview

Sanitizers automatically transform data before it’s stored in the database. Unlike validators (which reject invalid data), sanitizers modify data to conform to expected formats.

Key points:

• Sanitizers run before validators
• Data is transformed, not rejected
• Multiple sanitizers can be chained
• Sanitizers apply on both insert and update

Syntax

The #[sanitizer(...)] attribute adds sanitization rules to fields:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    // Unit struct sanitizer (no parameters)
    #[sanitizer(TrimSanitizer)]
    pub name: Text,

    // Tuple struct sanitizer (positional parameter)
    #[sanitizer(RoundToScaleSanitizer(2))]
    pub balance: Decimal,

    // Named fields sanitizer
    #[sanitizer(ClampSanitizer, min = 0, max = 120)]
    pub age: Uint8,
}
}

Built-in Sanitizers

All sanitizers are available in wasm_dbms_api::prelude.

String Sanitizers

TrimSanitizer - Remove leading/trailing whitespace

#![allow(unused)]
fn main() {
#[sanitizer(TrimSanitizer)]
pub name: Text,
// "  Alice  " → "Alice"
}

CollapseWhitespaceSanitizer - Collapse multiple spaces into one

#![allow(unused)]
fn main() {
#[sanitizer(CollapseWhitespaceSanitizer)]
pub description: Text,
// "Hello    World" → "Hello World"
}

LowerCaseSanitizer - Convert to lowercase

#![allow(unused)]
fn main() {
#[sanitizer(LowerCaseSanitizer)]
pub email: Text,
// "Alice@Example.COM" → "alice@example.com"
}

UpperCaseSanitizer - Convert to uppercase

#![allow(unused)]
fn main() {
#[sanitizer(UpperCaseSanitizer)]
pub country_code: Text,
// "us" → "US"
}

SlugSanitizer - Convert to URL-safe slug

#![allow(unused)]
fn main() {
#[sanitizer(SlugSanitizer)]
pub slug: Text,
// "Hello World! This is a Test" → "hello-world-this-is-a-test"
}

UrlEncodingSanitizer - URL encode special characters

#![allow(unused)]
fn main() {
#[sanitizer(UrlEncodingSanitizer)]
pub path: Text,
// "hello world" → "hello%20world"
}

Numeric Sanitizers

RoundToScaleSanitizer - Round decimal to specific precision

#![allow(unused)]
fn main() {
#[sanitizer(RoundToScaleSanitizer(2))]
pub price: Decimal,
// 19.999 → 20.00
// 19.994 → 19.99
}

ClampSanitizer - Clamp value to range (signed)

#![allow(unused)]
fn main() {
#[sanitizer(ClampSanitizer, min = -100, max = 100)]
pub temperature: Int32,
// 150 → 100
// -150 → -100
}

ClampUnsignedSanitizer - Clamp value to range (unsigned)

#![allow(unused)]
fn main() {
#[sanitizer(ClampUnsignedSanitizer, min = 0, max = 100)]
pub percentage: Uint8,
// 150 → 100
// 0 → 0
}

DateTime Sanitizers

TimezoneSanitizer - Convert to specific timezone

#![allow(unused)]
fn main() {
#[sanitizer(TimezoneSanitizer("America/New_York"))]
pub local_time: DateTime,
}

UtcSanitizer - Convert to UTC

#![allow(unused)]
fn main() {
#[sanitizer(UtcSanitizer)]
pub timestamp: DateTime,
// Any timezone → UTC
}

Null Sanitizers

NullIfEmptySanitizer - Convert empty strings to null

#![allow(unused)]
fn main() {
#[sanitizer(NullIfEmptySanitizer)]
pub bio: Nullable<Text>,
// "" → Null
// "Hello" → "Hello"
}

Implementing Custom Sanitizers

Create a struct implementing the Sanitize trait:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::{Sanitize, Value, DbmsResult};

/// Capitalizes the first letter of each word
pub struct TitleCaseSanitizer;

impl Sanitize for TitleCaseSanitizer {
    fn sanitize(&self, value: Value) -> DbmsResult<Value> {
        match value {
            Value::Text(text) => {
                let title_case = text
                    .as_str()
                    .split_whitespace()
                    .map(|word| {
                        let mut chars = word.chars();
                        match chars.next() {
                            None => String::new(),
                            Some(first) => {
                                first.to_uppercase().to_string() +
                                chars.as_str().to_lowercase().as_str()
                            }
                        }
                    })
                    .collect::<Vec<_>>()
                    .join(" ");
                Ok(Value::Text(title_case.into()))
            }
            other => Ok(other),  // Pass through non-text values
        }
    }
}

// Usage
#[sanitizer(TitleCaseSanitizer)]
pub title: Text,
// "hello world" → "Hello World"
}

Custom sanitizer with parameters:

#![allow(unused)]
fn main() {
/// Truncates string to max length
pub struct TruncateSanitizer(pub usize);

impl Sanitize for TruncateSanitizer {
    fn sanitize(&self, value: Value) -> DbmsResult<Value> {
        match value {
            Value::Text(text) => {
                let truncated: String = text.as_str().chars().take(self.0).collect();
                Ok(Value::Text(truncated.into()))
            }
            other => Ok(other),
        }
    }
}

// Usage
#[sanitizer(TruncateSanitizer(100))]
pub summary: Text,
// "very long text..." → truncated to 100 chars
}

Custom sanitizer with named parameters:

#![allow(unused)]
fn main() {
/// Replaces a pattern with replacement
pub struct ReplaceSanitizer {
    pub pattern: &'static str,
    pub replacement: &'static str,
}

impl Sanitize for ReplaceSanitizer {
    fn sanitize(&self, value: Value) -> DbmsResult<Value> {
        match value {
            Value::Text(text) => {
                let replaced = text.as_str().replace(self.pattern, self.replacement);
                Ok(Value::Text(replaced.into()))
            }
            other => Ok(other),
        }
    }
}

// Usage
#[sanitizer(ReplaceSanitizer, pattern = "\n", replacement = " ")]
pub single_line: Text,
}

Sanitization Order

When multiple sanitizers are applied, they run in declaration order:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    // Order matters!
    #[sanitizer(TrimSanitizer)]              // 1. Trim whitespace
    #[sanitizer(CollapseWhitespaceSanitizer)] // 2. Collapse spaces
    #[sanitizer(LowerCaseSanitizer)]         // 3. Lowercase
    pub email: Text,
}

// Input: "  Alice@Example.COM  "
// After TrimSanitizer: "Alice@Example.COM"
// After CollapseWhitespaceSanitizer: "Alice@Example.COM" (no change)
// After LowerCaseSanitizer: "alice@example.com"
}

Sanitizers run before validators:

#![allow(unused)]
fn main() {
#[sanitizer(TrimSanitizer)]           // 1. Trim
#[validate(MaxStrlenValidator(100))]  // 2. Validate length (after trim)
pub name: Text,
}

Examples

User profile sanitization:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    // Clean up name
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(CollapseWhitespaceSanitizer)]
    pub name: Text,

    // Normalize email
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(LowerCaseSanitizer)]
    pub email: Text,

    // Convert empty to null
    #[sanitizer(NullIfEmptySanitizer)]
    pub bio: Nullable<Text>,

    // Uppercase country code
    #[sanitizer(UpperCaseSanitizer)]
    pub country: Nullable<Text>,
}
}

Financial data sanitization:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "transactions"]
pub struct Transaction {
    #[primary_key]
    pub id: Uuid,

    // Round to cents
    #[sanitizer(RoundToScaleSanitizer(2))]
    pub amount: Decimal,

    // Ensure positive (clamp negatives to 0)
    #[sanitizer(ClampUnsignedSanitizer, min = 0, max = 1000000)]
    pub fee: Uint32,

    // Always store in UTC
    #[sanitizer(UtcSanitizer)]
    pub timestamp: DateTime,
}
}

Content sanitization:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "articles"]
pub struct Article {
    #[primary_key]
    pub id: Uuid,

    // Clean title
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(CollapseWhitespaceSanitizer)]
    pub title: Text,

    // Generate URL-safe slug
    #[sanitizer(SlugSanitizer)]
    pub slug: Text,

    // Clean up content
    #[sanitizer(TrimSanitizer)]
    pub content: Text,

    // Optional summary
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(NullIfEmptySanitizer)]
    pub summary: Nullable<Text>,
}
}

Combined sanitization and validation:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "products"]
pub struct Product {
    #[primary_key]
    pub id: Uuid,

    // Sanitize then validate
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(CollapseWhitespaceSanitizer)]
    #[validate(RangeStrlenValidator(1, 200))]
    pub name: Text,

    // Sanitize price to 2 decimals, no validation needed
    #[sanitizer(RoundToScaleSanitizer(2))]
    pub price: Decimal,

    // Create slug and validate format
    #[sanitizer(SlugSanitizer)]
    #[validate(KebabCaseValidator)]
    #[validate(MaxStrlenValidator(100))]
    pub slug: Text,

    // Clean URL and validate format
    #[sanitizer(TrimSanitizer)]
    #[validate(UrlValidator)]
    pub image_url: Nullable<Text>,
}
}

JSON Reference

• Overview
• Defining JSON Columns
• Creating JSON Values
• JSON Filtering
  • Path Syntax
• Filter Operations
  • Contains (Structural Containment)
  • Extract (Path Extraction + Comparison)
  • HasKey (Path Existence)
• Combining JSON Filters
• Type Conversion
• Complete Example
• Error Handling

Overview

The Json data type allows you to store and query semi-structured JSON data within your database tables. This is useful for:

• Flexible schemas where structure varies between records
• Metadata storage
• User preferences and settings
• Any scenario where data structure may evolve

Defining JSON Columns

To use JSON in your schema, use the Json type:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    pub metadata: Json,          // Required JSON field
    pub settings: Nullable<Json>, // Optional JSON field
}
}

Creating JSON Values

From string:

#![allow(unused)]
fn main() {
use std::str::FromStr;
use wasm_dbms_api::prelude::Json;

let json = Json::from_str(r#"{"name": "Alice", "age": 30}"#).unwrap();
}

From serde_json::Value:

#![allow(unused)]
fn main() {
use serde_json::json;
use wasm_dbms_api::prelude::Json;

let json: Json = json!({
    "name": "Alice",
    "age": 30,
    "tags": ["developer", "rust"],
    "address": {
        "city": "New York",
        "country": "US"
    }
}).into();
}

In insert requests:

#![allow(unused)]
fn main() {
let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    metadata: json!({
        "role": "admin",
        "permissions": ["read", "write", "delete"]
    }).into(),
    settings: Nullable::Value(json!({
        "theme": "dark",
        "notifications": true
    }).into()),
};
}

JSON Filtering

wasm-dbms provides powerful JSON filtering through the JsonFilter enum:

• Contains: Check if JSON contains a pattern (structural containment)
• Extract: Extract value at path and compare
• HasKey: Check if a path exists

Path Syntax

Paths use dot notation with bracket array indices:

Path examples:

{
  "name": "Alice",           // Path: "name"
  "user": {
    "email": "a@b.com"       // Path: "user.email"
  },
  "tags": ["a", "b", "c"],   // Path: "tags[0]" = "a"
  "matrix": [[1,2], [3,4]]   // Path: "matrix[1][0]" = 3
}

Filter Operations

Contains (Structural Containment)

Checks if the JSON column contains a specified pattern. Implements PostgreSQL @> style containment:

• Objects: All key-value pairs in pattern must exist in target (recursive)
• Arrays: All elements in pattern must exist in target (order-independent)
• Primitives: Must be equal

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;
use std::str::FromStr;

// Filter where metadata contains {"active": true}
let pattern = Json::from_str(r#"{"active": true}"#).unwrap();
let filter = Filter::json("metadata", JsonFilter::contains(pattern));
}

Containment behavior:

Use cases:

• Check if user has specific role: contains({"role": "admin"})
• Check if array contains value: contains({"tags": ["important"]})
• Check nested properties: contains({"settings": {"theme": "dark"}})

Extract (Path Extraction + Comparison)

Extract a value at path and apply comparison:

#![allow(unused)]
fn main() {
// Equal
let filter = Filter::json("metadata",
    JsonFilter::extract_eq("user.name", Value::Text("Alice".into()))
);

// Greater than
let filter = Filter::json("metadata",
    JsonFilter::extract_gt("user.age", Value::Int64(18.into()))
);

// In list
let filter = Filter::json("metadata",
    JsonFilter::extract_in("status", vec![
        Value::Text("active".into()),
        Value::Text("pending".into()),
    ])
);

// Is null (path doesn't exist or value is null)
let filter = Filter::json("metadata",
    JsonFilter::extract_is_null("deleted_at")
);

// Not null (path exists and value is not null)
let filter = Filter::json("metadata",
    JsonFilter::extract_not_null("email")
);
}

Available comparison methods:

HasKey (Path Existence)

Check if a path exists in the JSON:

#![allow(unused)]
fn main() {
// Check for root-level key
let filter = Filter::json("metadata", JsonFilter::has_key("email"));

// Check for nested path
let filter = Filter::json("metadata", JsonFilter::has_key("user.address.city"));

// Check for array element
let filter = Filter::json("metadata", JsonFilter::has_key("items[0]"));
}

Note: HasKey returns true even if the value at path is null. It only checks for path existence.

Combining JSON Filters

JSON filters combine with other filters using and(), or(), not():

#![allow(unused)]
fn main() {
// has email AND age > 18
let filter = Filter::json("metadata", JsonFilter::has_key("email"))
    .and(Filter::json("metadata", JsonFilter::extract_gt("age", Value::Int64(18.into()))));

// role = "admin" OR role = "moderator"
let filter = Filter::json("metadata", JsonFilter::extract_eq("role", Value::Text("admin".into())))
    .or(Filter::json("metadata", JsonFilter::extract_eq("role", Value::Text("moderator".into()))));

// Combine with regular filters
let pattern = Json::from_str(r#"{"active": true}"#).unwrap();
let filter = Filter::eq("id", Value::Int32(1.into()))
    .and(Filter::json("metadata", JsonFilter::contains(pattern)));

// NOT has deleted_at
let filter = Filter::json("metadata", JsonFilter::has_key("deleted_at")).not();
}

Type Conversion

When extracting JSON values, they’re converted to DBMS types:

Comparison examples:

#![allow(unused)]
fn main() {
// JSON: {"count": 42}
// Extracted as Int64, compare with Int64
JsonFilter::extract_eq("count", Value::Int64(42.into()))

// JSON: {"price": 19.99}
// Extracted as Decimal, compare with Decimal
JsonFilter::extract_gt("price", Value::Decimal(10.0.into()))

// JSON: {"active": true}
// Extracted as Boolean
JsonFilter::extract_eq("active", Value::Boolean(true))

// JSON: {"name": "Alice"}
// Extracted as Text
JsonFilter::extract_eq("name", Value::Text("Alice".into()))
}

Complete Example

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::*;
use std::str::FromStr;

#[derive(Debug, Table, Clone, PartialEq, Eq)]
#[table = "products"]
pub struct Product {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    pub attributes: Json,  // {"color": "red", "size": "M", "tags": ["sale", "new"], "price": 29.99}
}

fn example_queries(database: &impl Database) -> Result<(), Box<dyn std::error::Error>> {
    // Find all red products
    let filter = Filter::json("attributes",
        JsonFilter::extract_eq("color", Value::Text("red".into()))
    );
    let query = Query::builder().filter(filter).build();
    let red_products = database.select::<Product>(query)?;

    // Find products with "sale" tag
    let pattern = Json::from_str(r#"{"tags": ["sale"]}"#)?;
    let filter = Filter::json("attributes", JsonFilter::contains(pattern));
    let query = Query::builder().filter(filter).build();
    let sale_products = database.select::<Product>(query)?;

    // Find products with size attribute
    let filter = Filter::json("attributes", JsonFilter::has_key("size"));
    let query = Query::builder().filter(filter).build();
    let sized_products = database.select::<Product>(query)?;

    // Find red products with price > 20
    let filter = Filter::json("attributes", JsonFilter::extract_eq("color", Value::Text("red".into())))
        .and(Filter::json("attributes", JsonFilter::extract_gt("price", Value::Decimal(20.0.into()))));
    let query = Query::builder().filter(filter).build();
    let expensive_red = database.select::<Product>(query)?;

    // Find products in specific sizes
    let filter = Filter::json("attributes",
        JsonFilter::extract_in("size", vec![
            Value::Text("S".into()),
            Value::Text("M".into()),
            Value::Text("L".into()),
        ])
    );
    let query = Query::builder().filter(filter).build();
    let standard_sizes = database.select::<Product>(query)?;

    Ok(())
}
}

Error Handling

JSON filter operations return errors for:

Invalid path syntax:

• Empty paths
• Trailing dots ("user.")
• Unclosed brackets ("items[0")
• Negative indices ("items[-1]")
• Non-numeric array indices ("items[abc]")

Non-JSON column:

• Applying JSON filter to a non-JSON column

#![allow(unused)]
fn main() {
// Invalid path - will error
let filter = Filter::json("metadata", JsonFilter::has_key("user."));  // Trailing dot

// Non-JSON column - will error
let filter = Filter::json("name", JsonFilter::has_key("field"));  // "name" is Text, not Json

let result = database.select::<User>(
    Query::builder().filter(filter).build(),
);

match result {
    Err(DbmsError::Query(QueryError::InvalidQuery)) => {
        println!("Invalid JSON filter");
    }
    _ => {}
}
}

Errors Reference

• Errors Reference
  • Overview
  • Error Hierarchy
  • DbmsError
  • Query Errors
    • PrimaryKeyConflict
    • UniqueConstraintViolation
    • BrokenForeignKeyReference
    • ForeignKeyConstraintViolation
    • UnknownColumn
    • MissingNonNullableField
    • RecordNotFound
    • InvalidQuery
  • Transaction Errors
    • TransactionNotFound
  • Validation Errors
  • Sanitization Errors
  • Memory Errors
  • Error Handling Examples

Overview

wasm-dbms uses a structured error system to provide clear information about what went wrong. Errors are categorized by their source:

Error Hierarchy

DbmsError
├── Query(QueryError)
│   ├── PrimaryKeyConflict
│   ├── UniqueConstraintViolation
│   ├── BrokenForeignKeyReference
│   ├── ForeignKeyConstraintViolation
│   ├── UnknownColumn
│   ├── MissingNonNullableField
│   ├── RecordNotFound
│   └── InvalidQuery
├── Transaction(TransactionError)
│   └── NotFound
├── Validation(String)
├── Sanitize(String)
├── Memory(MemoryError)
└── Table(TableError)

DbmsError

The top-level error enum:

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::DbmsError;

pub enum DbmsError {
    Memory(MemoryError),
    Query(QueryError),
    Table(TableError),
    Transaction(TransactionError),
    Sanitize(String),
    Validation(String),
}
}

Matching on error types:

#![allow(unused)]
fn main() {
match error {
    DbmsError::Query(query_err) => {
        // Handle query errors
    }
    DbmsError::Transaction(tx_err) => {
        // Handle transaction errors
    }
    DbmsError::Validation(msg) => {
        // Handle validation errors
        println!("Validation failed: {}", msg);
    }
    DbmsError::Sanitize(msg) => {
        // Handle sanitization errors
        println!("Sanitization failed: {}", msg);
    }
    DbmsError::Memory(mem_err) => {
        // Handle memory errors (rare)
    }
    DbmsError::Table(table_err) => {
        // Handle table errors (rare)
    }
}
}

Query Errors

Query errors occur during database operations.

PrimaryKeyConflict

Cause: Attempting to insert a record with a primary key that already exists.

#![allow(unused)]
fn main() {
// Insert first user
database.insert::<User>(UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    ..
})?;

// Insert second user with same ID - FAILS
let result = database.insert::<User>(UserInsertRequest {
    id: 1.into(),  // Same ID!
    name: "Bob".into(),
    ..
});

match result {
    Err(DbmsError::Query(QueryError::PrimaryKeyConflict)) => {
        println!("A user with this ID already exists");
    }
    _ => {}
}
}

Solutions:

• Use a unique primary key (e.g., UUID)
• Check if record exists before inserting
• Use upsert pattern (check, then insert or update)

UniqueConstraintViolation

Cause: Attempting to insert or update a record with a value that violates a #[unique] constraint.

#![allow(unused)]
fn main() {
// Insert first user
database.insert::<User>(UserInsertRequest {
    id: 1.into(),
    email: "alice@example.com".into(),
    ..
})?;

// Insert second user with same email - FAILS
let result = database.insert::<User>(UserInsertRequest {
    id: 2.into(),
    email: "alice@example.com".into(),  // Duplicate!
    ..
});

match result {
    Err(DbmsError::Query(QueryError::UniqueConstraintViolation { field })) => {
        println!("Duplicate value on field: {}", field);
        // field == "email"
    }
    _ => {}
}
}

Also triggered on update:

#![allow(unused)]
fn main() {
// Update user 2's email to match user 1's email - FAILS
let result = database.update::<User>(
    UserUpdateRequest::from_values(
        &[(email_col, Value::Text("alice@example.com".into()))],
        Some(Filter::eq("id", Value::Uint32(2.into()))),
    ),
);
}

Solutions:

• Check if a record with the same value exists before inserting
• Use a different value

BrokenForeignKeyReference

Cause: Foreign key references a record that doesn’t exist.

#![allow(unused)]
fn main() {
// Insert post with non-existent author
let result = database.insert::<Post>(PostInsertRequest {
    id: 1.into(),
    title: "My Post".into(),
    author_id: 999.into(),  // User 999 doesn't exist!
    ..
});

match result {
    Err(DbmsError::Query(QueryError::BrokenForeignKeyReference)) => {
        println!("Referenced user does not exist");
    }
    _ => {}
}
}

Solutions:

• Ensure referenced record exists before inserting
• Create referenced record first in a transaction

ForeignKeyConstraintViolation

Cause: Attempting to delete a record that is referenced by other records (with Restrict behavior).

#![allow(unused)]
fn main() {
// User has posts - cannot delete with Restrict
let result = database.delete::<User>(
    DeleteBehavior::Restrict,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
);

match result {
    Err(DbmsError::Query(QueryError::ForeignKeyConstraintViolation)) => {
        println!("Cannot delete: user has related records");
    }
    _ => {}
}
}

Solutions:

• Delete related records first
• Use DeleteBehavior::Cascade to delete related records automatically

UnknownColumn

Cause: Referencing a column that doesn’t exist in the table.

#![allow(unused)]
fn main() {
// Filter with wrong column name
let filter = Filter::eq("username", Value::Text("alice".into()));  // Column is "name", not "username"

let result = database.select::<User>(
    Query::builder().filter(filter).build(),
);

match result {
    Err(DbmsError::Query(QueryError::UnknownColumn)) => {
        println!("Column does not exist in table");
    }
    _ => {}
}
}

Solutions:

• Check column names in your schema
• Use IDE autocompletion with typed column names

MissingNonNullableField

Cause: Required field not provided in insert/update.

#![allow(unused)]
fn main() {
// This typically happens at compile time with the generated types,
// but can occur if manually constructing requests or using dynamic queries
}

Solutions:

• Provide all required fields
• Use Nullable<T> for optional fields

RecordNotFound

Cause: Operation targets a record that doesn’t exist.

#![allow(unused)]
fn main() {
// Update non-existent record
let update = UserUpdateRequest::builder()
    .set_name("New Name".into())
    .filter(Filter::eq("id", Value::Uint32(999.into())))  // Doesn't exist
    .build();

let affected = database.update::<User>(update)?;

// affected == 0 indicates no records matched
if affected == 0 {
    println!("No records found to update");
}
}

Note: Update and delete operations return the count of affected rows. A count of 0 isn’t necessarily an error but indicates no matches.

InvalidQuery

Cause: Malformed query (invalid JSON path, bad filter syntax, etc.).

#![allow(unused)]
fn main() {
// Invalid JSON path
let filter = Filter::json("metadata", JsonFilter::has_key("user."));  // Trailing dot

let result = database.select::<User>(
    Query::builder().filter(filter).build(),
);

match result {
    Err(DbmsError::Query(QueryError::InvalidQuery)) => {
        println!("Query is malformed");
    }
    _ => {}
}
}

Common causes:

• Invalid JSON paths (trailing dots, unclosed brackets)
• Applying JSON filter to non-JSON column
• Type mismatches in comparisons

Transaction Errors

TransactionNotFound

Cause: Invalid transaction ID or transaction already completed.

#![allow(unused)]
fn main() {
use wasm_dbms_api::prelude::{DbmsError, TransactionError};

match database.commit() {
    Err(DbmsError::Transaction(TransactionError::NoActiveTransaction)) => {
        println!("No active transaction to commit");
    }
    _ => {}
}
}

Causes:

• Transaction ID never existed
• Transaction was already committed
• Transaction was already rolled back

Validation Errors

Cause: Data fails validation rules.

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "users"]
pub struct User {
    #[validate(EmailValidator)]
    pub email: Text,
}

// Insert with invalid email
let result = database.insert::<User>(UserInsertRequest {
    id: 1.into(),
    email: "not-an-email".into(),  // Invalid!
    ..
});

match result {
    Err(DbmsError::Validation(msg)) => {
        println!("Validation failed: {}", msg);
        // msg might be: "Invalid email format"
    }
    _ => {}
}
}

Common validation errors:

• String too long (MaxStrlenValidator)
• String too short (MinStrlenValidator)
• Invalid email format (EmailValidator)
• Invalid URL format (UrlValidator)
• Invalid phone format (PhoneNumberValidator)

Sanitization Errors

Cause: Sanitizer fails to process the data.

#![allow(unused)]
fn main() {
// Sanitization errors are rare but can occur with malformed data
match result {
    Err(DbmsError::Sanitize(msg)) => {
        println!("Sanitization failed: {}", msg);
    }
    _ => {}
}
}

Sanitization errors are less common than validation errors since sanitizers typically transform data rather than reject it.

Memory Errors

Cause: Low-level memory errors.

#![allow(unused)]
fn main() {
pub enum MemoryError {
    OutOfBounds,           // Read/write outside allocated memory
    ProviderError(String),      // Memory provider error
    InsufficientSpace,     // Not enough space to allocate
}
}

Memory errors are rare and usually indicate:

• Running out of available memory
• Corrupted memory state
• Bug in wasm-dbms (please report!)

Error Handling Examples

Basic error handling:

#![allow(unused)]
fn main() {
let result = database.insert::<User>(user);

match result {
    Ok(()) => println!("Insert successful"),
    Err(DbmsError::Query(QueryError::PrimaryKeyConflict)) => {
        println!("User already exists");
    }
    Err(DbmsError::Query(QueryError::UniqueConstraintViolation { field })) => {
        println!("Duplicate value on field: {}", field);
    }
    Err(DbmsError::Query(QueryError::BrokenForeignKeyReference)) => {
        println!("Referenced record doesn't exist");
    }
    Err(DbmsError::Validation(msg)) => {
        println!("Validation error: {}", msg);
    }
    Err(e) => {
        println!("Database error: {:?}", e);
    }
}
}

Helper function pattern:

#![allow(unused)]
fn main() {
fn handle_db_error(error: DbmsError) -> String {
    match error {
        DbmsError::Query(QueryError::PrimaryKeyConflict) =>
            "Record with this ID already exists".to_string(),
        DbmsError::Query(QueryError::UniqueConstraintViolation { field }) =>
            format!("Duplicate value on unique field: {}", field),
        DbmsError::Query(QueryError::BrokenForeignKeyReference) =>
            "Referenced record not found".to_string(),
        DbmsError::Query(QueryError::ForeignKeyConstraintViolation) =>
            "Cannot delete: record has dependencies".to_string(),
        DbmsError::Validation(msg) =>
            format!("Invalid data: {}", msg),
        _ =>
            format!("Unexpected error: {:?}", error),
    }
}
}

For IC client-specific error handling (double result pattern with CallError), see the IC Errors Reference.

Architecture

• Architecture
  • Overview
  • Three-Layer Architecture
    • Layer 1: Memory Layer
    • Layer 2: DBMS Layer
    • Layer 3: API Layer
  • Crate Organization
    • Dependency Graph
    • Generic Layer (wasm-dbms)
      • wasm-dbms-api
      • wasm-dbms-memory
      • wasm-dbms
      • wasm-dbms-macros
    • IC Layer (ic-dbms)
      • ic-dbms-api
      • ic-dbms-canister
      • ic-dbms-macros
      • ic-dbms-client
  • Data Flow
    • Insert Operation
    • Select Operation
    • Select with Join
    • Transaction Flow
  • Extension Points
    • Custom Sanitizers
    • Custom Validators
    • Custom Data Types
    • Memory Provider

Overview

wasm-dbms is built as a layered architecture where each layer has specific responsibilities and builds upon the layer below. The core DBMS engine is runtime-agnostic (wasm-dbms-* crates), while the IC-specific adapter layer (ic-dbms-* crates) provides Internet Computer integration.

This design provides:

• Separation of concerns: Each layer focuses on one aspect
• Testability: Layers can be tested independently
• Portability: The generic layer runs on any WASM runtime (Wasmtime, Wasmer, WasmEdge)
• Flexibility: Internal implementations can change without affecting APIs

Three-Layer Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Layer 3: API Layer                       │
│  Canister endpoints, Candid interface, access control        │
│  (DbmsCanister macro, ACL guards, request/response types)    │
├─────────────────────────────────────────────────────────────┤
│                     Layer 2: DBMS Layer                      │
│  Tables, CRUD operations, transactions, foreign keys         │
│  (TableRegistry, TransactionManager, query execution)        │
├─────────────────────────────────────────────────────────────┤
│                    Layer 1: Memory Layer                     │
│  Stable memory management, encoding/decoding, page allocation│
│  (MemoryProvider, MemoryManager, Encode trait)               │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │  IC Stable      │
                    │    Memory       │
                    │   (or Heap)     │
                    └─────────────────┘

Layer 1: Memory Layer

Crate: wasm-dbms-memory

Responsibilities:

• Manage stable memory allocation (64 KiB pages)
• Encode/decode data to/from binary format
• Track free space and handle fragmentation
• Provide abstraction for testing (heap vs stable memory)

Key components:

Memory layout:

Page 0: Schema Registry (table → page mapping)
Page 1: ACL (allowed principals)
Page 2+: Table data (Page Ledger, Free Segments, Index Ledger, Autoincrement Ledger, Records, B-tree nodes)

See Memory Documentation for detailed technical information.

Layer 2: DBMS Layer

Crate: wasm-dbms

Responsibilities:

• Implement CRUD operations
• Manage transactions with ACID properties
• Enforce foreign key constraints
• Handle sanitization and validation
• Execute queries with filters

Key components:

Transaction model:

┌──────────────────────────────────────────┐
│           Active Transactions             │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐   │
│  │  Tx 1   │  │  Tx 2   │  │  Tx 3   │   │
│  │ (overlay)│  │(overlay)│  │(overlay)│   │
│  └────┬────┘  └────┬────┘  └────┬────┘   │
│       │            │            │         │
│       └────────────┼────────────┘         │
│                    │                      │
│                    ▼                      │
│         ┌─────────────────┐              │
│         │  Committed Data │              │
│         │   (in memory)   │              │
│         └─────────────────┘              │
└──────────────────────────────────────────┘

Transactions use an overlay pattern:

• Changes are written to an overlay (in-memory)
• Reading checks overlay first, then committed data
• Index changes are tracked in a separate IndexOverlay per table
• IndexReader merges base index results with overlay additions/removals
• Commit merges overlay to committed data and flushes index changes to B-trees
• Rollback discards the overlay — on-disk B-trees remain untouched

Layer 3: API Layer

Crate: ic-dbms-canister (IC-specific)

Responsibilities:

• Expose Candid interface
• Handle request/response encoding
• Enforce access control (ACL)
• Route requests to DBMS layer
• Generate table-specific endpoints

Key components:

Generated API structure:

#![allow(unused)]
fn main() {
// For each table "users":
insert_users(UserInsertRequest, Option<TxId>) -> Result<()>
select_users(Query, Option<TxId>) -> Result<Vec<UserRecord>>
update_users(UserUpdateRequest, Option<TxId>) -> Result<u64>
delete_users(DeleteBehavior, Option<Filter>, Option<TxId>) -> Result<u64>

// Untyped select (supports joins):
select(table: String, Query, Option<TxId>) -> Result<Vec<Vec<(JoinColumnDef, Value)>>>

// Global operations:
begin_transaction() -> TxId
commit(TxId) -> Result<()>
rollback(TxId) -> Result<()>
acl_add_principal(Principal) -> Result<()>
acl_remove_principal(Principal) -> Result<()>
acl_allowed_principals() -> Vec<Principal>
}

Crate Organization

wasm-dbms/
├── crates/
│   ├── wasm-dbms/                  # Generic WASM DBMS crates
│   │   ├── wasm-dbms-api/          # Shared types and traits
│   │   ├── wasm-dbms-memory/       # Memory abstraction and page management
│   │   ├── wasm-dbms/              # Core DBMS engine
│   │   └── wasm-dbms-macros/       # Procedural macros (Encode, Table, CustomDataType, DatabaseSchema)
│   │
│   └── ic-dbms/                    # IC-specific crates
│       ├── ic-dbms-api/            # IC-specific types (re-exports wasm-dbms-api)
│       ├── ic-dbms-canister/       # Core IC canister implementation
│       ├── ic-dbms-macros/         # IC-specific macros (DatabaseSchema, DbmsCanister)
│       ├── ic-dbms-client/         # Client libraries
│       ├── example/                # Reference implementation
│       └── integration-tests/      # PocketIC integration tests
│
└── .artifact/                      # Build outputs (.wasm, .did, .wasm.gz)

Dependency Graph

wasm-dbms-macros <── wasm-dbms-api <── wasm-dbms-memory <── wasm-dbms
                                                                 ^
ic-dbms-macros <── ic-dbms-canister ─────────────────────────────┘
                        ^
                   ic-dbms-client

Generic Layer (wasm-dbms)

wasm-dbms-api

Purpose: Runtime-agnostic shared types and traits

Contents:

• Data types (Uint32, Text, DateTime, etc.)
• Value enum for runtime values
• Filter, Query, and Join types
• Database trait
• Sanitizer and Validator traits
• CustomDataType trait and CustomValue
• Error types (DbmsError, DbmsResult)

Dependencies: Minimal (serde, thiserror). Candid support via optional candid feature.

wasm-dbms-memory

Purpose: Memory abstraction and page management

Contents:

• MemoryProvider trait
• HeapMemoryProvider (testing)
• MemoryManager (page-level operations)
• SchemaRegistry (table-to-page mapping)
• AccessControl trait (identity-based ACL abstraction)
• AccessControlList (default AccessControl impl with Vec<u8> identity)
• NoAccessControl (no-op ACL for runtimes that don’t need access control)
• TableRegistry (record-level operations)

wasm-dbms

Purpose: Core DBMS engine (runtime-agnostic)

Contents:

• DbmsContext<M, A = AccessControlList> (owns all mutable state)
• WasmDbmsDatabase<'ctx, M, A> (session-scoped operations)
• Transaction management (overlay pattern)
• Foreign key integrity checks
• JOIN execution engine
• DatabaseSchema trait for dynamic dispatch

wasm-dbms-macros

Purpose: Generic procedural macros

Macros:

• #[derive(Encode)] - Binary serialization
• #[derive(Table)] - Table schema and related types
• #[derive(CustomDataType)] - Custom data type bridge
• #[derive(DatabaseSchema)] - Generates DatabaseSchema<M> trait implementation for schema dispatch

IC Layer (ic-dbms)

ic-dbms-api

Purpose: IC-specific types, re-exports generic API

Contents:

• Re-exports all types from wasm-dbms-api
• Principal custom data type (wraps candid::Principal)
• IcDbmsCanisterArgs init/upgrade arguments
• IcDbmsError / IcDbmsResult type aliases

ic-dbms-canister

Purpose: Thin IC adapter over wasm-dbms

Contents:

• IcMemoryProvider (IC stable memory)
• DBMS_CONTEXT thread-local wrapping DbmsContext<IcMemoryProvider>
• Canister API layer with ACL guards

Dependencies: ic-dbms-api, ic-dbms-macros, wasm-dbms, wasm-dbms-memory, ic-cdk

ic-dbms-macros

Purpose: IC-specific code generation

Macros:

• #[derive(DatabaseSchema)] - Generates DatabaseSchema<M> trait implementation (IC-specific paths)
• #[derive(DbmsCanister)] - Generates complete canister API

ic-dbms-client

Purpose: Client libraries for canister interaction

Implementations:

• IcDbmsCanisterClient - Inter-canister calls
• IcDbmsAgentClient - External via ic-agent (feature-gated)
• IcDbmsPocketIcClient - Testing with PocketIC (feature-gated)

Data Flow

Insert Operation

1. Client calls insert_users(request, tx_id)
              │
2. ACL guard checks caller authorization
              │
3. API layer deserializes request
              │
4. DBMS layer:
   a. Apply sanitizers to values
   b. Apply validators to values
   c. Check primary key uniqueness
   d. Validate foreign key references
   e. If tx_id: write to transaction overlay
      (index overlay tracks added keys)
      Else: write directly
              │
5. Memory layer:
   a. Encode record to bytes
   b. Find space (free segment or new page)
   c. Write to stable memory
   d. Update all indexes with the new
      key → RecordAddress mapping
              │
6. Return Result<()>

Select Operation

1. Client calls select_users(query, tx_id)
              │
2. ACL guard checks caller authorization
              │
3. API layer deserializes query
              │
4. DBMS layer:
   a. Parse filters
   b. Analyze filter for index plan
      (equality, range, or IN on indexed column)
   c. If index plan found:
      - Use IndexReader to get RecordAddresses
      - Load only matching records
      - Apply remaining filter as residual
   d. If no index plan (fallback):
      - Full table scan across all pages
   e. If tx_id: merge with overlay
   f. Apply ordering
   g. Apply limit/offset
   h. Select requested columns
   i. Handle eager loading
              │
5. Memory layer:
   a. Read pages
   b. Decode records
              │
6. Return Result<Vec<Record>>

Select with Join

1. Client calls select(table, query_with_joins, tx_id)
              │
2. ACL guard checks caller authorization
              │
3. API layer checks query.has_joins()
              │ (true)
4. JoinEngine:
   a. Read all rows from FROM table
   b. For each JOIN clause:
      - Read all rows from joined table
      - Resolve column references
      - Execute nested-loop join
   c. Apply filter on combined rows
   d. Apply ordering
   e. Apply offset/limit
   f. Flatten to output with JoinColumnDef
              │
5. Return Result<Vec<Vec<(JoinColumnDef, Value)>>>

See Join Engine for implementation details.

Transaction Flow

begin_transaction():
  1. Generate transaction ID
  2. Create empty overlay
  3. Record owner (caller identity)
  4. Return transaction ID

Operation with tx_id:
  1. Verify caller owns transaction
  2. Read from: overlay first, then committed
  3. Write to: overlay only

commit(tx_id):
  1. Verify caller owns transaction
  2. For each change in overlay:
     - Write to committed data (stable memory)
  3. Delete overlay
  4. Transaction ID becomes invalid

rollback(tx_id):
  1. Verify caller owns transaction
  2. Delete overlay (discard all changes)
  3. Transaction ID becomes invalid

Extension Points

ic-dbms provides several extension points for customization:

Custom Sanitizers

Implement the Sanitize trait:

#![allow(unused)]
fn main() {
pub trait Sanitize {
    fn sanitize(&self, value: Value) -> DbmsResult<Value>;
}
}

Custom Validators

Implement the Validate trait:

#![allow(unused)]
fn main() {
pub trait Validate {
    fn validate(&self, value: &Value) -> DbmsResult<()>;
}
}

Custom Data Types

Define custom data types with the CustomDataType derive macro:

#![allow(unused)]
fn main() {
#[derive(Encode, CustomDataType, Clone, Debug, PartialEq, Eq)]
#[type_tag = "status"]
pub enum Status {
    Active,
    Inactive,
}
}

Memory Provider

Implement MemoryProvider for custom memory backends:

#![allow(unused)]
fn main() {
pub trait MemoryProvider {
    const PAGE_SIZE: u64;
    fn size(&self) -> u64;
    fn pages(&self) -> u64;
    fn grow(&mut self, new_pages: u64) -> MemoryResult<u64>;
    fn read(&mut self, offset: u64, buf: &mut [u8]) -> MemoryResult<()>;
    fn write(&mut self, offset: u64, buf: &[u8]) -> MemoryResult<()>;
}
}

Built-in providers:

• IcMemoryProvider - Uses IC stable memory (IC production)
• WasiMemoryProvider - Uses a single flat file (WASI production)
• HeapMemoryProvider - Uses heap memory (testing)

Memory Management

• Memory Management
  • Overview
  • How Internet Computer Memory Works
  • Memory Model
  • Memory Provider
  • Memory Manager and MemoryAccess
  • Encode Trait
  • Schema Registry
  • ACL Storage
  • Table Registry
    • Page Ledger
    • Free Segments Ledger
    • Autoincrement Ledger
  • Record Storage
    • Record Encoding
    • Record Alignment
    • Table Reader
  • Index Registry
    • Index Ledger
    • B-Tree Structure
    • Internal Node Layout
    • Leaf Node Layout
    • Index Maintenance
    • Index Tree Walker

Overview

This document provides the technical details of memory management in wasm-dbms, also known as Layer 0 (the Memory Layer). Understanding this layer is useful for:

• Performance optimization
• Debugging memory issues
• Contributing to wasm-dbms
• Understanding storage costs

How Internet Computer Memory Works

On the Internet Computer, canisters have access to stable memory that persists across upgrades. Key characteristics:

• Page-based: Memory is divided into 64 KiB (65,536 bytes) pages
• Growable: Canisters start small and can allocate additional pages
• Persistent: Survives canister upgrades
• Limited: Subject to subnet memory limits

wasm-dbms uses stable memory directly (not the heap) to ensure data persistence.

Memory Model

┌──────────────────────────────────────────────────┐
│ Page 0: Schema Registry (65 KiB)                 │
│   - Table fingerprints → Page ledger pages       │
├──────────────────────────────────────────────────┤
│ Page 1: ACL Table (65 KiB)                       │
│   - List of allowed principals                   │
├──────────────────────────────────────────────────┤
│ Page 2: Table "users" Page Ledger                │
├──────────────────────────────────────────────────┤
│ Page 3: Table "users" Free Segments Ledger       │
├──────────────────────────────────────────────────┤
│ Page 4: Table "users" Index Ledger               │
├──────────────────────────────────────────────────┤
│ Page 5: Table "users" Autoincrement Ledger (*)   │
├──────────────────────────────────────────────────┤
│ Page 6: Table "posts" Page Ledger                │
├──────────────────────────────────────────────────┤
│ Page 7: Table "posts" Free Segments Ledger       │
├──────────────────────────────────────────────────┤
│ Page 8: Table "posts" Index Ledger               │
├──────────────────────────────────────────────────┤
│ Page 9: Table "users" Records - Page 1           │
├──────────────────────────────────────────────────┤
│ Page 10: Table "users" Records - Page 2          │
├──────────────────────────────────────────────────┤
│ Page 11: B-Tree Node (index on users.id)         │
├──────────────────────────────────────────────────┤
│ Page 12: B-Tree Node (index on users.email)      │
├──────────────────────────────────────────────────┤
│ Page 13: Table "posts" Records - Page 1          │
├──────────────────────────────────────────────────┤
│ ...                                              │
└──────────────────────────────────────────────────┘
(*) Only allocated for tables with #[autoincrement] columns

Layout characteristics:

• Reserved pages (0-1) are allocated at initialization
• Each table gets a Page Ledger, Free Segments Ledger, and Index Ledger
• Tables with #[autoincrement] columns also get an Autoincrement Ledger page
• Record pages and B-tree node pages are allocated on demand
• Pages can be interleaved between tables

Memory Provider

The MemoryProvider trait abstracts memory access:

#![allow(unused)]
fn main() {
pub trait MemoryProvider {
    /// Size of a memory page in bytes (64 KiB for IC)
    const PAGE_SIZE: u64;

    /// Current memory size in bytes
    fn size(&self) -> u64;

    /// Number of allocated pages
    fn pages(&self) -> u64;

    /// Grow memory by new_pages
    /// Returns previous size on success
    fn grow(&mut self, new_pages: u64) -> MemoryResult<u64>;

    /// Read bytes from memory at offset
    fn read(&mut self, offset: u64, buf: &mut [u8]) -> MemoryResult<()>;

    /// Write bytes to memory at offset
    fn write(&mut self, offset: u64, buf: &[u8]) -> MemoryResult<()>;
}
}

Implementations:

#![allow(unused)]
fn main() {
// Production: Uses IC stable memory APIs
pub struct IcMemoryProvider;

#[cfg(target_family = "wasm")]
impl MemoryProvider for IcMemoryProvider {
    const PAGE_SIZE: u64 = ic_cdk::stable::WASM_PAGE_SIZE_IN_BYTES;

    fn grow(&mut self, new_pages: u64) -> MemoryResult<u64> {
        ic_cdk::stable::stable_grow(new_pages)
            .map_err(MemoryError::ProviderError)
    }

    fn read(&mut self, offset: u64, buf: &mut [u8]) -> MemoryResult<()> {
        ic_cdk::stable::stable_read(offset, buf);
        Ok(())
    }

    fn write(&mut self, offset: u64, buf: &[u8]) -> MemoryResult<()> {
        ic_cdk::stable::stable_write(offset, buf);
        Ok(())
    }
}

// Testing: Uses heap memory
pub struct HeapMemoryProvider {
    memory: Vec<u8>,
}
}

Memory Manager and MemoryAccess

The MemoryManager builds on MemoryProvider to handle page allocation. Its page-level read/write operations are exposed through the MemoryAccess trait, which allows the DBMS layer to substitute a journaled writer for atomic transactions (see Atomicity).

#![allow(unused)]
fn main() {
/// Abstracts page-level read/write operations.
///
/// `MemoryManager` implements this trait directly. The DBMS layer provides
/// `JournaledWriter`, which wraps a `MemoryManager` and records original
/// bytes before each write for rollback support.
pub trait MemoryAccess {
    fn page_size(&self) -> u64;
    fn allocate_page(&mut self) -> MemoryResult<Page>;
    fn read_at<D: Encode>(&mut self, page: Page, offset: PageOffset) -> MemoryResult<D>;
    fn write_at<E: Encode>(&mut self, page: Page, offset: PageOffset, data: &E) -> MemoryResult<()>;
    fn zero<E: Encode>(&mut self, page: Page, offset: PageOffset, data: &E) -> MemoryResult<()>;
    fn read_at_raw(&mut self, page: Page, offset: PageOffset, buf: &mut [u8]) -> MemoryResult<usize>;
}
}

#![allow(unused)]
fn main() {
pub struct MemoryManager<P: MemoryProvider> {
    provider: P,
}

// Global instance (thread-local for IC)
// All state is consolidated in a single DbmsContext:
thread_local! {
    pub static DBMS_CONTEXT: DbmsContext<IcMemoryProvider> =
        DbmsContext::new(IcMemoryProvider::default());
}

impl<P: MemoryProvider> MemoryManager<P> {
    /// Initialize and allocate reserved pages
    fn init(provider: P) -> Self;

    /// ACL page number (always 1)
    pub const fn acl_page(&self) -> Page;

    /// Schema registry page (always 0)
    pub const fn schema_page(&self) -> Page;
}

// MemoryAccess is implemented for MemoryManager<P>,
// delegating directly to the underlying MemoryProvider.
impl<P: MemoryProvider> MemoryAccess for MemoryManager<P> { /* ... */ }
}

All table-registry and ledger functions are generic over impl MemoryAccess rather than taking &[mut] MemoryManager directly. This makes it possible to intercept writes at the DBMS layer without modifying any memory-crate code.

Encode Trait

All data stored in memory implements the Encode trait:

#![allow(unused)]
fn main() {
pub trait Encode {
    /// Size characteristic: Fixed or Dynamic
    const SIZE: DataSize;

    /// Memory alignment in bytes
    /// - For Fixed: must equal size
    /// - For Dynamic: minimum 8, default 32
    const ALIGNMENT: PageOffset;

    /// Encode to bytes
    fn encode(&'_ self) -> Cow<'_, [u8]>;

    /// Decode from bytes
    fn decode(data: Cow<[u8]>) -> MemoryResult<Self>
    where
        Self: Sized;

    /// Size of encoded data
    fn size(&self) -> MSize;
}

pub enum DataSize {
    /// Fixed size in bytes (e.g., integers)
    Fixed(MSize),
    /// Variable size (e.g., strings, blobs)
    Dynamic,
}
}

Examples:

Schema Registry

The Schema Registry maps tables to their storage pages:

#![allow(unused)]
fn main() {
/// Information about a table's storage pages
pub struct TableRegistryPage {
    pub pages_list_page: Page,                      // Page Ledger location
    pub free_segments_page: Page,                   // Free Segments Ledger location
    pub index_registry_page: Page,                  // Index Ledger location
    pub autoincrement_registry_page: Option<Page>,  // Autoincrement Ledger (if needed)
}

/// Maps table fingerprints to storage locations
pub struct SchemaRegistry {
    tables: HashMap<TableFingerprint, TableRegistryPage>,
}
}

The autoincrement_registry_page is only allocated when a table has at least one column with the #[autoincrement] attribute. For tables without autoincrement columns, this field is None, avoiding unnecessary page allocation.

Table Fingerprint:

• Unique identifier derived from table schema
• Used to detect schema changes on upgrade
• Enables multiple tables in one canister

ACL Storage

The Access Control List is stored in Page 1. Access control is abstracted behind the AccessControl trait, which allows different runtimes to use different identity types (e.g., Principal on IC, Vec<u8> for generic use).

#![allow(unused)]
fn main() {
pub trait AccessControl: Default {
    type Id;

    fn load<M>(mm: &MemoryManager<M>) -> MemoryResult<Self>
    where
        M: MemoryProvider,
        Self: Sized;

    fn is_allowed(&self, identity: &Self::Id) -> bool;
    fn allowed_identities(&self) -> Vec<Self::Id>;
    fn add_identity<M>(&mut self, identity: Self::Id, mm: &mut MemoryManager<M>) -> MemoryResult<()>
    where
        M: MemoryProvider;
    fn remove_identity<M>(&mut self, identity: &Self::Id, mm: &mut MemoryManager<M>) -> MemoryResult<()>
    where
        M: MemoryProvider;
}
}

The default implementation AccessControlList uses Vec<u8> as its identity type. NoAccessControl is a no-op implementation (with type Id = ()) for runtimes that don’t require ACL. The IC layer provides IcAccessControlList which wraps AccessControlList and uses Principal as its identity type.

Table Registry

Each table has a TableRegistry managing its records, plus an optional AutoincrementLedger for tables with autoincrement columns:

#![allow(unused)]
fn main() {
pub struct TableRegistry {
    free_segments_ledger: FreeSegmentsLedger,
    page_ledger: PageLedger,
    index_ledger: IndexLedger,
}
}

Page Ledger

Tracks which pages contain records for this table:

#![allow(unused)]
fn main() {
pub struct PageLedger {
    ledger_page: Page,      // Where this ledger is stored
    pages: PageTable,       // List of data pages with free space info
}

impl PageLedger {
    /// Load from memory
    pub fn load(page: Page) -> MemoryResult<Self>;

    /// Get page for writing a record
    /// Returns existing page with space or allocates new
    pub fn get_page_for_record<R: Encode>(&mut self, record: &R) -> MemoryResult<Page>;

    /// Commit allocation (update free space tracking)
    pub fn commit<R: Encode>(&mut self, page: Page, record: &R) -> MemoryResult<()>;
}
}

Free Segments Ledger

Tracks free space from deleted/moved records:

#![allow(unused)]
fn main() {
pub struct FreeSegmentsLedger {
    free_segments_page: Page,
    tables: PagesTable,  // Pages containing FreeSegmentsTables
}

pub struct FreeSegment {
    pub page: Page,
    pub offset: PageOffset,
    pub size: MSize,
}

impl FreeSegmentsLedger {
    /// Insert a free segment (when record is deleted)
    pub fn insert_free_segment<E: Encode>(
        &mut self,
        page: Page,
        offset: PageOffset,
        record: &E,
    ) -> MemoryResult<()>;

    /// Find reusable space for a record
    pub fn find_reusable_segment<E: Encode>(
        &self,
        record: &E,
    ) -> MemoryResult<Option<FreeSegmentTicket>>;

    /// Commit reused space
    pub fn commit_reused_space<E: Encode>(
        &mut self,
        record: &E,
        segment: FreeSegmentTicket,
    ) -> MemoryResult<()>;
}
}

Space reuse logic:

1. When a record is deleted, its space is added to free segments
2. When inserting, check for suitable free segment first
3. If found, reuse the space; remaining space becomes new free segment
4. Adjacent free segments are merged to reduce fragmentation

Autoincrement Ledger

Tables with #[autoincrement] columns have a dedicated page storing the current counter value for each autoincrement column. The AutoincrementLedger manages these counters:

#![allow(unused)]
fn main() {
pub struct AutoincrementLedger {
    page: Page,
    registry: AutoincrementRegistry,  // column name → current Value
}
}

Serialization format:

Offset   Size    Field
0        1       Number of entries (u8)
1+       var     For each entry:
                 - 1 byte: column name length (u8)
                 - N bytes: UTF-8 column name
                 - var bytes: encoded Value (type-tagged)

Behavior:

• Initialized with zero values matching each column’s integer type when a table is registered
• next() increments the counter by one and persists the updated value to memory
• Uses checked_add — returns MemoryError::AutoincrementOverflow when a column reaches its type’s maximum value, preventing duplicate key generation
• Each column’s counter is independent; advancing one does not affect others
• State survives across load()/save() cycles (persisted to the dedicated page)

Supported types:

Record Storage

Record Encoding

Records are wrapped in RawRecord with a length header:

┌─────────────────────────────────────────┐
│  2 bytes: Data length (little-endian)   │
├─────────────────────────────────────────┤
│  N bytes: Encoded data                  │
├─────────────────────────────────────────┤
│  Padding to alignment boundary          │
└─────────────────────────────────────────┘

Dynamic size example (alignment=32, data=24 bytes):

Bytes 0-1:   Data length (24)
Bytes 2-25:  Data (24 bytes)
Bytes 26-31: Padding (6 bytes)
Total: 32 bytes (aligned)

Fixed size example (size=14 bytes):

Bytes 0-1:   Data length (14)
Bytes 2-15:  Data (14 bytes)
Total: 16 bytes (no padding for fixed)

Record Alignment

Alignment ensures efficient memory access:

#![allow(unused)]
fn main() {
impl<E: Encode> Alignment for E {
    fn alignment() -> usize {
        match E::SIZE {
            DataSize::Fixed(size) => size as usize,
            DataSize::Dynamic => E::ALIGNMENT as usize,
        }
    }
}

fn align_up<E: Encode>(size: usize) -> usize {
    let align = E::alignment();
    (size + align - 1) / align * align
}
}

Configuring alignment:

#![allow(unused)]
fn main() {
#[derive(Table, ...)]
#[table = "large_records"]
#[alignment = 64]  // Custom alignment for this table
pub struct LargeRecord {
    // ...
}
}

Table Reader

Reading records from a table:

#![allow(unused)]
fn main() {
impl<E: Encode> TableRegistry<E> {
    pub fn read_all(&self) -> MemoryResult<Vec<E>> {
        let mut records = Vec::new();

        for page in self.page_ledger.pages() {
            let mut offset = 0;

            while offset < PAGE_SIZE {
                // Read length header
                let len = read_u16_le(page, offset);

                if len == 0 {
                    // Skip empty slot
                    offset += E::alignment();
                    continue;
                }

                // Read and decode record
                let data = read_bytes(page, offset + 2, len);
                let record = E::decode(data)?;
                records.push(record);

                // Move to next aligned position
                offset += align_up::<E>(len + 2);
            }
        }

        Ok(records)
    }
}
}

Read process:

1. Read 2 bytes at offset for data length
2. If length is 0, skip to next aligned position
3. Read length bytes of data
4. Decode data into record
5. Move to next aligned position
6. Repeat until end of page

Index Registry

Each table has an IndexLedger that maps index definitions (column sets) to B-tree root pages. Indexes are always B+ trees where each node occupies exactly one memory page (64 KiB).

Every table automatically gets an index on its primary key. Additional indexes can be declared with the #[index] attribute (see Schema Reference).

Index Ledger

The IndexLedger is stored in a single page per table and maps column sets to B-tree root pages:

#![allow(unused)]
fn main() {
pub struct IndexLedger {
    ledger_page: Page,
    tables: HashMap<Vec<String>, Page>,  // column names → root page
}
}

Serialization format:

Offset   Size    Field
0-7      8       Number of indexes (u64)
8+       var     For each index:
                 - 8 bytes: column count (u64)
                 - For each column name:
                   - 1 byte: name length (u8)
                   - N bytes: UTF-8 column name
                 - 4 bytes: root page (u32)

When a table is registered via SchemaRegistry::register_table(), the index ledger is initialized by allocating one root page per index definition. The ledger supports insert, delete, update, exact-match search, and range scan operations — all delegated to the underlying B-tree for the appropriate column set.

B-Tree Structure

Indexes use a B+ tree where values (record pointers) are stored only in leaf nodes. Internal nodes contain separator keys that guide traversal. Each node is a single page.

#![allow(unused)]
fn main() {
struct RecordAddress {
    page: Page,         // 4 bytes, u32
    offset: PageOffset, // 2 bytes, u16
}
}

RecordAddress is the pointer stored in leaf entries, pointing to the exact location of the record in the table’s data pages. It is 6 bytes when serialized.

Key characteristics:

• Variable-size keys: Entries are packed as many as fit in a 64 KiB page
• Non-unique: The same key can map to multiple RecordAddress values
• Linked leaves: Leaf nodes form a doubly-linked list for range scans
• Node type tag: Byte 0 distinguishes internal (0x00) from leaf (0x01) nodes

Internal Node Layout

┌──────────────────────────────────────────────────────┐
│ Byte 0:     Node type (0x00 = INTERNAL)              │
│ Bytes 1-4:  Parent page (u32, u32::MAX if root)      │
│ Bytes 5-6:  Entry count (u16)                        │
│ Bytes 7-10: Rightmost child page (u32)               │
├──────────────────────────────────────────────────────┤
│ Entry 0:                                             │
│   Bytes 0-1: Key size (u16)                          │
│   Bytes 2+:  Key data (variable)                     │
│   Next 4:    Child page (u32)                        │
├──────────────────────────────────────────────────────┤
│ Entry 1: ...                                         │
├──────────────────────────────────────────────────────┤
│ ...                                                  │
└──────────────────────────────────────────────────────┘

Header size: 11 bytes. Entries are sorted by key. A search for key K routes to the child page of the first entry whose key is >= K, or to rightmost_child if K is greater than all entries.

Leaf Node Layout

┌──────────────────────────────────────────────────────┐
│ Byte 0:      Node type (0x01 = LEAF)                 │
│ Bytes 1-4:   Parent page (u32, u32::MAX if root)     │
│ Bytes 5-6:   Entry count (u16)                       │
│ Bytes 7-10:  Previous leaf page (u32, u32::MAX=none) │
│ Bytes 11-14: Next leaf page (u32, u32::MAX=none)     │
├──────────────────────────────────────────────────────┤
│ Entry 0:                                             │
│   Bytes 0-1: Key size (u16)                          │
│   Bytes 2+:  Key data (variable)                     │
│   Next 6:    RecordAddress (4-byte page + 2-byte     │
│              offset)                                 │
├──────────────────────────────────────────────────────┤
│ Entry 1: ...                                         │
├──────────────────────────────────────────────────────┤
│ ...                                                  │
└──────────────────────────────────────────────────────┘

Header size: 15 bytes. Entries are sorted by (key, record address). The prev_leaf / next_leaf pointers form a doubly-linked list across all leaves, enabling efficient forward and backward range scans.

Index Maintenance

Indexes are updated eagerly on every write operation:

• INSERT: After writing the record and obtaining its RecordAddress, the key is inserted into every index defined on the table.
• DELETE: After removing the record, the key-pointer pair is removed from every index.
• UPDATE: If indexed columns changed, those indexes are updated (delete old key + insert new key). If the record moved (size change), all indexes are updated with the new RecordAddress.

When a leaf node overflows during insertion, it splits at its midpoint. The first key of the new right sibling is promoted to the parent internal node. If the parent also overflows, the split propagates upward. When the root splits, a new root is created and the tree height increases by one.

When a leaf becomes empty after deletion (and is not the root), it is unlinked from the leaf chain and its parent is updated.

Index Tree Walker

Range scans use an IndexTreeWalker that iterates through leaf entries across linked leaf pages:

#![allow(unused)]
fn main() {
pub struct IndexTreeWalker<K: Encode + Ord> {
    entries: Vec<LeafEntry<K>>,   // Current leaf's entries
    cursor: usize,                // Position within current leaf
    next_leaf: Option<Page>,      // Next leaf page for continuation
    end_key: Option<K>,           // Optional upper bound (inclusive)
}
}

The walker starts at the first leaf entry >= start_key and advances through the linked-leaf chain until it reaches an entry > end_key (or exhausts all leaves). This provides efficient iteration for range queries without revisiting internal nodes.

Join Engine

• Overview
• Architecture
• Processing Pipeline
• Nested-Loop Join Algorithm
• NULL Padding
• Column Resolution
• Output Format
• Limitations

Overview

The join engine executes cross-table join queries, combining rows from two or more tables based on column equality conditions. It supports four join types — INNER, LEFT, RIGHT, and FULL — and integrates with the existing query pipeline for filtering, ordering, pagination, and column selection.

The implementation lives in crates/ic-dbms-canister/src/dbms/join.rs.

Architecture

The engine is implemented as a generic struct:

#![allow(unused)]
fn main() {
pub struct JoinEngine<'a, Schema: ?Sized>
where
    Schema: DatabaseSchema,
{
    schema: &'a Schema,
}
}

Key design decisions:

• Schema: ?Sized — The ?Sized bound allows the engine to work with Box<dyn DatabaseSchema>, which is how the API layer passes the schema at runtime.
• Borrows DatabaseSchema — The engine borrows the schema to read rows from tables via schema.select(dbms, table, query).
• Stateless — The engine holds no mutable state; it takes a Query and returns results in a single call.

The DatabaseSchema trait provides the select method that the engine uses to read all rows from each table involved in the join.

Processing Pipeline

The join() method processes a query through these steps:

                    ┌──────────────────────────┐
                    │ 1. Read FROM table rows  │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼─────────────┐
                    │ 2. For each JOIN clause:  │◄──── left-to-right
                    │    Read right table rows  │
                    │    Nested-loop join       │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼─────────────┐
                    │ 3. Apply filter           │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼─────────────┐
                    │ 4. Apply ordering         │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼─────────────┐
                    │ 5. Apply offset           │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼─────────────┐
                    │ 6. Apply limit            │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼─────────────┐
                    │ 7. Flatten to output      │
                    └──────────────────────────┘

1. Read FROM table: All rows from the primary table are loaded using an unfiltered Query::builder().all().build().
2. Process JOINs: Each Join clause is processed left-to-right. For each clause, the right table is read in full, column references are resolved, and the nested-loop join is executed against the accumulated result.
3. Filter: The query’s filter is applied to the combined rows using filter.matches_joined_row(), which supports qualified table.column references.
4. Order: Order-by clauses are applied in reverse (stable sort), so the primary sort key ends up correctly ordered.
5. Offset: Rows are skipped according to the offset value.
6. Limit: The result is truncated to the limit.
7. Flatten: Each joined row is converted from the internal JoinedRow representation to the output Vec<(JoinColumnDef, Value)> format, applying column selection.

Nested-Loop Join Algorithm

All four join types are handled by a single nested_loop_join method using two boolean flags:

The algorithm:

1. For each left row, iterate over all right rows.
2. If the left column value equals the right column value (and is not None), emit a combined row and mark the right row as matched.
3. After scanning all right rows for a given left row: if keep_unmatched_left is true and no match was found, emit the left row with NULL-padded right columns.
4. After all left rows are processed: if keep_unmatched_right is true, emit each unmatched right row with NULL-padded left columns.

This unified approach avoids code duplication across join types while keeping the logic straightforward.

NULL Padding

When a row has no match on the opposite side (in LEFT, RIGHT, or FULL joins), the missing columns are filled with Value::Null. The engine determines which columns to pad by inspecting a sample row from the opposite table:

#![allow(unused)]
fn main() {
fn null_pad_columns(&self, sample_row: &[(ColumnDef, Value)]) -> Vec<(ColumnDef, Value)> {
    sample_row
        .iter()
        .map(|(col, _)| (*col, Value::Null))
        .collect()
}
}

This preserves the correct column definitions (name, type, nullability) while setting every value to NULL. If the opposite table is empty (no sample row available), the padded group has zero columns.

Column Resolution

Column references in join ON conditions, filters, and ordering can be either qualified or unqualified:

• Qualified: "users.id" — explicitly specifies the table.
• Unqualified: "id" — defaults to the FROM table (for ON left-column) or the joined table (for ON right-column).

Resolution is handled by resolve_column_ref:

#![allow(unused)]
fn main() {
fn resolve_column_ref(&self, field: &str, default_table: &str) -> (String, &str) {
    if let Some((table, column)) = field.split_once('.') {
        (table.to_string(), column)
    } else {
        (default_table.to_string(), field)
    }
}
}

For filters and ordering on joined results, the same qualified/unqualified pattern applies. Unqualified names are searched across all table groups in the row, returning the first match.

Output Format

Join results use JoinColumnDef instead of ColumnDef:

#![allow(unused)]
fn main() {
pub struct JoinColumnDef {
    pub table: Option<String>,  // Source table name
    pub name: String,
    pub data_type: DataTypeKind,
    pub nullable: bool,
    pub primary_key: bool,
}
}

The table field is Some(table_name) for join results, allowing consumers to distinguish columns that share the same name across different tables.

At the API layer, the generated select endpoint checks query.has_joins():

• With joins: Routes to select_join, which uses JoinEngine.
• Without joins: Routes to select_raw, the standard single-table path.

Both paths return Vec<Vec<(JoinColumnDef, Value)>>, but for non-join queries the table field is None.

Limitations

• O(n*m) nested-loop join: Each join performs a full nested-loop comparison. For two tables of size n and m, this is O(n*m) per join clause.
• Full table scans for join matching: The join ON condition itself does not use indexes — both sides are compared via linear scan. However, if the query has a filter, the individual table reads that feed the join may use indexes (via the standard select path).
• All rows loaded into memory: Every table involved in the join is fully materialized in memory before processing. This can be a concern for very large tables on the IC.
• Equality joins only: The ON condition only supports column equality (left_col = right_col). Range conditions, expressions, and multi-column ON clauses are not supported.

Atomicity

• Atomicity
  • Overview
  • The Problem
  • Write-Ahead Journal
    • Architecture
    • Journal Flow
    • What is Journaled
  • Transaction Commit Atomicity
  • Edge Cases
    • Page Allocation
    • Nested Atomic Calls
    • Rollback Failure

Overview

A DBMS must guarantee atomicity: either all writes in an operation succeed, or none of them persist. Without atomicity, a crash or error mid-operation can leave the database in an inconsistent state (e.g., a record written but the page ledger not updated, or half the rows in a transaction committed while the rest are lost).

The Problem

The original atomic() implementation relied on panic semantics:

#![allow(unused)]
fn main() {
fn atomic<F, R>(&self, f: F) -> R
where
    F: FnOnce(&WasmDbmsDatabase<M, A>) -> DbmsResult<R>,
{
    match f(self) {
        Ok(res) => res,
        Err(err) => panic!("{err}"),
    }
}
}

On the Internet Computer, a panic (trap) automatically reverts all stable-memory writes made during that call. This gave IC canisters free atomicity. However, on non-IC WASM runtimes (e.g., Wasmtime, Wasmer, browser WASM), a panic does not revert memory. The host simply sees the guest abort, and any writes already flushed to linear memory remain. This made wasm-dbms effectively IC-only for write operations.

Write-Ahead Journal

The fix is a write-ahead journal. Before overwriting any bytes, the journal saves the original content at that offset. On error, the journal replays saved entries in reverse order, restoring every modified byte.

Architecture

The journal lives in the wasm-dbms crate’s transaction module, not in the memory layer. This separation keeps the memory crate (wasm-dbms-memory) focused on page-level I/O while the DBMS layer owns the transaction concern.

The key types are:

• MemoryAccess trait (in wasm-dbms-memory): Abstracts page-level read/write operations. MemoryManager implements this trait with direct writes.
• Journal (in wasm-dbms): A heap-only collection of JournalEntry records. Each entry stores the page, offset, and original bytes before a write.
• JournaledWriter (in wasm-dbms): Wraps a &mut MemoryManager and a &mut Journal, implementing MemoryAccess. Every write_at or zero call reads the original bytes first, records them in the journal, then delegates to the underlying MemoryManager.

All memory-crate functions that perform writes (in TableRegistry, PageLedger, FreeSegmentsLedger, etc.) are generic over impl MemoryAccess. When called with a plain MemoryManager, writes go directly to memory. When called with a JournaledWriter, writes are automatically recorded for rollback.

Journal Flow

┌─────────────────┐
│  Journal::new() │   Creates empty journal
└────────┬────────┘
         │
         ▼
┌─────────────────────────┐
│  JournaledWriter wraps  │
│  MemoryManager + Journal│
└────────┬────────────────┘
         │
         ▼
┌──────────────┐
│   write_at   │──► Reads original bytes, records in journal, then writes new data
│     zero     │──► Reads original bytes, records in journal, then writes zeros
└──────┬───────┘
       │
       ├── success ──► journal.commit()   ──► Drops entries (no-op)
       │
       └── error   ──► journal.rollback() ──► Replays entries in reverse via MemoryManager

Each journal entry is:

#![allow(unused)]
fn main() {
struct JournalEntry {
    page: Page,
    offset: PageOffset,
    original_bytes: Vec<u8>,
}
}

What is Journaled

Transaction Commit Atomicity

When a transaction is committed, all buffered operations (inserts, updates, deletes) are flushed to memory. Previously, each operation was wrapped in its own atomic() call. If operation 3 of 5 failed, operations 1 and 2 were already persisted and could not be undone.

Now, commit() uses a single journal spanning all operations:

#![allow(unused)]
fn main() {
fn commit(&mut self) -> DbmsResult<()> {
    // ... take transaction ...

    *self.ctx.journal.borrow_mut() = Some(Journal::new());

    for op in transaction.operations {
        let result = match op { /* execute insert/update/delete */ };

        if let Err(err) = result {
            if let Some(journal) = self.ctx.journal.borrow_mut().take() {
                journal
                    .rollback(&mut self.ctx.mm.borrow_mut())
                    .expect("critical: failed to rollback journal");
            }
            return Err(err);
        }
    }

    if let Some(journal) = self.ctx.journal.borrow_mut().take() {
        journal.commit();
    }
    Ok(())
}
}

This ensures that either all transaction operations are applied, or none of them persist, regardless of the WASM runtime.

Edge Cases

Page Allocation

allocate_page writes directly via the memory provider, bypassing the journal. This is intentional: a newly allocated page has no meaningful prior content to restore, and after a rollback, nothing references it (the page ledger update that would have pointed to it was itself journaled and rolled back). The page remains allocated but unused — a minor space leak that is acceptable since it will be reused by subsequent allocations.

Nested Atomic Calls

During commit(), each transaction operation is dispatched through the Database trait methods (insert, update, delete), which internally call atomic(). Since commit() has already placed a Journal in DbmsContext, atomic() detects this via self.ctx.journal.borrow().is_some() and delegates to the outer journal instead of starting its own. This ensures a single journal spans the entire commit.

Rollback Failure

If journal.rollback() itself fails (e.g., the memory provider returns an I/O error during the restore writes), the program panics. A failed rollback means memory is in an indeterminate state — some bytes restored, some not. There is no recovery path, so immediate termination is the only safe response (per M-PANIC-ON-BUG).

WASI Memory Provider

• Overview
• Installation
• Usage
  • Creating a Provider
  • Using with DbmsContext
  • TryFrom Conversions
• File Layout and Portability
• Error Handling
• Concurrency
• Comparison with Other Providers

Overview

The wasi-dbms-memory crate provides WasiMemoryProvider, a persistent file-backed implementation of the MemoryProvider trait. It enables wasm-dbms databases to run on any WASI-compliant runtime (Wasmer, Wasmtime, WasmEdge, etc.) with durable data persistence across process restarts.

The provider stores all database pages in a single flat file on the filesystem. Each page is 64 KiB (65,536 bytes), matching the WASM memory page size.

Installation

Add wasi-dbms-memory to your Cargo.toml:

[dependencies]
wasi-dbms-memory = "0.8"

The crate depends on wasm-dbms-api and wasm-dbms-memory (pulled in transitively).

Usage

Creating a Provider

#![allow(unused)]
fn main() {
use wasi_dbms_memory::WasiMemoryProvider;
use wasm_dbms_memory::MemoryProvider;

// Opens the file if it exists, or creates it empty.
let mut provider = WasiMemoryProvider::new("./data/mydb.bin").unwrap();

// Allocate pages as needed.
provider.grow(1).unwrap(); // 1 page = 64 KiB

// Read and write at arbitrary offsets.
provider.write(0, b"hello").unwrap();

let mut buf = vec![0u8; 5];
provider.read(0, &mut buf).unwrap();
assert_eq!(&buf, b"hello");
}

When opening an existing file, the page count is inferred from the file size. The file size must be a multiple of 64 KiB; otherwise WasiMemoryProvider::new returns an error.

The parent directory must already exist before creating the provider.

Using with DbmsContext

Pass the provider directly to DbmsContext:

#![allow(unused)]
fn main() {
use wasi_dbms_memory::WasiMemoryProvider;
use wasm_dbms::DbmsContext;

let provider = WasiMemoryProvider::new("./data/mydb.bin").unwrap();
let ctx = DbmsContext::new(provider, acl);
}

From this point on, all database operations use the file as persistent storage.

TryFrom Conversions

The crate provides TryFrom implementations for convenient construction:

#![allow(unused)]
fn main() {
use std::path::{Path, PathBuf};
use wasi_dbms_memory::WasiMemoryProvider;

// From &Path
let provider = WasiMemoryProvider::try_from(Path::new("./mydb.bin")).unwrap();

// From PathBuf
let path = PathBuf::from("./mydb.bin");
let provider = WasiMemoryProvider::try_from(path).unwrap();
}

File Layout and Portability

The backing file is byte-for-byte equivalent to IC stable memory: a contiguous sequence of 64 KiB pages, zero-filled on allocation. This means database snapshots are portable between different MemoryProvider implementations.

A file created by WasiMemoryProvider can be loaded by any provider that uses the same page layout, and vice versa. This enables workflows such as:

• Exporting a database from an IC canister and loading it locally for debugging
• Developing and testing with WASI, then deploying to the Internet Computer
• Migrating data between different WASM runtimes

Error Handling

All operations return MemoryResult<T> (an alias for Result<T, MemoryError>). The possible errors are:

Concurrency

WasiMemoryProvider assumes single-writer access. WASM is single-threaded by default, so this is generally not a concern. If you run multiple instances pointing at the same file, you are responsible for external synchronization. WASI file-lock support varies across runtimes.

Comparison with Other Providers

All three share the same page layout, so data is portable across implementations.

IC-DBMS: Internet Computer Integration

• Overview
• Architecture
• Crates
• Quick Start
• Guides
• Reference

Overview

IC-DBMS is an adapter layer that brings the wasm-dbms relational database engine to the Internet Computer (IC). While wasm-dbms provides the core database functionality (tables, CRUD operations, transactions, memory management), ic-dbms adds everything needed to run it as an IC canister:

• Candid serialization for all types and API endpoints
• Canister lifecycle management (init, upgrade, inspect)
• ACL-based access control using IC principals
• Procedural macros to generate complete canister APIs from schema definitions
• Client libraries for inter-canister calls, external agent access, and integration testing

If you are using wasm-dbms outside the Internet Computer (e.g., in a standalone WASM runtime), you do not need ic-dbms. See the generic wasm-dbms documentation instead.

Architecture

┌─────────────────────────────────────────────────┐
│              Your Application                    │
│  (Frontend canister, backend canister, CLI, etc.)│
└──────────────────────┬──────────────────────────┘
                       │  Candid calls
                       ▼
┌─────────────────────────────────────────────────┐
│           ic-dbms-client                         │
│  (IcDbmsCanisterClient / IcDbmsAgentClient /     │
│   IcDbmsPocketIcClient)                          │
└──────────────────────┬──────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────┐
│           ic-dbms-canister                       │
│  (Generated canister API, ACL, init/upgrade)     │
│                                                  │
│  ┌───────────────────────────────────────────┐   │
│  │           wasm-dbms (core engine)          │   │
│  │  Tables, CRUD, Transactions, Memory Mgmt  │   │
│  └───────────────────────────────────────────┘   │
└─────────────────────────────────────────────────┘

Crates

IC-DBMS is composed of four crates:

Import convention:

#![allow(unused)]
fn main() {
// In your schema crate
use ic_dbms_api::prelude::*;  // Re-exports wasm_dbms_api types

// In your canister crate
use ic_dbms_canister::prelude::DbmsCanister;

// In your client code
use ic_dbms_client::{IcDbmsCanisterClient, Client as _};
}

Quick Start

1. Define your schema with IC-compatible derives:

#![allow(unused)]
fn main() {
use candid::{CandidType, Deserialize};
use ic_dbms_api::prelude::*;

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
    pub email: Text,
}
}

2. Generate the canister:

#![allow(unused)]
fn main() {
use ic_dbms_canister::prelude::{DatabaseSchema, DbmsCanister};

#[derive(DatabaseSchema, DbmsCanister)]
#[tables(User = "users")]
pub struct MyDbmsCanister;

ic_cdk::export_candid!();
}

3. Build, deploy, and interact:

cargo build --target wasm32-unknown-unknown --release
dfx deploy my_dbms --argument '(variant { Init = record { allowed_principals = vec { principal "your-principal" } } })'

#![allow(unused)]
fn main() {
let client = IcDbmsCanisterClient::new(canister_id);
client.insert::<User>(User::table_name(), user, None).await??;
}

For the full walkthrough, see the Get Started guide.

Guides

• Get Started - Set up and deploy your first IC database canister
• CRUD Operations - Insert, select, update, delete via the IC client
• Access Control - ACL management with IC principals
• Client API - All client types and usage patterns

For core wasm-dbms guides (querying, transactions, relationships, validators, sanitizers, custom data types), see the generic guides.

Reference

• Schema (IC) - DbmsCanister macro, Candid API generation, IC-specific derives
• Data Types (IC) - Principal type, Candid type mappings
• Errors (IC) - IcDbmsError alias, double-Result pattern, client error handling

For the complete reference (all data types, error variants, sanitizers, validators, JSON operations), see the generic reference.

Get Started with IC-DBMS (IC)

Note: This is the IC-specific getting started guide for deploying wasm-dbms as an Internet Computer canister. For the generic wasm-dbms getting started guide (schema definition, core concepts), see the generic get-started guide.

• Prerequisites
• Project Setup
  • Workspace Structure
  • Cargo Configuration
• Define Your Schema
  • Create the Schema Crate
  • Define Tables
• Create the DBMS Canister
  • Canister Dependencies
  • Generate the Canister API
  • Build the Canister
• Deploy the Canister
  • Canister Init Arguments
  • Deploy with dfx
• Quick Example: Complete Workflow
• Integration Testing
• Next Steps

This guide walks you through setting up a complete database canister on the Internet Computer using ic-dbms. The ic-dbms framework is built on top of the wasm-dbms core engine, adding IC-specific functionality such as Candid serialization, canister lifecycle management, ACL-based access control, and inter-canister communication. By the end of this guide, you will have a working canister with CRUD operations, transactions, and access control.

Prerequisites

Before starting, ensure you have:

• Rust 1.91.1 or later
• wasm32-unknown-unknown target: rustup target add wasm32-unknown-unknown
• dfx (Internet Computer SDK)
• ic-wasm: cargo install ic-wasm
• candid-extractor: cargo install candid-extractor

Project Setup

Workspace Structure

We recommend organizing your project as a Cargo workspace with two crates:

my-dbms-project/
├── Cargo.toml          # Workspace manifest
├── schema/             # Schema definitions (reusable types)
│   ├── Cargo.toml
│   └── src/
│       └── lib.rs
└── canister/           # The DBMS canister
    ├── Cargo.toml
    └── src/
        └── lib.rs

Workspace Cargo.toml:

[workspace]
members = ["schema", "canister"]
resolver = "2"

Cargo Configuration

Create .cargo/config.toml to configure the getrandom crate for WebAssembly:

[target.wasm32-unknown-unknown]
rustflags = ['--cfg', 'getrandom_backend="custom"']

This is required because the uuid crate depends on getrandom.

Define Your Schema

Create the Schema Crate

Create schema/Cargo.toml:

[package]
name = "my-schema"
version = "0.1.0"
edition = "2024"

[dependencies]
candid = "0.10"
ic-dbms-api = "0.6"
serde = "1"

Note: ic-dbms-api re-exports types from wasm-dbms-api, so use ic_dbms_api::prelude::* gives you access to the full set of wasm-dbms data types, validators, and sanitizers.

Define Tables

In schema/src/lib.rs, define your database tables using the Table derive macro:

#![allow(unused)]
fn main() {
use candid::{CandidType, Deserialize};
use ic_dbms_api::prelude::*;

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[sanitizer(TrimSanitizer)]
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,
    #[validate(EmailValidator)]
    pub email: Text,
    pub created_at: DateTime,
}

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uint32,
    #[validate(MaxStrlenValidator(200))]
    pub title: Text,
    pub content: Text,
    pub published: Boolean,
    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,
}
}

Required derives: Table, CandidType, Deserialize, Clone. The #[candid] attribute ensures generated types (Record, InsertRequest, UpdateRequest) also derive Candid/Serde traits.

The Table macro generates additional types for each table:

Create the DBMS Canister

Canister Dependencies

Create canister/Cargo.toml:

[package]
name = "my-canister"
version = "0.1.0"
edition = "2024"

[lib]
crate-type = ["cdylib"]

[dependencies]
candid = "0.10"
ic-cdk = "0.19"
ic-dbms-api = "0.6"
ic-dbms-canister = "0.6"
my-schema = { path = "../schema" }
serde = "1"

Generate the Canister API

In canister/src/lib.rs:

#![allow(unused)]
fn main() {
use ic_dbms_canister::prelude::{DatabaseSchema, DbmsCanister};
use my_schema::{User, Post};

#[derive(DatabaseSchema, DbmsCanister)]
#[tables(User = "users", Post = "posts")]
pub struct MyDbmsCanister;

ic_cdk::export_candid!();
}

The DatabaseSchema derive generates the DatabaseSchema<M> trait implementation that provides schema dispatch ( routing operations to the correct table by name). The DbmsCanister derive generates the complete canister API:

service : (IcDbmsCanisterArgs) -> {
  // ACL Management
  acl_add_principal : (principal) -> (Result);
  acl_allowed_principals : () -> (vec principal) query;
  acl_remove_principal : (principal) -> (Result);

  // Transactions
  begin_transaction : () -> (nat);
  commit : (nat) -> (Result);
  rollback : (nat) -> (Result);

  // Users CRUD
  insert_users : (UserInsertRequest, opt nat) -> (Result);
  select_users : (Query, opt nat) -> (Result_1) query;
  update_users : (UserUpdateRequest, opt nat) -> (Result_2);
  delete_users : (DeleteBehavior, opt Filter, opt nat) -> (Result_2);

  // Posts CRUD
  insert_posts : (PostInsertRequest, opt nat) -> (Result);
  select_posts : (Query, opt nat) -> (Result_3) query;
  update_posts : (PostUpdateRequest, opt nat) -> (Result_2);
  delete_posts : (DeleteBehavior, opt Filter, opt nat) -> (Result_2);
}

Build the Canister

Create a build script or use the following commands:

# Build the canister
cargo build --target wasm32-unknown-unknown --release -p my-canister

# Optimize the WASM
ic-wasm target/wasm32-unknown-unknown/release/my_canister.wasm \
    -o my_canister.wasm shrink

# Extract Candid interface
candid-extractor my_canister.wasm > my_canister.did

# Optionally compress
gzip -k my_canister.wasm --force

Deploy the Canister

Canister Init Arguments

The canister requires initialization arguments specifying which principals can access the database:

type IcDbmsCanisterArgs = variant {
  Init : IcDbmsCanisterInitArgs;
  Upgrade;
};

type IcDbmsCanisterInitArgs = record {
  allowed_principals : vec principal;
};

Warning: Only principals in allowed_principals can perform database operations. Make sure to include all necessary principals (your frontend canister, admin principal, etc.).

Deploy with dfx

Create dfx.json:

{
  "canisters": {
    "my_dbms": {
      "type": "custom",
      "candid": "my_canister.did",
      "wasm": "my_canister.wasm",
      "build": []
    }
  }
}

Deploy:

dfx deploy my_dbms --argument '(variant { Init = record { allowed_principals = vec { principal "your-principal-here" } } })'

Quick Example: Complete Workflow

Here’s a complete example showing insert, query, update, and delete operations:

#![allow(unused)]
fn main() {
use ic_dbms_client::{IcDbmsCanisterClient, Client as _};
use my_schema::{User, UserInsertRequest, UserUpdateRequest};
use ic_dbms_api::prelude::*;

async fn example(canister_id: Principal) -> Result<(), Box<dyn std::error::Error>> {
    let client = IcDbmsCanisterClient::new(canister_id);

    // 1. INSERT a new user
    let insert_req = UserInsertRequest {
        id: 1.into(),
        name: "Alice".into(),
        email: "alice@example.com".into(),
        created_at: DateTime::now(),
    };
    client.insert::<User>(User::table_name(), insert_req, None).await??;

    // 2. SELECT users
    let query = Query::builder()
        .filter(Filter::eq("name", Value::Text("Alice".into())))
        .build();
    let users = client.select::<User>(User::table_name(), query, None).await??;
    println!("Found {} user(s)", users.len());

    // 3. UPDATE the user
    let update_req = UserUpdateRequest::builder()
        .set_email("alice.new@example.com".into())
        .filter(Filter::eq("id", Value::Uint32(1.into())))
        .build();
    let updated = client.update::<User>(User::table_name(), update_req, None).await??;
    println!("Updated {} record(s)", updated);

    // 4. DELETE the user
    let deleted = client.delete::<User>(
        User::table_name(),
        DeleteBehavior::Restrict,
        Some(Filter::eq("id", Value::Uint32(1.into()))),
        None
    ).await??;
    println!("Deleted {} record(s)", deleted);

    Ok(())
}
}

Integration Testing

For integration tests using PocketIC, add ic-dbms-client with the pocket-ic feature:

[dev-dependencies]
ic-dbms-client = { version = "0.8", features = ["pocket-ic"] }
pocket-ic = "9"

Example test:

#![allow(unused)]
fn main() {
use ic_dbms_client::prelude::{Client as _, IcDbmsPocketIcClient};
use my_schema::{User, UserInsertRequest};
use pocket_ic::PocketIc;

#[tokio::test]
async fn test_insert_and_select() {
    let pic = PocketIc::new();
    // ... setup canister ...

    let client = IcDbmsPocketIcClient::new(canister_id, admin_principal, &pic);

    let insert_req = UserInsertRequest {
        id: 1.into(),
        name: "Test User".into(),
        email: "test@example.com".into(),
        created_at: DateTime::now(),
    };

    client
        .insert::<User>(User::table_name(), insert_req, None)
        .await
        .expect("call failed")
        .expect("insert failed");

    let query = Query::builder().all().build();
    let users = client
        .select::<User>(User::table_name(), query, None)
        .await
        .expect("call failed")
        .expect("select failed");

    assert_eq!(users.len(), 1);
    assert_eq!(users[0].name.as_str(), "Test User");
}
}

Next Steps

Now that you have a working canister, explore these topics:

• CRUD Operations (IC) - Detailed guide on all database operations via the IC client
• Access Control - Managing the ACL
• Client API - All client types and usage patterns
• Schema Definition (IC) - IC-specific schema reference (DbmsCanister macro, Candid API)
• Data Types (IC) - IC-specific data types (Principal, Candid mappings)
• Errors (IC) - IC-specific error handling (double-Result pattern)

For core wasm-dbms concepts (querying, transactions, relationships, validators, sanitizers), see the generic guides.

CRUD Operations (IC)

Note: This is the IC-specific CRUD operations guide, covering usage via the ic-dbms-client. For core CRUD concepts (filtering, delete behaviors, error types), see the generic CRUD operations guide.

• Overview
• Insert
  • Basic Insert
  • Handling Primary Keys
  • Nullable Fields
  • Insert with Transaction
• Select
  • Select All Records
  • Select with Filter
  • Select Specific Columns
  • Select with Eager Loading
• Update
  • Basic Update
  • Partial Updates
  • Update with Filter
  • Update Return Value
• Delete
  • Delete with Filter
  • Delete Behaviors
  • Delete All Records
• Operations with Transactions
• Error Handling

Overview

ic-dbms provides four fundamental database operations, accessed through the ic-dbms-client crate’s Client trait. All operations use Candid serialization under the hood and support the IC’s inter-canister call model.

All operations:

• Respect access control (caller must be in ACL)
• Support optional transaction IDs
• Validate and sanitize data according to schema rules
• Enforce foreign key constraints
• Return a double Result (see Error Handling)

Insert

Basic Insert

To insert a record, create an InsertRequest and call the insert method:

#![allow(unused)]
fn main() {
use ic_dbms_client::{IcDbmsCanisterClient, Client as _};
use my_schema::{User, UserInsertRequest};
use ic_dbms_api::prelude::*;

let client = IcDbmsCanisterClient::new(canister_id);

let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    email: "alice@example.com".into(),
    created_at: DateTime::now(),
};

// Insert without transaction (None)
client
    .insert::<User>(User::table_name(), user, None)
    .await??;
}

Handling Primary Keys

Every table must have a primary key. Insert will fail if a record with the same primary key already exists:

#![allow(unused)]
fn main() {
// First insert succeeds
client.insert::<User>(User::table_name(), user1, None).await??;

// Second insert with same ID fails with PrimaryKeyConflict
let result = client.insert::<User>(User::table_name(), user2_same_id, None).await?;
assert!(matches!(result, Err(IcDbmsError::Query(QueryError::PrimaryKeyConflict))));
}

Nullable Fields

For fields wrapped in Nullable<T>, you can insert either a value or null:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "profiles"]
pub struct Profile {
    #[primary_key]
    pub id: Uint32,
    pub bio: Nullable<Text>,      // Optional field
    pub website: Nullable<Text>,  // Optional field
}

// Insert with value
let profile = ProfileInsertRequest {
    id: 1.into(),
    bio: Nullable::Value("Hello world".into()),
    website: Nullable::Null,  // No website
};

client.insert::<Profile>(Profile::table_name(), profile, None).await??;
}

Insert with Transaction

To insert within a transaction, pass the transaction ID:

#![allow(unused)]
fn main() {
// Begin transaction
let tx_id = client.begin_transaction().await?;

// Insert within transaction
client.insert::<User>(User::table_name(), user, Some(tx_id)).await??;

// Commit or rollback
client.commit(tx_id).await??;
}

Select

Select All Records

Use Query::builder().all() to select all records:

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::*;

let query = Query::builder().all().build();
let users: Vec<UserRecord> = client
    .select::<User>(User::table_name(), query, None)
    .await??;

for user in users {
    println!("User: {} ({})", user.name, user.email);
}
}

Select with Filter

Add filters to narrow down results:

#![allow(unused)]
fn main() {
// Select users with specific name
let query = Query::builder()
    .filter(Filter::eq("name", Value::Text("Alice".into())))
    .build();

let users = client.select::<User>(User::table_name(), query, None).await??;
}

See the Querying Guide for comprehensive filter documentation.

Select Specific Columns

Select only the columns you need:

#![allow(unused)]
fn main() {
let query = Query::builder()
    .columns(vec!["id".to_string(), "name".to_string()])
    .build();

let users = client.select::<User>(User::table_name(), query, None).await??;
// Only id and name are populated; other fields have default values
}

Select with Eager Loading

Load related records in a single query using with():

#![allow(unused)]
fn main() {
// Load posts with their authors
let query = Query::builder()
    .all()
    .with("users")  // Eager load the related users table
    .build();

let posts = client.select::<Post>(Post::table_name(), query, None).await??;
}

See the Relationships Guide for more on eager loading.

Update

Basic Update

Create an UpdateRequest to modify records:

#![allow(unused)]
fn main() {
use my_schema::UserUpdateRequest;

let update = UserUpdateRequest::builder()
    .set_name("Alice Smith".into())
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

let affected_rows = client
    .update::<User>(User::table_name(), update, None)
    .await??;

println!("Updated {} row(s)", affected_rows);
}

Partial Updates

Only specify the fields you want to change. Unspecified fields remain unchanged:

#![allow(unused)]
fn main() {
// Only update the email, keep everything else
let update = UserUpdateRequest::builder()
    .set_email("new.email@example.com".into())
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

client.update::<User>(User::table_name(), update, None).await??;
}

Update with Filter

The filter determines which records are updated:

#![allow(unused)]
fn main() {
// Update all users with a specific domain
let update = UserUpdateRequest::builder()
    .set_verified(true.into())
    .filter(Filter::like("email", "%@company.com"))
    .build();

let affected = client.update::<User>(User::table_name(), update, None).await??;
println!("Verified {} company users", affected);
}

Update Return Value

Update returns the number of affected rows:

#![allow(unused)]
fn main() {
let affected = client.update::<User>(User::table_name(), update, None).await??;

if affected == 0 {
    println!("No records matched the filter");
} else {
    println!("Updated {} record(s)", affected);
}
}

Delete

Delete with Filter

Delete records matching a filter:

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::DeleteBehavior;

let filter = Filter::eq("id", Value::Uint32(1.into()));

let deleted = client
    .delete::<User>(
        User::table_name(),
        DeleteBehavior::Restrict,
        Some(filter),
        None  // No transaction
    )
    .await??;

println!("Deleted {} record(s)", deleted);
}

Delete Behaviors

When deleting records that are referenced by foreign keys, you must specify a behavior:

Restrict Example:

#![allow(unused)]
fn main() {
// Will fail if any posts reference this user
let result = client.delete::<User>(
    User::table_name(),
    DeleteBehavior::Restrict,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
    None
).await?;

match result {
    Ok(count) => println!("Deleted {} user(s)", count),
    Err(IcDbmsError::Query(QueryError::ForeignKeyConstraintViolation)) => {
        println!("Cannot delete: user has posts");
    }
    Err(e) => return Err(e.into()),
}
}

Cascade Example:

#![allow(unused)]
fn main() {
// Deletes the user AND all their posts
client.delete::<User>(
    User::table_name(),
    DeleteBehavior::Cascade,
    Some(Filter::eq("id", Value::Uint32(1.into()))),
    None
).await??;
}

Delete All Records

Pass None as the filter to delete all records (use with caution):

#![allow(unused)]
fn main() {
// Delete ALL users (respecting foreign key behavior)
let deleted = client
    .delete::<User>(
        User::table_name(),
        DeleteBehavior::Cascade,
        None,  // No filter = all records
        None
    )
    .await??;

println!("Deleted all {} users and their related records", deleted);
}

Operations with Transactions

All CRUD operations accept an optional transaction ID. When provided, the operation is performed within that transaction and won’t be visible to other callers until committed:

#![allow(unused)]
fn main() {
// Begin transaction
let tx_id = client.begin_transaction().await?;

// Perform operations within transaction
client.insert::<User>(User::table_name(), user1, Some(tx_id)).await??;
client.insert::<User>(User::table_name(), user2, Some(tx_id)).await??;

// Update within same transaction
let update = UserUpdateRequest::builder()
    .set_verified(true.into())
    .filter(Filter::all())
    .build();
client.update::<User>(User::table_name(), update, Some(tx_id)).await??;

// Commit all changes atomically
client.commit(tx_id).await??;
}

See the Transactions Guide for comprehensive transaction documentation.

Error Handling

CRUD operations via the IC client return a double Result: Result<Result<T, IcDbmsError>, CallError>.

• Outer Result: Network/canister call errors (canister unreachable, cycles exhausted)
• Inner Result: Database logic errors (validation, constraint violations, etc.)

Use ?? to propagate both:

#![allow(unused)]
fn main() {
client.insert::<User>(User::table_name(), user, None).await??;
}

Or handle each layer explicitly:

#![allow(unused)]
fn main() {
match client.insert::<User>(User::table_name(), user, None).await {
    Ok(Ok(())) => println!("Insert successful"),
    Ok(Err(db_error)) => {
        // Handle database errors
        match db_error {
            IcDbmsError::Query(QueryError::PrimaryKeyConflict) => {
                println!("User already exists");
            }
            IcDbmsError::Validation(msg) => {
                println!("Validation error: {}", msg);
            }
            _ => println!("Database error: {:?}", db_error),
        }
    }
    Err(call_error) => {
        // Handle network/call errors
        println!("Failed to call canister: {:?}", call_error);
    }
}
}

Common error types:

See the Errors Reference (IC) for complete IC-specific error documentation, or the generic Errors Reference for the full error hierarchy.

Access Control (IC)

Note: This is the IC-specific access control guide. Access control is an IC-only feature. For general wasm-dbms documentation, see the generic guides.

• Overview
• Initial Configuration
  • Init Arguments
  • Deployment Example
• Managing Principals
  • Add Principal
  • Remove Principal
  • List Allowed Principals
• Authorization Enforcement
• Common Patterns
  • Frontend Canister Access
  • Admin Principal
  • Multiple Services
• Security Considerations

Overview

ic-dbms uses an Access Control List (ACL) to manage which principals can interact with the database canister. Only principals in the ACL can:

• Perform CRUD operations (insert, select, update, delete)
• Manage transactions (begin, commit, rollback)
• Modify the ACL itself (add/remove principals)

Key points:

• The ACL is set during canister initialization
• Principals can be added or removed at runtime
• All database operations check the caller against the ACL
• The ACL persists across canister upgrades

Initial Configuration

Init Arguments

When deploying the canister, you must provide the initial list of allowed principals:

type IcDbmsCanisterArgs = variant {
  Init : IcDbmsCanisterInitArgs;
  Upgrade;
};

type IcDbmsCanisterInitArgs = record {
  allowed_principals : vec principal;
};

Warning: If you deploy without including your own principal in the list, you won’t be able to interact with the canister or add yourself later!

Deployment Example

Using dfx:

# Single principal
dfx deploy my_dbms --argument '(variant { Init = record { allowed_principals = vec { principal "aaaaa-aa" } } })'

# Multiple principals
dfx deploy my_dbms --argument '(variant { Init = record { allowed_principals = vec { principal "aaaaa-aa"; principal "bbbbb-bb"; principal "ccccc-cc" } } })'

Using dfx with identity:

# Get your principal ID
dfx identity get-principal

# Deploy with your principal
ADMIN_PRINCIPAL=$(dfx identity get-principal)
dfx deploy my_dbms --argument "(variant { Init = record { allowed_principals = vec { principal \"$ADMIN_PRINCIPAL\" } } })"

Programmatically (in another canister):

#![allow(unused)]
fn main() {
use candid::Principal;
use ic_cdk::api::management_canister::main::{create_canister, install_code};

let init_args = IcDbmsCanisterArgs::Init(IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        Principal::from_text("aaaaa-aa").unwrap(),
        ic_cdk::caller(),  // Include the installing canister
    ],
});

// Install canister with init args...
}

Managing Principals

Add Principal

Add a new principal to the ACL:

#![allow(unused)]
fn main() {
use candid::Principal;
use ic_dbms_client::{IcDbmsCanisterClient, Client as _};

let client = IcDbmsCanisterClient::new(canister_id);

// Add a new principal
let new_principal = Principal::from_text("aaaaa-aa").unwrap();
client.acl_add_principal(new_principal).await??;

println!("Principal added to ACL");
}

Notes:

• Only principals already in the ACL can add new principals
• Adding an already-allowed principal is a no-op (succeeds silently)

Remove Principal

Remove a principal from the ACL:

#![allow(unused)]
fn main() {
// Remove a principal
let principal_to_remove = Principal::from_text("aaaaa-aa").unwrap();
client.acl_remove_principal(principal_to_remove).await??;

println!("Principal removed from ACL");
}

Notes:

• Only principals in the ACL can remove principals
• A principal can remove itself (be careful!)
• Removing a non-existent principal is a no-op (succeeds silently)

Warning: If you remove all principals from the ACL, no one will be able to interact with the canister. This effectively locks the canister.

List Allowed Principals

Query the current ACL:

#![allow(unused)]
fn main() {
let allowed = client.acl_allowed_principals().await?;

println!("Allowed principals:");
for principal in allowed {
    println!("  - {}", principal);
}
}

This is a query call (no cost, fast response).

Authorization Enforcement

Every canister method checks the caller against the ACL:

#![allow(unused)]
fn main() {
// If caller is NOT in ACL, all operations fail
let result = client.select::<User>(User::table_name(), query, None).await?;
// Returns error if caller not authorized
}

The canister uses an inspect function to reject unauthorized calls before execution:

#![allow(unused)]
fn main() {
// Internal canister behavior (you don't write this):
#[inspect_message]
fn inspect_message() {
    let caller = ic_cdk::caller();
    if !ACL.with(|acl| acl.borrow().is_allowed(&caller)) {
        ic_cdk::trap("Unauthorized");
    }
    ic_cdk::accept_message();
}
}

This means:

• Unauthorized calls are rejected immediately
• No cycles are consumed for unauthorized calls
• The caller receives an error response

Common Patterns

Frontend Canister Access

Allow your frontend canister to access the database:

# Get the frontend canister ID
FRONTEND_ID=$(dfx canister id my_frontend)

# Add to ACL during deployment
dfx deploy my_dbms --argument "(variant { Init = record { allowed_principals = vec { principal \"$FRONTEND_ID\" } } })"

Or add at runtime:

#![allow(unused)]
fn main() {
// In your admin script or another canister
let frontend_id = Principal::from_text("rrkah-fqaaa-aaaaa-aaaaq-cai").unwrap();
client.acl_add_principal(frontend_id).await??;
}

Admin Principal

Include an admin principal for maintenance:

#![allow(unused)]
fn main() {
let init_args = IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        frontend_canister_id,
        backend_canister_id,
        admin_principal,  // Your dfx identity for maintenance
    ],
};
}

Recommended setup:

• Include your dfx identity principal for administrative tasks
• Include all canisters that need database access
• Consider a separate admin canister for complex ACL management

Multiple Services

If multiple canisters need database access:

#![allow(unused)]
fn main() {
let init_args = IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        user_service_canister,
        order_service_canister,
        analytics_canister,
        admin_principal,
    ],
};
}

Architecture example:

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  User Service   │     │ Order Service   │     │   Analytics     │
│    Canister     │     │    Canister     │     │    Canister     │
└────────┬────────┘     └────────┬────────┘     └────────┬────────┘
         │                       │                       │
         │    All in ACL         │                       │
         └───────────────────────┼───────────────────────┘
                                 │
                                 ▼
                    ┌────────────────────────┐
                    │    IC-DBMS Canister    │
                    │      (Database)        │
                    └────────────────────────┘

Security Considerations

Principle of Least Privilege

Only add principals that genuinely need database access:

#![allow(unused)]
fn main() {
// BAD: Too permissive
let init_args = IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        frontend_canister,
        logging_canister,      // Does logging need DB access?
        monitoring_canister,   // Does monitoring need DB access?
    ],
};

// GOOD: Only necessary principals
let init_args = IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        backend_canister,  // Only the backend talks to DB
    ],
};
}

Avoid Locking Yourself Out

Always ensure at least one admin principal is in the ACL:

#![allow(unused)]
fn main() {
// DANGEROUS: Only include service canisters
let init_args = IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        frontend_canister,
    ],
};
// If frontend is deleted/upgraded incorrectly, you can't manage the DB

// SAFE: Include an admin
let init_args = IcDbmsCanisterInitArgs {
    allowed_principals: vec![
        frontend_canister,
        admin_principal,  // Fallback access
    ],
};
}

Don’t Share Admin Principals

Each developer should use their own principal:

# Each developer gets their own dfx identity
dfx identity new dev-alice
dfx identity new dev-bob

# Add each to ACL separately

Audit ACL Changes

Log ACL modifications in your application:

#![allow(unused)]
fn main() {
async fn add_principal_with_audit(
    client: &impl Client,
    principal: Principal,
    added_by: Principal,
) -> Result<(), IcDbmsError> {
    // Add to ACL
    client.acl_add_principal(principal).await??;

    // Log the change (in your own audit table)
    let audit_log = AuditLogInsertRequest {
        id: Uuid::new_v4().into(),
        action: "ACL_ADD".into(),
        target_principal: principal.to_string().into(),
        performed_by: added_by.to_string().into(),
        timestamp: DateTime::now(),
    };
    client.insert::<AuditLog>(AuditLog::table_name(), audit_log, None).await??;

    Ok(())
}
}

Consider Time-Limited Access

For temporary access (contractors, debugging), add and remove principals promptly:

#![allow(unused)]
fn main() {
// Grant temporary access
client.acl_add_principal(contractor_principal).await??;

// ... contractor does their work ...

// Revoke access when done
client.acl_remove_principal(contractor_principal).await??;
}

Client API (IC)

Note: This is the IC-specific client API guide. For general wasm-dbms documentation, see the generic docs.

• Client API (IC)
  • Overview
  • Client Types
    • IcDbmsCanisterClient
    • IcDbmsAgentClient
    • IcDbmsPocketIcClient
  • Installation
  • The Client Trait
  • Operations
    • Insert
    • Select
    • Update
    • Delete
    • Transactions
    • ACL Management
  • Error Handling
  • Examples
    • Inter-Canister Communication
    • External Application
    • Integration Testing

Overview

The ic-dbms-client crate provides type-safe Rust clients for interacting with ic-dbms canisters. Instead of manually constructing Candid calls, you use a high-level API that handles serialization and error handling.

Benefits:

• Type-safe operations with compile-time checking
• Automatic Candid encoding/decoding
• Consistent API across different environments
• Built-in error handling

Client Types

ic-dbms provides three client implementations for different use cases:

IcDbmsCanisterClient

For calls from one IC canister to another:

#![allow(unused)]
fn main() {
use ic_dbms_client::{IcDbmsCanisterClient, Client as _};
use candid::Principal;

// In your canister code
let dbms_canister_id = Principal::from_text("rrkah-fqaaa-aaaaa-aaaaq-cai").unwrap();
let client = IcDbmsCanisterClient::new(dbms_canister_id);

// Use the client
let users = client.select::<User>(User::table_name(), query, None).await??;
}

IcDbmsAgentClient

For external applications using the IC Agent:

#![allow(unused)]
fn main() {
use ic_dbms_client::{IcDbmsAgentClient, Client as _};
use ic_agent::Agent;
use candid::Principal;

// Create an IC Agent (with identity, etc.)
let agent = Agent::builder()
    .with_url("https://ic0.app")
    .with_identity(identity)
    .build()?;

agent.fetch_root_key().await?;  // Only needed for local replica

let dbms_canister_id = Principal::from_text("rrkah-fqaaa-aaaaa-aaaaq-cai").unwrap();
let client = IcDbmsAgentClient::new(dbms_canister_id, &agent);

// Use the client
let users = client.select::<User>(User::table_name(), query, None).await??;
}

IcDbmsPocketIcClient

For integration tests using PocketIC:

#![allow(unused)]
fn main() {
use ic_dbms_client::{IcDbmsPocketIcClient, Client as _};
use pocket_ic::PocketIc;
use candid::Principal;

let pic = PocketIc::new();
// ... setup canister ...

let client = IcDbmsPocketIcClient::new(
    canister_id,
    caller_principal,  // The principal making calls
    &pic
);

// Use the client in tests
let users = client.select::<User>(User::table_name(), query, None).await??;
}

Installation

Add ic-dbms-client to your Cargo.toml:

For canister development (inter-canister calls):

[dependencies]
ic-dbms-client = "0.6"

For external applications:

[dependencies]
ic-dbms-client = { version = "0.8", features = ["ic-agent"] }

For integration tests:

[dev-dependencies]
ic-dbms-client = { version = "0.8", features = ["pocket-ic"] }

The Client Trait

All clients implement the Client trait, providing a consistent API:

#![allow(unused)]
fn main() {
pub trait Client {
    // CRUD Operations
    async fn insert<T: Table>(&self, table: &str, record: T::InsertRequest, tx: Option<u64>) -> Result<Result<(), IcDbmsError>>;
    async fn select<T: Table>(&self, table: &str, query: Query<T>, tx: Option<u64>) -> Result<Result<Vec<T::Record>, IcDbmsError>>;
    async fn update<T: Table>(&self, table: &str, update: T::UpdateRequest, tx: Option<u64>) -> Result<Result<u64, IcDbmsError>>;
    async fn delete<T: Table>(&self, table: &str, behavior: DeleteBehavior, filter: Option<Filter>, tx: Option<u64>) -> Result<Result<u64, IcDbmsError>>;

    // Transactions
    async fn begin_transaction(&self) -> Result<u64>;
    async fn commit(&self, tx: u64) -> Result<Result<(), IcDbmsError>>;
    async fn rollback(&self, tx: u64) -> Result<Result<(), IcDbmsError>>;

    // ACL Management
    async fn acl_add_principal(&self, principal: Principal) -> Result<Result<(), IcDbmsError>>;
    async fn acl_remove_principal(&self, principal: Principal) -> Result<Result<(), IcDbmsError>>;
    async fn acl_allowed_principals(&self) -> Result<Vec<Principal>>;
}
}

Note the double Result:

• Outer Result: Network/communication errors
• Inner Result: Business logic errors (IcDbmsError)

Operations

Insert

#![allow(unused)]
fn main() {
use ic_dbms_client::Client as _;
use my_schema::{User, UserInsertRequest};

let user = UserInsertRequest {
    id: 1.into(),
    name: "Alice".into(),
    email: "alice@example.com".into(),
};

// Without transaction
client.insert::<User>(User::table_name(), user, None).await??;

// With transaction
client.insert::<User>(User::table_name(), user, Some(tx_id)).await??;
}

Select

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::*;

// Select all
let query = Query::builder().all().build();
let users: Vec<UserRecord> = client
    .select::<User>(User::table_name(), query, None)
    .await??;

// Select with filter
let query = Query::builder()
    .filter(Filter::eq("status", Value::Text("active".into())))
    .order_by("created_at", OrderDirection::Descending)
    .limit(10)
    .build();
let users = client.select::<User>(User::table_name(), query, None).await??;
}

Update

#![allow(unused)]
fn main() {
use my_schema::UserUpdateRequest;

let update = UserUpdateRequest::builder()
    .set_email("new@example.com".into())
    .filter(Filter::eq("id", Value::Uint32(1.into())))
    .build();

let affected_rows: u64 = client
    .update::<User>(User::table_name(), update, None)
    .await??;
}

Delete

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::DeleteBehavior;

// Delete with filter
let deleted: u64 = client
    .delete::<User>(
        User::table_name(),
        DeleteBehavior::Restrict,
        Some(Filter::eq("id", Value::Uint32(1.into()))),
        None
    )
    .await??;

// Delete all (be careful!)
let deleted: u64 = client
    .delete::<User>(
        User::table_name(),
        DeleteBehavior::Cascade,
        None,  // No filter = all records
        None
    )
    .await??;
}

Transactions

#![allow(unused)]
fn main() {
// Begin transaction
let tx_id = client.begin_transaction().await?;

// Perform operations
client.insert::<User>(User::table_name(), user1, Some(tx_id)).await??;
client.insert::<User>(User::table_name(), user2, Some(tx_id)).await??;

// Commit or rollback
match some_condition {
    true => client.commit(tx_id).await??,
    false => client.rollback(tx_id).await??,
}
}

ACL Management

#![allow(unused)]
fn main() {
use candid::Principal;

// Add principal
let new_principal = Principal::from_text("aaaaa-aa").unwrap();
client.acl_add_principal(new_principal).await??;

// Remove principal
client.acl_remove_principal(new_principal).await??;

// List principals
let allowed = client.acl_allowed_principals().await?;
for p in allowed {
    println!("Allowed: {}", p);
}
}

Error Handling

Client operations return nested Results:

#![allow(unused)]
fn main() {
// Full error handling
match client.insert::<User>(User::table_name(), user, None).await {
    Ok(Ok(())) => {
        println!("Insert successful");
    }
    Ok(Err(db_error)) => {
        // Database error (validation, constraint violation, etc.)
        match db_error {
            IcDbmsError::Query(QueryError::PrimaryKeyConflict) => {
                println!("User with this ID already exists");
            }
            IcDbmsError::Validation(msg) => {
                println!("Validation failed: {}", msg);
            }
            _ => println!("Database error: {:?}", db_error),
        }
    }
    Err(call_error) => {
        // Network/canister call error
        println!("Call failed: {:?}", call_error);
    }
}
}

Simplified with ??:

#![allow(unused)]
fn main() {
// Propagate both error types
client.insert::<User>(User::table_name(), user, None).await??;
}

Examples

Inter-Canister Communication

A backend canister calling the database canister:

#![allow(unused)]
fn main() {
use ic_cdk::update;
use ic_dbms_client::{IcDbmsCanisterClient, Client as _};
use candid::Principal;

const DBMS_CANISTER: &str = "rrkah-fqaaa-aaaaa-aaaaq-cai";

#[update]
async fn create_user(name: String, email: String) -> Result<u32, String> {
    let client = IcDbmsCanisterClient::new(
        Principal::from_text(DBMS_CANISTER).unwrap()
    );

    let user_id = generate_id();
    let user = UserInsertRequest {
        id: user_id.into(),
        name: name.into(),
        email: email.into(),
    };

    client
        .insert::<User>(User::table_name(), user, None)
        .await
        .map_err(|e| format!("Call failed: {:?}", e))?
        .map_err(|e| format!("Insert failed: {:?}", e))?;

    Ok(user_id)
}

#[update]
async fn get_users() -> Result<Vec<UserRecord>, String> {
    let client = IcDbmsCanisterClient::new(
        Principal::from_text(DBMS_CANISTER).unwrap()
    );

    let query = Query::builder().all().build();

    client
        .select::<User>(User::table_name(), query, None)
        .await
        .map_err(|e| format!("Call failed: {:?}", e))?
        .map_err(|e| format!("Query failed: {:?}", e))
}
}

External Application

A CLI tool or backend service:

use ic_agent::{Agent, identity::BasicIdentity};
use ic_dbms_client::{IcDbmsAgentClient, Client as _};
use candid::Principal;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Load identity from PEM file
    let identity = BasicIdentity::from_pem_file("identity.pem")?;

    // Create agent
    let agent = Agent::builder()
        .with_url("https://ic0.app")
        .with_identity(identity)
        .build()?;

    // For local development, fetch root key
    // agent.fetch_root_key().await?;

    let canister_id = Principal::from_text("rrkah-fqaaa-aaaaa-aaaaq-cai")?;
    let client = IcDbmsAgentClient::new(canister_id, &agent);

    // List all users
    let query = Query::builder().all().build();
    let users = client.select::<User>(User::table_name(), query, None).await??;

    for user in users {
        println!("User: {} ({})", user.name, user.email);
    }

    Ok(())
}

Integration Testing

Testing with PocketIC:

#![allow(unused)]
fn main() {
use ic_dbms_client::{IcDbmsPocketIcClient, Client as _};
use pocket_ic::PocketIc;
use candid::{encode_one, Principal};

#[tokio::test]
async fn test_user_crud() {
    // Setup PocketIC
    let pic = PocketIc::new();

    // Create and install canister
    let canister_id = pic.create_canister();
    pic.add_cycles(canister_id, 2_000_000_000_000);

    let wasm = std::fs::read("path/to/canister.wasm").unwrap();
    let init_args = IcDbmsCanisterArgs::Init(IcDbmsCanisterInitArgs {
        allowed_principals: vec![admin_principal],
    });

    pic.install_canister(
        canister_id,
        wasm,
        encode_one(init_args).unwrap(),
        None
    );

    // Create client
    let client = IcDbmsPocketIcClient::new(canister_id, admin_principal, &pic);

    // Test insert
    let user = UserInsertRequest {
        id: 1.into(),
        name: "Test User".into(),
        email: "test@example.com".into(),
    };
    client.insert::<User>(User::table_name(), user, None).await.unwrap().unwrap();

    // Test select
    let query = Query::builder().all().build();
    let users = client.select::<User>(User::table_name(), query, None).await.unwrap().unwrap();
    assert_eq!(users.len(), 1);
    assert_eq!(users[0].name.as_str(), "Test User");

    // Test update
    let update = UserUpdateRequest::builder()
        .set_name("Updated User".into())
        .filter(Filter::eq("id", Value::Uint32(1.into())))
        .build();
    let affected = client.update::<User>(User::table_name(), update, None).await.unwrap().unwrap();
    assert_eq!(affected, 1);

    // Test delete
    let deleted = client.delete::<User>(
        User::table_name(),
        DeleteBehavior::Restrict,
        Some(Filter::eq("id", Value::Uint32(1.into()))),
        None
    ).await.unwrap().unwrap();
    assert_eq!(deleted, 1);

    // Verify deletion
    let users = client.select::<User>(User::table_name(), Query::builder().all().build(), None).await.unwrap().unwrap();
    assert_eq!(users.len(), 0);
}
}

Schema Reference (IC)

Note: This is the IC-specific schema reference. For complete Table macro details, column attributes, generated types, and best practices, see the generic schema reference.

• Schema Reference (IC)
  • Overview
  • IC-Specific Required Derives
  • DatabaseSchema Macro
  • DbmsCanister Macro
    • Basic Usage
    • Generated Candid API
  • Candid Integration
    • CandidType and Deserialize
    • Candid Export
  • Complete IC Example

Overview

When deploying wasm-dbms on the Internet Computer, your schema definitions need additional IC-specific derives, the #[candid] attribute, and a canister generation macro. The core Table macro, column attributes (#[primary_key], #[unique], #[index], #[foreign_key(...)], #[sanitizer(...)], #[validate(...)], #[custom_type], #[alignment]), and generated types (Record, InsertRequest, UpdateRequest, ForeignFetcher) work exactly as described in the generic schema reference. This document covers only the IC-specific additions.

IC-Specific Required Derives

Every table struct for IC deployment must include CandidType and Deserialize in addition to the standard Table and Clone derives. You must also add the #[candid] attribute so that generated types (Record, InsertRequest, UpdateRequest) derive CandidType, Serialize, and Deserialize as well:

#![allow(unused)]
fn main() {
use candid::{CandidType, Deserialize};
use ic_dbms_api::prelude::*;

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    pub name: Text,
}
}

Without CandidType, Deserialize, and #[candid], the generated canister API will not compile because Candid is the serialization format used for all IC inter-canister calls.

DatabaseSchema Macro

The DatabaseSchema derive macro generates a DatabaseSchema<M, A> trait implementation that provides schema dispatch – routing database operations to the correct table by name at runtime. This is required by the DbmsCanister macro.

The DatabaseSchema macro is provided by wasm-dbms-macros and re-exported through the ic-dbms-canister prelude.

#![allow(unused)]
fn main() {
use ic_dbms_canister::prelude::{DatabaseSchema, DbmsCanister};
use my_schema::{User, Post};

#[derive(DatabaseSchema, DbmsCanister)]
#[tables(User = "users", Post = "posts")]
pub struct MyDbmsCanister;
}

The macro reads the #[tables(...)] attribute and generates:

• A DatabaseSchema<M, A> trait implementation that dispatches select, insert, update, delete, and select_raw calls to the correct table by name
• A register_tables associated method for convenient table registration during canister initialization

DbmsCanister Macro

The DbmsCanister macro is an IC-specific procedural macro that generates a complete Internet Computer canister API from your table definitions. It is provided by the ic-dbms-canister crate. It requires the DatabaseSchema derive to also be present on the same struct.

Basic Usage

#![allow(unused)]
fn main() {
use ic_dbms_canister::prelude::{DatabaseSchema, DbmsCanister};
use my_schema::{User, Post, Comment};

#[derive(DatabaseSchema, DbmsCanister)]
#[tables(User = "users", Post = "posts", Comment = "comments")]
pub struct MyDbmsCanister;

ic_cdk::export_candid!();
}

Format: #[tables(StructName = "table_name", ...)]

• StructName is the Rust struct name (must be in scope via use)
• "table_name" is the table name matching the #[table = "..."] attribute on the struct

Generated Candid API

For each table, the macro generates four CRUD endpoints plus shared transaction and ACL endpoints:

service : (IcDbmsCanisterArgs) -> {
  // Per-table CRUD (example for "users" table)
  insert_users : (UserInsertRequest, opt nat) -> (Result);
  select_users : (Query, opt nat) -> (Result_Vec_UserRecord) query;
  update_users : (UserUpdateRequest, opt nat) -> (Result_u64);
  delete_users : (DeleteBehavior, opt Filter, opt nat) -> (Result_u64);

  // Per-table CRUD (example for "posts" table)
  insert_posts : (PostInsertRequest, opt nat) -> (Result);
  select_posts : (Query, opt nat) -> (Result_Vec_PostRecord) query;
  update_posts : (PostUpdateRequest, opt nat) -> (Result_u64);
  delete_posts : (DeleteBehavior, opt Filter, opt nat) -> (Result_u64);

  // Transaction methods (shared)
  begin_transaction : () -> (nat);
  commit : (nat) -> (Result);
  rollback : (nat) -> (Result);

  // ACL methods (shared)
  acl_add_principal : (principal) -> (Result);
  acl_remove_principal : (principal) -> (Result);
  acl_allowed_principals : () -> (vec principal) query;
}

Method naming convention: {operation}_{table_name} (e.g., insert_users, select_posts, delete_comments)

Parameter patterns:

• opt nat is the optional transaction ID
• select methods are query calls (no state changes, no cycles consumed)
• All other methods are update calls

Init arguments:

The generated canister expects IcDbmsCanisterArgs at initialization:

type IcDbmsCanisterArgs = variant {
  Init : IcDbmsCanisterInitArgs;
  Upgrade;
};

type IcDbmsCanisterInitArgs = record {
  allowed_principals : vec principal;
};

Candid Integration

CandidType and Deserialize

These derives are needed because the IC uses Candid as its interface description language. All data crossing canister boundaries must be Candid-serializable.

The ic-dbms-api types (via wasm-dbms-api) already implement CandidType and Deserialize, so your struct only needs the derives:

#![allow(unused)]
fn main() {
use candid::{CandidType, Deserialize};
use ic_dbms_api::prelude::*;

// All field types (Uint32, Text, DateTime, etc.) already implement CandidType
#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "events"]
pub struct Event {
    #[primary_key]
    pub id: Uuid,
    pub name: Text,
    pub date: DateTime,
    pub metadata: Nullable<Json>,
}
}

Candid Export

The ic_cdk::export_candid!() macro at the end of your canister lib.rs generates the .did file that describes your canister’s interface. This is required for:

• dfx deployment
• Frontend integration
• Inter-canister calls with type checking
• Candid UI interaction

#![allow(unused)]
fn main() {
// canister/src/lib.rs
use ic_dbms_canister::prelude::{DatabaseSchema, DbmsCanister};
use my_schema::{User, Post};

#[derive(DatabaseSchema, DbmsCanister)]
#[tables(User = "users", Post = "posts")]
pub struct MyDbmsCanister;

// This MUST be at the end of the file
ic_cdk::export_candid!();
}

Complete IC Example

#![allow(unused)]
fn main() {
// schema/src/lib.rs
use candid::{CandidType, Deserialize};
use ic_dbms_api::prelude::*;

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,

    #[sanitizer(TrimSanitizer)]
    #[validate(MaxStrlenValidator(100))]
    pub name: Text,

    #[unique]
    #[sanitizer(TrimSanitizer)]
    #[sanitizer(LowerCaseSanitizer)]
    #[validate(EmailValidator)]
    pub email: Text,

    pub created_at: DateTime,
    pub is_active: Boolean,
}

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "posts"]
pub struct Post {
    #[primary_key]
    pub id: Uuid,

    #[validate(MaxStrlenValidator(200))]
    pub title: Text,

    pub content: Text,
    pub published: Boolean,

    #[index(group = "author_date")]
    #[foreign_key(entity = "User", table = "users", column = "id")]
    pub author_id: Uint32,

    pub metadata: Nullable<Json>,

    #[index(group = "author_date")]
    pub created_at: DateTime,
}
}

#![allow(unused)]
fn main() {
// canister/src/lib.rs
use ic_dbms_canister::prelude::{DatabaseSchema, DbmsCanister};
use my_schema::{User, Post};

#[derive(DatabaseSchema, DbmsCanister)]
#[tables(User = "users", Post = "posts")]
pub struct BlogDbmsCanister;

ic_cdk::export_candid!();
}

Data Types Reference (IC)

Note: This is the IC-specific data types reference. For the complete list of all data types, usage examples, and general documentation, see the generic data types reference.

• Overview
• Principal Type
  • Usage
  • Common Patterns
• Candid Type Mapping
• IC-Specific Considerations

Overview

All wasm-dbms data types are available in ic-dbms through ic_dbms_api::prelude::* (which re-exports wasm_dbms_api types). This document covers the IC-specific aspects: the Principal type (which is unique to the Internet Computer) and the Candid type mappings used for canister API serialization.

Principal Type

Principal is an Internet Computer-specific identifier type. It represents a canister ID, user identity, or the anonymous principal. This type is only meaningful in the IC context.

Usage

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::*;

#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "users"]
pub struct User {
    #[primary_key]
    pub id: Uint32,
    #[custom_type]
    pub owner: Principal,  // IC principal who owns this record
}
}

Creating principals:

#![allow(unused)]
fn main() {
use candid::Principal;

// From text representation
let principal = Principal::from_text("aaaaa-aa").unwrap();

// Anonymous principal
let anon = Principal::anonymous();

// Caller principal (inside a canister)
let caller = ic_cdk::caller();

// Management canister
let mgmt = Principal::management_canister();
}

Using in insert requests:

#![allow(unused)]
fn main() {
let user = UserInsertRequest {
    id: 1.into(),
    owner: ic_cdk::caller(),  // Store the caller's principal
};

client.insert::<User>(User::table_name(), user, None).await??;
}

Common Patterns

Recording ownership:

#![allow(unused)]
fn main() {
#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "documents"]
pub struct Document {
    #[primary_key]
    pub id: Uuid,
    pub title: Text,
    #[custom_type]
    pub owner: Principal,      // Who created this
    #[custom_type]
    pub last_editor: Principal, // Who last modified this
}
}

Filtering by principal:

#![allow(unused)]
fn main() {
// Find all documents owned by the caller
let filter = Filter::eq("owner", ic_cdk::caller().into());
let query = Query::builder().filter(filter).build();
let my_docs = client.select::<Document>(Document::table_name(), query, None).await??;
}

Nullable principal (optional ownership):

#![allow(unused)]
fn main() {
#[derive(Debug, Table, CandidType, Deserialize, Clone, PartialEq, Eq)]
#[candid]
#[table = "tasks"]
pub struct Task {
    #[primary_key]
    pub id: Uint32,
    pub title: Text,
    #[custom_type]
    pub assignee: Nullable<Principal>,  // May be unassigned
}
}

Candid Type Mapping

When ic-dbms generates the Candid interface (.did file) for your canister, each wasm-dbms type maps to a specific Candid type. This mapping is important for frontend integration, inter-canister calls, and using the Candid UI.

Frontend integration example (JavaScript/TypeScript):

// Calling from a frontend using @dfinity/agent
const user = await actor.select_users({
  filter: [{ Eq: ["name", { Text: "Alice" }] }],
  order_by: [],
  limit: [10n],  // nat64 maps to bigint
  columns: [],
  with_tables: [],
}, []);  // No transaction ID

// Principal values
import { Principal } from "@dfinity/principal";
const owner = Principal.fromText("aaaaa-aa");

IC-Specific Considerations

Re-exports: ic_dbms_api::prelude::* re-exports all types from wasm_dbms_api::prelude::* plus IC-specific additions. You do not need to import wasm_dbms_api directly.

CandidType requirement: All data types used in your table schemas must implement CandidType. The built-in types already do. If you define custom data types, they must also derive CandidType.

Principal storage: The Principal type is stored in binary format in stable memory (29 bytes max). It is serialized to/from its Candid principal representation when crossing canister boundaries.

Decimal precision: The Decimal type is serialized as text in Candid to preserve arbitrary precision. Frontends should parse the string representation rather than using floating-point conversion.

Errors Reference (IC)

Note: This is the IC-specific error handling reference. For the complete error hierarchy, all error variants, and their causes, see the generic errors reference.

• Overview
• IcDbmsError Type Alias
• Double Result Pattern
  • Why Two Results?
  • Using the ?? Operator
  • Explicit Error Handling
• Client Error Handling Examples
  • Basic Pattern
  • Detailed Matching
  • Helper Function Pattern
  • Retry Pattern for Transient Errors

Overview

When using ic-dbms through the ic-dbms-client crate, error handling has an additional layer compared to direct wasm-dbms usage. The IC’s inter-canister call model introduces network-level errors alongside database-level errors, resulting in the double Result pattern.

IcDbmsError Type Alias

IcDbmsError is a re-export of DbmsError from wasm-dbms-api, provided by ic-dbms-api for convenience:

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::IcDbmsError;

// IcDbmsError is the same as wasm_dbms_api::DbmsError
// It provides the full error hierarchy:
pub enum IcDbmsError {
    Memory(MemoryError),
    Query(QueryError),
    Table(TableError),
    Transaction(TransactionError),
    Sanitize(String),
    Validation(String),
}
}

You can use IcDbmsError or DbmsError interchangeably. The IcDbmsError alias is conventional in IC codebases.

Double Result Pattern

Why Two Results?

Client operations return Result<Result<T, IcDbmsError>, CallError>:

Result<                          -- Outer: IC call result
    Result<T, IcDbmsError>,      -- Inner: Database operation result
    CallError                    -- Network/canister call error
>

• Outer Result (CallError): The inter-canister call itself failed. This happens when:

  • The canister is unreachable or stopped
  • The canister ran out of cycles
  • The message was rejected (e.g., unauthorized caller)
  • Network timeout on agent calls

• Inner Result (IcDbmsError): The call succeeded but the database operation failed. This happens when:

  • Primary key conflict
  • Foreign key constraint violation
  • Validation failure
  • Transaction not found
  • Any other database logic error

Using the ?? Operator

The simplest approach is to use ?? to unwrap both layers:

#![allow(unused)]
fn main() {
// Propagates both CallError and IcDbmsError
let users = client.select::<User>(User::table_name(), query, None).await??;
}

This requires your function to return an error type that both CallError and IcDbmsError can convert into (e.g., Box<dyn std::error::Error>, anyhow::Error, or a custom enum).

Explicit Error Handling

#![allow(unused)]
fn main() {
match client.insert::<User>(User::table_name(), user, None).await {
    Ok(Ok(())) => {
        // Success: call succeeded AND database operation succeeded
        println!("Insert successful");
    }
    Ok(Err(db_error)) => {
        // Call succeeded but database operation failed
        println!("Database error: {:?}", db_error);
    }
    Err(call_error) => {
        // Inter-canister call itself failed
        println!("Call failed: {:?}", call_error);
    }
}
}

Client Error Handling Examples

Basic Pattern

#![allow(unused)]
fn main() {
use ic_dbms_api::prelude::{IcDbmsError, QueryError};

let result = client.insert::<User>(User::table_name(), user, None).await;

match result {
    Ok(Ok(())) => println!("Insert successful"),
    Ok(Err(e)) => println!("Database error: {:?}", e),
    Err(e) => println!("Call failed: {:?}", e),
}
}

Detailed Matching

#![allow(unused)]
fn main() {
match client.insert::<User>(User::table_name(), user, None).await {
    Ok(Ok(())) => {
        println!("Insert successful");
    }
    Ok(Err(db_error)) => {
        match db_error {
            IcDbmsError::Query(QueryError::PrimaryKeyConflict) => {
                println!("User already exists");
            }
            IcDbmsError::Query(QueryError::BrokenForeignKeyReference) => {
                println!("Referenced record doesn't exist");
            }
            IcDbmsError::Validation(msg) => {
                println!("Validation error: {}", msg);
            }
            _ => {
                println!("Database error: {:?}", db_error);
            }
        }
    }
    Err(call_error) => {
        println!("Failed to call canister: {:?}", call_error);
    }
}
}

Helper Function Pattern

#![allow(unused)]
fn main() {
fn handle_db_error(error: IcDbmsError) -> String {
    match error {
        IcDbmsError::Query(QueryError::PrimaryKeyConflict) =>
            "Record with this ID already exists".to_string(),
        IcDbmsError::Query(QueryError::BrokenForeignKeyReference) =>
            "Referenced record not found".to_string(),
        IcDbmsError::Query(QueryError::ForeignKeyConstraintViolation) =>
            "Cannot delete: record has dependencies".to_string(),
        IcDbmsError::Validation(msg) =>
            format!("Invalid data: {}", msg),
        _ =>
            format!("Unexpected error: {:?}", error),
    }
}

// Usage
let result = client.insert::<User>(User::table_name(), user, None).await;
match result {
    Ok(Ok(())) => Ok(()),
    Ok(Err(e)) => Err(handle_db_error(e)),
    Err(e) => Err(format!("Call failed: {:?}", e)),
}
}

Retry Pattern for Transient Errors

Network-level errors (outer Result) may be transient. Database errors (inner Result) are deterministic and should not be retried.

#![allow(unused)]
fn main() {
async fn insert_with_retry<T: Table>(
    client: &impl Client,
    table: &str,
    record: T::InsertRequest,
    max_retries: u32,
) -> Result<(), String> {
    for attempt in 0..max_retries {
        match client.insert::<T>(table, record.clone(), None).await {
            Ok(Ok(())) => return Ok(()),
            Ok(Err(e)) => {
                // Database errors are deterministic - don't retry
                return Err(format!("Database error: {:?}", e));
            }
            Err(call_err) => {
                // Call errors might be transient - retry
                if attempt < max_retries - 1 {
                    println!("Attempt {} failed, retrying...", attempt + 1);
                    continue;
                }
                return Err(format!("Call failed after {} attempts: {:?}", max_retries, call_err));
            }
        }
    }
    unreachable!()
}
}