Creating Extensions in C++ - VillageSQL Server for MySQL

VEF Protocol 3 is stable as of v0.0.4. Protocol 4 is under development and is available only via opt-in dev ABI headers (-DVSQL_USE_DEV_ABI=ON). Extensions built against the old Protocol 2 are rejected by the server and must be rebuilt.

Overview

VillageSQL’s extension framework (VEF) lets you add custom functionality to the database server. This guide walks through building an extension in C++ using the C++ SDK and the extension template. For writing VDF implementations in depth — argument and result types, aggregates, system variables, parameterized types — see the Development Guide.

If you prefer Rust, see Creating Extensions in Rust.

What is a VillageSQL Extension?

A VillageSQL extension is packaged as a VEB file (VillageSQL Extension Bundle) containing:

Manifest - Metadata about the extension (name, version, description)
Shared library - Compiled C++ code implementing the functionality
Optional metadata - Additional resources or configuration

C++ extensions are built using the C++ SDK, the C++ binding for VEF (the VillageSQL Extension Framework). It provides:

C++ API for defining types and functions
Automatic registration without SQL scripts
Type-safe argument and result types
Builder pattern for extension definition

VDFs vs Traditional UDFs: Functions registered through VEF are called VDFs (VillageSQL Defined Functions). VillageSQL also supports traditional MySQL UDFs registered via CREATE FUNCTION ... SONAME, but VDFs are recommended for new extensions.

Calling VDFs in SQL

VDFs can be called with or without the extension prefix:

-- Unqualified (preferred for cleaner code)
SELECT complex_abs(impedance) FROM signals;

-- Qualified with extension name (explicit)
SELECT vsql_complex.complex_abs(impedance) FROM signals;

Function Resolution Order: When you call a function without qualification, VillageSQL resolves it in this order:

System functions (built-in MySQL functions like NOW(), CONCAT())
UDFs (traditional MySQL user-defined functions)
VDFs (extension functions) - only if there’s exactly one function with that name
Stored functions (created with CREATE FUNCTION)

When to Use Qualified Names:

Use extension.function_name when multiple extensions provide functions with the same name
Use unqualified names for cleaner code when there’s no ambiguity
Qualification is never required if only one extension provides that function name

Extensions can add:

Custom functions (VDFs) - SQL functions with automatic type checking and validation
Custom data types - New column types like COMPLEX, UUID, or VECTOR that work with ORDER BY and indexes
Type operations - Encode, decode, compare, and hash functions for custom types

Prerequisites

Before you begin, build VillageSQL from source — extensions link against the server’s SDK headers and build tree. Follow the Build from Source guide first. You also need:

Git - For cloning and version control
CMake 3.18 or higher - Build system
C++ Compiler - GCC 8+, Clang 8+, or MSVC 2019+ with C++17 support
Basic C++ knowledge - Understanding of C++ and function pointers

Building with an AI agent? The vsql-extension-builder skill automates this entire workflow — from scaffolding to testing — using Claude Code, Gemini, or other supported agents. Install it with:

curl -sSL https://villagesql.com/skills | bash

Step 1: Get the Extension Template

You can start with the extension template in two ways:

Option A: Use Template from VillageSQL Source

If you have VillageSQL source code, the template is included:

cd /path/to/villagesql-source
cp -r villagesql/sdk/template my-extension
cd my-extension

Option B: Fork from GitHub

Start by forking the VillageSQL extension template repository:

Visit the template repository on GitHub:

https://github.com/villagesql/vsql-extension-template

Click the “Fork” button to create your own copy

Clone your fork locally:

git clone https://github.com/YOUR_USERNAME/vsql-extension-template.git
cd vsql-extension-template

Alternatively, use the “Use this template” button on GitHub to create a new repository based on the template without forking history.

Step 2: Update the Manifest

Edit manifest.json to define your extension’s metadata:

{
  "$schema": "https://raw.githubusercontent.com/villagesql/villagesql-docs/main/mysql-8.4/0.0.5/manifest.json.schema.json",
  "name": "my_awesome_extension",
  "version": "1.0.0",
  "description": "My custom VillageSQL extension that does amazing things",
  "author": "Your Name",
  "license": "GPL-2.0"
}

The $schema field is optional but enables IDE autocomplete and inline validation for all manifest fields.

manifest.json Schema

Field	Required	Format	Description
`name`	✅ Yes	letters, digits, `_`, `-`	Unique identifier. Must match `INSTALL EXTENSION` name.
`version`	✅ Yes	MAJOR.MINOR.PATCH	Semantic version string
`description`	No	String	Brief explanation of functionality
`author`	No	String	Author name or organization
`license`	No	String	License identifier (GPL-2.0 recommended)

Validation rules:

name: Must start with a letter and end with a letter or digit. May contain lowercase letters, digits, underscores, and hyphens. Max 64 characters. Use underscores — hyphens require backtick quoting in SQL.
version: Must follow semantic versioning (e.g., 1.0.0, 0.2.1)
Invalid manifest will cause INSTALL EXTENSION to fail

Example:

-- manifest.json has "name": "my_awesome_extension"
INSTALL EXTENSION my_awesome_extension;  -- ✅ Correct: no quoting needed
INSTALL EXTENSION `my-awesome-extension`;  -- ⚠️ Works, but requires backtick quoting

For the full naming convention across SQL, filenames, and repository names, see Extension Naming Conventions.

Step 3: Implement Your Extension with the C++ SDK

The C++ SDK provides a C++ API for defining extensions using a fluent builder pattern:

Type-safe function definitions with compile-time checking
Automatic argument validation and type conversion
Support for custom types with compare/hash functions (enables ORDER BY and indexes)

Include VillageSQL Headers

Create your main extension file (e.g., src/extension.cc) and include the C++ SDK headers:

#include <villagesql/vsql.h>

// Your implementation code here

The <villagesql/vsql.h> header pulls in the type builder, function builder, and extension builder, and re-exports commonly used symbols into the vsql namespace.

Define Your Extension

Use the VEF_GENERATE_ENTRY_POINTS() macro to define your extension:

VEF_GENERATE_ENTRY_POINTS(
  make_extension()
    .func(make_func<&my_reverse_impl>("my_reverse")
      .returns(STRING)
      .param(STRING)
      .build())
    .func(make_func<&count_vowels_impl>("count_vowels")
      .returns(INT)
      .param(STRING)
      .build())
);

Function Builder Methods:

make_func<&impl>("name") - Create function with implementation pointer
.returns(type) - Set return type (STRING, INT, REAL, or custom type name)
.param(type) - Add parameter (maximum 8 parameters)
.buffer_size(size_t) - Request specific output buffer size for STRING/CUSTOM returns
.max_result_length(size_t) - Size the result column for a STRING return so a materialized result isn’t truncated at the argument width. Values above VEF_MAX_RESULT_LENGTH (16 MiB) are capped by the server. STRING only; call .returns(STRING) first. Requires Protocol 4 (dev ABI).
.deterministic(bool = true) - Declare that this function always returns the same output for the same inputs and has no side effects. Default is non-deterministic.
.prerun<func>() - Set per-statement setup function (optional)
.postrun<func>() - Set per-statement cleanup function (optional)
.build() - Finalize function registration

Parameter Limit: Functions support a maximum of 8 parameters (defined by kMaxParams). If you need more, consider using structured types or multiple functions.

VDFs with Custom Type Arguments and Return Values

A VDF can take and return custom type values using .param(TYPE_NAME) and .returns(TYPE_NAME) in the builder. The implementation uses CustomArg for input and CustomResult for output — the same argument and result types used for type operations:

void complex_conjugate_impl(CustomArg in, CustomResult out) {
    if (in.is_null()) { out.set_null(); return; }
    auto src = in.value();    // Span<const unsigned char> — raw binary
    auto dst = out.buffer();  // Span<unsigned char>
    // ... read src, write result to dst ...
    out.set_length(src.size());
}

.func(make_func<&complex_conjugate_impl>("complex_conjugate")
          .returns(COMPLEX)
          .param(COMPLEX)
          .build())

See the development guide for the full CustomArg/CustomResult API, including CustomArgWith<P> and CustomResultWith<P> for parameterized types.

Deterministic Functions

By default, VDFs are registered as non-deterministic. A non-deterministic function is blocked from three SQL contexts: generated columns, CHECK constraints, and expression default values (DEFAULT (expr) on a column) — using any of these features with a non-deterministic VDF returns an error. If your function always produces the same output for the same inputs and has no side effects, you can declare it deterministic by adding .deterministic() to the builder chain. The optimizer may use this information to evaluate the function once per statement and reuse the value across rows, rather than calling it per row. Incorrectly marking a non-deterministic function as deterministic can therefore cause the server to return the same result for inputs that should produce different outputs. Only add .deterministic() when your function truly has no dependency on external state, randomness, or time. Builder signature: .deterministic(bool d = true) — the zero-argument form defaults to true. Example:

.func(make_func<&complex_add_impl>("complex_add")
          .returns(COMPLEX)
          .param(COMPLEX)
          .param(COMPLEX)
          .deterministic()
          .build())

Because complex_add is declared deterministic, it can be used in a generated column definition:

-- Deterministic VDFs can be used in generated columns
CREATE TABLE t (
    a COMPLEX,
    b COMPLEX,
    result COMPLEX GENERATED ALWAYS AS (complex_add(a, b)) STORED
);
-- STORED vs VIRTUAL follows standard MySQL generated column rules

Custom Buffer Sizes

For functions returning variable-length data, request a specific buffer size:

make_func<&large_result_impl>("large_result")
  .returns(STRING)
  .param(INT)
  .buffer_size(65536)  // Request 64KB buffer
  .build()

When you build the value yourself with buffer() and set_length(), the buffer is fixed at buffer().size() bytes — writing past it overflows memory, so guard against it:

void large_result_impl(StringArg input, StringResult out) {
    if (input.is_null()) { out.set_null(); return; }
    size_t needed = calculate_output_size(input.value());

    auto buf = out.buffer();
    if (needed > buf.size()) {
        out.error("Output exceeds buffer size");
        return;
    }

    // Write output into buf.data()
    out.set_length(actual_output_length);
}

For STRING results, out.set(sv) follows a snprintf-style overflow contract: it copies as many bytes as fit and reports the value’s full size via set_length. When the reported size exceeds the buffer, the server grows the result buffer and re-invokes the function, so a STRING value larger than the requested buffer is no longer truncated. That contract applies at row time. A materialized STRING result — in a GROUP BY/DISTINCT temp table, CREATE TABLE ... SELECT, or UNION — still truncates at the argument width unless the function declares .max_result_length(n), which sizes the result column (in characters, capped at VEF_MAX_RESULT_LENGTH, 16 MiB):

make_func<&large_result_impl>("large_result")
  .returns(STRING)
  .max_result_length(65536)
  .build()

Request sufficient buffer size via .buffer_size() based on your function’s maximum output size. Sizing the buffer correctly avoids the cost of a grow-and-retry round trip.

Before implementing your functions, review the C++ API Reference for the complete VDF contracts: null checking, result types, buffer sizing, and error handling.

Step 4: Creating Custom Types

Custom types let you define new column types — like COMPLEX, UUID, or VECTOR — that work with ORDER BY, indexes, and aggregate functions. If your extension only registers functions, skip to Step 5. See Custom Types in C++ for the full implementation walkthrough. If your type takes parameters (e.g., VECTOR(1536)), see Parameterized Types.

Step 5: Update Build Configuration

Edit CMakeLists.txt to build your extension as a VEB file:

cmake_minimum_required(VERSION 3.18)
project(my_extension)

# Find VillageSQL Extension Framework
find_package(VillageSQLExtensionFramework QUIET)

# The framework detects build flags via 4 methods (in order):
# 1. Explicit MYSQL_INCLUDE_FLAGS/MYSQL_CXXFLAGS
# 2. VillageSQL_BUILD_DIR (reads CMakeCache.txt from VillageSQL build)
# 3. VSQL_BASE_DIR (uses mysql_config)
# 4. Default - mysql_config from PATH

# Build shared library with your source files
add_library(extension SHARED
    src/extension.cc
    src/my_functions.cc
)

# Create VEB archive
VEF_CREATE_VEB(
    NAME my_extension
    LIBRARY_TARGET extension
    MANIFEST ${CMAKE_CURRENT_SOURCE_DIR}/manifest.json
)

# Install VEB to VillageSQL extensions directory
install(FILES ${VEB_OUTPUT} DESTINATION ${INSTALL_DIR})

Configuration notes:

VillageSQLExtensionFramework provides CMake helpers for building extensions
VEF_CREATE_VEB() packages your library, manifest, and metadata into a .veb archive
The framework automatically detects MySQL/VillageSQL build flags
Library target name is typically extension (can be anything)
VEB name must match your manifest.json name
By default, extensions build against the stable ABI headers. Set -DVSQL_USE_DEV_ABI=ON to build against the unstable dev headers instead

Step 6: Create a Build Directory

Create a separate build directory:

mkdir build
cd build

Step 7: Build with CMake and Make

Configure and build your extension:

# Configure the build
cmake ..

# Or, if building against VillageSQL source:
cmake .. -DVillageSQL_BUILD_DIR=/path/to/villagesql/build

# Or, to build against the unstable dev ABI headers:
cmake .. -DVSQL_USE_DEV_ABI=ON

# Build the extension
make

This creates:

Compiled shared library (.so file)
VEB package (.veb file) - a tar archive containing manifest and library

Verify the Build

Check the contents of your VEB file:

make show_veb

You should see:

manifest.json
lib/myext.so

Step 8: Install and Test

Option A: Install to VillageSQL Extensions Directory

Use the install target to copy the VEB to your VillageSQL installation:

make install

This copies the .veb file to the directory configured via VillageSQL_VEB_INSTALL_DIR.

Option B: Manual Installation

Copy the VEB file manually:

# Find the VEF directory
mysql -u root -p -e "SHOW VARIABLES LIKE 'veb_dir';"

# Copy the VEB file
sudo cp my-awesome-extension.veb /path/to/veb_dir/

Test Your Extension

Connect to VillageSQL:
mysql -u root -p

Install the extension:

INSTALL EXTENSION my_awesome_extension;

Verify installation:

SELECT * FROM INFORMATION_SCHEMA.EXTENSIONS;

Test your functions:

SELECT my_reverse('Hello, World!');
-- Output: !dlroW ,olleH

SELECT count_vowels('VillageSQL');
-- Output: 3

Creating Tests

Add test files to validate your extension works correctly:

Create a test file in mysql-test/t/:

-- mysql-test/t/my_basic.test
SELECT my_reverse('abc');
SELECT my_reverse('');
SELECT my_reverse(NULL);

Generate expected results:

cd /path/to/villagesql/build/mysql-test
perl mysql-test-run.pl --suite=/path/to/your/extension/mysql-test --record

Run tests:

perl mysql-test-run.pl --suite=/path/to/your/extension/mysql-test

Troubleshooting

Extension Won’t Load

Check the error log and verify the VEB contents:

make show_veb
tail -f /var/log/mysql/error.log

Function Not Found

Verify installation and registration:

SELECT * FROM INFORMATION_SCHEMA.EXTENSIONS;

Build Errors

# Verify mysql_config is available
which mysql_config
mysql_config --version

# Check compiler version
gcc --version  # or clang --version

# Verify CMake version (3.18+ required)
cmake --version

Example Extensions

Learn from existing VillageSQL extensions:

vsql_complex

Complex number data type implementation

vsql_extension_template

Minimal template for creating extensions

Next Steps

Using Extensions

Learn how to install and manage extensions

Development Guide

Argument and result types, aggregates, system variables, and testing

Extension Architecture

Lifecycle, caching, performance, and security model

Build from Source

Build VillageSQL from source code

​Overview

​What is a VillageSQL Extension?

​Calling VDFs in SQL

​Prerequisites

​Step 1: Get the Extension Template

​Option A: Use Template from VillageSQL Source

​Option B: Fork from GitHub

​Step 2: Update the Manifest

​manifest.json Schema

​Step 3: Implement Your Extension with the C++ SDK

​Include VillageSQL Headers

​Define Your Extension

​VDFs with Custom Type Arguments and Return Values

​Deterministic Functions

​Custom Buffer Sizes

​Step 4: Creating Custom Types

​Step 5: Update Build Configuration

​Step 6: Create a Build Directory

​Step 7: Build with CMake and Make

​Verify the Build

​Step 8: Install and Test

​Option A: Install to VillageSQL Extensions Directory

​Option B: Manual Installation

​Test Your Extension

​Creating Tests

​Troubleshooting

​Extension Won’t Load

​Function Not Found

​Build Errors

​Example Extensions

vsql_complex

vsql_extension_template

​Next Steps

Using Extensions

Development Guide

Extension Architecture

Build from Source

​Resources

Overview

What is a VillageSQL Extension?

Calling VDFs in SQL

Prerequisites

Step 1: Get the Extension Template

Option A: Use Template from VillageSQL Source

Option B: Fork from GitHub

Step 2: Update the Manifest

manifest.json Schema

Step 3: Implement Your Extension with the C++ SDK

Include VillageSQL Headers

Define Your Extension

VDFs with Custom Type Arguments and Return Values

Deterministic Functions

Custom Buffer Sizes

Step 4: Creating Custom Types

Step 5: Update Build Configuration

Step 6: Create a Build Directory

Step 7: Build with CMake and Make

Verify the Build

Step 8: Install and Test

Option A: Install to VillageSQL Extensions Directory

Option B: Manual Installation

Test Your Extension

Creating Tests

Troubleshooting

Extension Won’t Load

Function Not Found

Build Errors

Example Extensions

Next Steps

Resources