C++ Development

This guide is the deep reference for writing C++ VDF implementations. It is the companion to Creating Extensions in C++, which covers the end-to-end build steps, and C++ Testing, which covers the test-and-iterate loop.

VEF Protocol 3 is stable as of v0.0.4. Protocol 4 is under development and is available only via opt-in dev ABI headers (-DVSQL_USE_DEV_ABI=ON). Extensions built against the old Protocol 2 are rejected by the server and must be rebuilt.

Writing Extension Functions

Extension functions are written in C++ and registered with VEF. Include a single header to access the full SDK:

#include <villagesql/vsql.h>

Exception Safety

Containing exceptions is the extension’s responsibility. The SDK does not wrap your implementation in a try/catch, and neither does the server, so an exception that escapes an entry point unwinds across the C ABI boundary with no handler anywhere above it and the server process terminates. Ordinary SQL input reaches this: std::stoi() throws on a non-numeric string, and nlohmann::json::dump() throws on text that is not valid UTF-8. Wrap the body of every entry point you register: VDFs, aggregate result functions, and the from_string/to_string operations of a custom type.

#include <villagesql/vsql.h>
#include <string>

using namespace vsql;

void to_int_impl(StringArg in, IntResult out) {
  if (in.is_null()) { out.set_null(); return; }
  try {
    out.set(std::stoi(std::string(in.value())));  // throws on bad input
  } catch (const std::exception &e) {
    out.warning(e.what());
  } catch (...) {
    out.warning("unexpected error");
  }
}

Catch ... in addition to const std::exception &: a dependency may throw a type that does not derive from std::exception, and that one still terminates the server. Choose between warning() and error() on the same basis as any other failure — warning() for bad input the statement should survive, error() where continuing is unsafe.

Argument and Result Types

VDF parameters and results are passed as type-safe argument and result types. The framework detects them from your function signature and adapts automatically — the make_func registration syntax is unchanged. Argument types: IntArg, RealArg, StringArg, CustomArg — each provides is_null() and value(). For parameterized custom types, CustomArgWith<P> adds a params() accessor that returns the cached parsed params struct (see Parameterized Types). Result types: IntResult, RealResult, StringResult, CustomResult — each provides set_null(), warning(msg), and error(msg). Scalar results also provide set(value). Buffer results provide buffer() and set_length(len). StringResult also provides set(std::string_view), which copies up to buffer().size() bytes from the view and sets the length in one call. For parameterized custom types, CustomResultWith<P> adds a params() accessor. Span type: value() and buffer() on the byte-oriented argument and result types return a vsql::Span<T> — a non-owning view over a contiguous run of T with data(), size(), empty(), begin()/end(), and operator[]. Under C++20 it is an alias for std::span<T>; under C++17 this SDK provides a minimal compatible implementation, so the same code compiles in either standard. It is available through <villagesql/vsql.h>. warning(msg) returns SQL NULL for the row and appends a SQL warning. In strict mode (STRICT_TRANS_TABLES), MySQL promotes it to a statement error on INSERT/UPDATE, so it behaves like error(msg) in strict contexts. Use it for recoverable bad input, such as an unparseable string in an encode function. Use error(msg) for corrupt stored data or any condition where continuing is unsafe. The message for both is truncated to fit the server’s internal error buffer if necessary. Scalar example — add two integers:

using namespace vsql;

void add_impl(IntArg a, IntArg b, IntResult out) {
  if (a.is_null() || b.is_null()) { out.set_null(); return; }
  out.set(a.value() + b.value());
}

// Registration is unchanged:
make_func<&add_impl>("add").returns(INT).param(INT).param(INT).build();

Binary example — transform a custom type buffer in place:

using namespace vsql;

void rot13_impl(CustomArg in, CustomResult out) {
  if (in.is_null()) { out.set_null(); return; }
  auto src = in.value();   // vsql::Span<const unsigned char>
  auto dst = out.buffer(); // vsql::Span<unsigned char>
  for (size_t i = 0; i < src.size(); i++) { dst[i] = transform(src[i]); }
  out.set_length(src.size());
}

For StringResult and CustomResult, write into buffer(), then call set_length() with the number of bytes written. buffer().size() is the maximum capacity. For VDFs that return a custom type (returns(CUSTOM(MYTYPE))), the server sizes the result buffer to the resolved return type’s persisted_length automatically — extension authors do not need to declare .buffer_size(...) on the function builder for this case. If prerun grows the buffer further, that larger size is preserved. This is what lets, for example, SVECTOR::from_string('[…1024 floats…]') encode a wide vector without the result buffer running out of room. You can use different styles across functions in the same extension — each function’s style is determined by its own signature.

Aggregate VDFs

Aggregate VDFs accumulate state across rows within each GROUP BY group and return a single result per group, like SQL SUM or COUNT. Use make_aggregate_func<State, &result_fn>("name") to register one. The State type is the per-group accumulation buffer; prerun and postrun are auto-generated to allocate and delete it. The result function must have the signature void(const State&, ResultType) where ResultType is one of IntResult, RealResult, StringResult, CustomResult, or CustomResultWith<P>. Call out.set(value) to return a value or out.set_null() to return SQL NULL. Both .clear<>() and .accumulate<>() are required. The builder enforces this at compile time (via build()), and the server validates it again at INSTALL EXTENSION time — clear resets state, accumulate folds rows, and the result function reads the final state.

#include <villagesql/vsql.h>
#include <optional>

using namespace vsql;

// State type: nullopt means no non-NULL rows seen yet.
using SumState = std::optional<long long>;

void my_clear(SumState &s) { s = std::nullopt; }
void my_acc(SumState &s, IntArg v) {
  if (!v.is_null()) s = s.value_or(0) + v.value();
}
void my_result(const SumState &s, IntResult out) {
  if (!s.has_value()) { out.set_null(); return; }
  out.set(s.value());
}

// Registration:
// make_aggregate_func<SumState, &my_result>("my_sum")
//     .returns(INT)
//     .param(INT)
//     .clear<&my_clear>()
//     .accumulate<&my_acc>()
//     .build()

How the builder methods work:

make_aggregate_func<State, &result_fn>() auto-generates prerun and postrun (value-initializes and deletes State).
.clear<&fn>() registers your void(State&) reset function.
.accumulate<&fn>() registers your void(State&, TypedArgs...) fold function. TypedArgs are deduced from the function signature (IntArg, StringArg, etc.).
The result type (IntResult, RealResult, etc.) is deduced from the result function signature.

For a counter that never returns NULL, use a plain state type:

using CountState = long long;
void count_clear(CountState &s) { s = 0; }
void count_acc(CountState &s, IntArg v) { if (!v.is_null()) s++; }
void count_result(const CountState &s, IntResult out) { out.set(s); }

A StringResult aggregate VDF returns text: the result reports the utf8mb4_bin charset and collation, so clients display it as characters rather than hex — the same as the scalar VDF STRING path. It also honors .max_result_length(n) — which requires the opt-in dev-ABI headers (-DVSQL_USE_DEV_ABI=ON) and Protocol 4 — the same way, sizing a materialized aggregate result (a GROUP BY/DISTINCT temp table, CREATE TABLE ... SELECT, or UNION) so it isn’t truncated at the argument width. See Custom Buffer Sizes for the sizing rules and cap.

Per-Statement State (Prerun and Postrun)

Some VDFs need state that spans every row a single query touches — a call counter, a cached result, an open resource. Allocate it in a prerun hook, access it from the VDF body, and release it in a postrun hook. Both hooks run once per statement; the VDF body runs once per row. Register them with .prerun<&Hook>() and .postrun<&Hook>(). The required signatures are:

Hook	Required signature
Prerun	`void(vsql::PrerunArgs, vsql::PrerunResult)`
Postrun	`void(vsql::PostrunArgs)`

Use PrerunResult::set_user_data(void*) to stash state; use PostrunArgs::delete_state<T>() to release it. If prerun calls set_user_data(new T{}), postrun must call delete_state<T>() — the SDK does not auto-free. PrerunArgs::type_at(i) exposes the declared SQL type of each argument before any rows are read; the predicates is_int(), is_real(), is_str(), is_custom() on the returned PrerunArgType mirror the column types. Use this in prerun to validate argument types or call PrerunResult::request_buffer_size(n) to size the result buffer.

#include <villagesql/vsql.h>
using namespace vsql;

struct CallCounter { long long n = 0; };

void ba_call_index_prerun(PrerunArgs, PrerunResult out) {
  out.set_user_data(new CallCounter{});
}

void ba_call_index(CallCounter &state, IntResult out) {
  state.n++;
  out.set(state.n);
}

void ba_call_index_postrun(PostrunArgs args) {
  args.delete_state<CallCounter>();
}

// Registration:
// make_func<&ba_call_index>("ba_call_index")
//     .returns(INT).no_params()
//     .prerun<&ba_call_index_prerun>()
//     .postrun<&ba_call_index_postrun>()
//     .build()

Varargs VDFs

A varargs VDF accepts any number of arguments of any SQL type. Declare one with .varargs() on the func builder, which is mutually exclusive with .no_params() and .param(TYPE). The body receives a vsql::VarArgs argument instead of the usual fixed-arity argument types.

Varargs registration requires VEF Protocol 3. Older servers reject the extension at install time.

The framework cannot validate argument count or types for varargs VDFs. Pair every varargs registration with a prerun hook that calls PrerunResult::error() on invalid input or PrerunResult::request_buffer_size(n) to size the result buffer. Iterate over arguments with range-for. Each AnyArg element requires a type check before reading its value:

Predicate	Accessor	Return type
`is_int()`	`as_int()`	`long long`
`is_real()`	`as_real()`	`double`
`is_str()`	`as_str()`	`std::string_view`
`is_custom()`	`as_custom()`	`vsql::Span<const unsigned char>`

Check is_null() before any accessor — all four are undefined on a null argument. PrerunArgType::custom_type() returns the argument’s custom type name as a std::string_view — empty for non-custom arguments. is_custom() alone accepts every custom type registered on the server; pairing it with a custom_type() comparison narrows a varargs call to one specific type:

#include <villagesql/vsql.h>
#include <cstring>
using namespace vsql;

// POINT2D stores two little-endian int32 (x, y) in 8 bytes — registration
// omitted here; see Custom Types for make_type<>().

void point_path_prerun(PrerunArgs args, PrerunResult out) {
  if (args.size() == 0) {
    out.error("point_path requires at least one point");
    return;
  }
  for (size_t i = 0; i < args.size(); i++) {
    auto t = args.type_at(i);
    if (!t.is_custom() || t.custom_type() != "POINT2D") {
      out.error("point_path: every argument must be a POINT2D");
      return;
    }
  }
  out.request_buffer_size(16 + args.size() * 32);
}

void point_path(VarArgs args, StringResult out) {
  auto buf = out.buffer();
  size_t off = 0;
  bool first = true;
  for (auto a : args) {
    if (a.is_null()) { out.set_null(); return; }
    auto bytes = a.as_custom();
    int x, y;
    memcpy(&x, bytes.data(), 4);
    memcpy(&y, bytes.data() + 4, 4);
    if (!first) {
      memcpy(buf.data() + off, " -> ", 4);
      off += 4;
    }
    first = false;
    off += static_cast<size_t>(
        snprintf(buf.data() + off, buf.size() - off, "(%d,%d)", x, y));
  }
  out.set_length(off);
}

// Registration:
// make_func<&point_path>("point_path")
//     .returns(STRING).varargs()
//     .prerun<&point_path_prerun>()
//     .build()

SELECT point_path(POINT2D::from_string('0,0'), POINT2D::from_string('1,2'));
-- (0,0) -> (1,2)
SELECT point_path('1,2');
-- ERROR 1123 (HY000): Can't initialize function 'point_path'; point_path: every argument must be a POINT2D

#include <villagesql/vsql.h>
#include <cstring>
using namespace vsql;

constexpr size_t kBytearrayLen = 4;

void ba_concat_all_prerun(PrerunArgs args, PrerunResult out) {
  if (args.size() == 0) {
    out.error("ba_concat_all requires at least one argument");
    return;
  }
  for (size_t i = 0; i < args.size(); i++) {
    auto t = args.type_at(i);
    if (!t.is_custom() && !t.is_str()) {
      out.error("ba_concat_all: argument " + std::to_string(i) +
                " must be BYTEARRAY");
      return;
    }
  }
  out.request_buffer_size(args.size() * kBytearrayLen);
}

void ba_concat_all(VarArgs args, StringResult out) {
  auto dst = out.buffer();
  size_t off = 0;
  for (auto a : args) {
    if (a.is_null() || !a.is_custom()) { out.set_null(); return; }
    auto bytes = a.as_custom();
    std::memcpy(dst.data() + off, bytes.data(), bytes.size());
    off += bytes.size();
  }
  out.set_length(off);
}

// Registration:
// make_func<&ba_concat_all>("ba_concat_all")
//     .returns(STRING).varargs()
//     .prerun<&ba_concat_all_prerun>()
//     .build()

Extension Load and Unload Hooks

An extension sometimes has work to do once per load rather than once per call — picking a CPU-specific implementation, or building a lookup table the VDFs then read. Register that work with .on_init<&Fn>() on the extension builder, and the matching teardown with .on_deinit<&Fn>(). Both take a function with the signature void().

.on_init<&Fn>() and .on_deinit<&Fn>() are declared only in the opt-in dev ABI headers (-DVSQL_USE_DEV_ABI=ON). A default stable-ABI build has neither method, so registering a hook fails to compile.

on_init runs at every extension load: at INSTALL EXTENSION and again on each server startup, after the server has validated and accepted the extension. It never runs for an extension the server rejected. on_deinit runs at unload — UNINSTALL EXTENSION or server shutdown. Both hooks run inside the extension process with no access to the server: they cannot run SQL, read system variables, or reach server state. That is why they are the wrong place for setup that has to talk to the server — use a preview capability’s populate step for that.

#include <villagesql/vsql.h>
#include <cmath>
using namespace vsql;

static float *g_sqrt_table = nullptr;

void build_table() {
  g_sqrt_table = new float[256];
  for (int i = 0; i < 256; i++) g_sqrt_table[i] = std::sqrt(float(i));
}

void free_table() {
  delete[] g_sqrt_table;
  g_sqrt_table = nullptr;
}

void fast_sqrt(IntArg n, RealResult out) {
  if (n.is_null() || n.value() < 0 || n.value() > 255) {
    out.set_null();
    return;
  }
  out.set(g_sqrt_table[n.value()]);
}

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .on_init<&build_table>()
        .on_deinit<&free_table>()
        .func(make_func<&fast_sqrt>("fast_sqrt")
                  .returns(REAL)
                  .param(INT)
                  .build()))

After INSTALL EXTENSION my_extension, SELECT fast_sqrt(16) returns 4 — a result that is only reachable if build_table already ran.

on_deinit is called from the vef_unregister entry point that VEF_GENERATE_ENTRY_POINTS generates. VEF_GENERATE_REGISTRATION does not define that entry point, so an extension using it must call its teardown from the vef_unregister it writes itself.

VEF_GENERATE_REGISTRATION

VEF_GENERATE_REGISTRATION creates an internal _vef_do_register() helper that performs extension registration but does not define the extern "C" entry points. Use it when you need to customize vef_register behavior — for example, to patch descriptors after registration in a test build. For normal extensions, use VEF_GENERATE_ENTRY_POINTS instead.

VEF_GENERATE_REGISTRATION(
    make_extension()
        .func(make_func<&my_impl>("my_func").returns(INT).build()))

// Then define your own extern "C" vef_register/vef_unregister that call
// _vef_do_register() and optionally modify the result.

Custom Type Operations

For the full type operation builder reference — encode, decode, compare, hash, intrinsic defaults, and parameterized types — see Type Operations.

Preview Capabilities

The following VEF capabilities are available as opt-in preview headers. ABI and API are still under active development; see Preview Capabilities for the full reference.

Extension system variables — Preview Capabilities → System Variables
Extension status variables — Preview Capabilities → Status Variables
Keyring access — Preview Capabilities → Keyring Access
Column storage — Preview Capabilities → Column Storage

Inspecting Extension Registration Metadata

INFORMATION_SCHEMA.EXTENSION_REGISTRATION exposes the in-memory VEF registration struct for each loaded extension as a JSON document. Use it to verify that the server parsed your extension’s functions, types, and system variables correctly after INSTALL EXTENSION.

SELECT EXTENSION_NAME, NEGOTIATED_PROTOCOL, REGISTRATION_JSON
FROM INFORMATION_SCHEMA.EXTENSION_REGISTRATION
WHERE EXTENSION_NAME = 'my_ext';

Column	Type	Description
`EXTENSION_NAME`	`VARCHAR(64)`	Name of the installed extension.
`NEGOTIATED_PROTOCOL`	`BIGINT UNSIGNED`	VEF protocol version negotiated between the extension and the server.
`REGISTRATION_JSON`	`VARCHAR(65535)`	JSON serialization of the `vef_registration_t` struct, including `funcs` and `types` arrays.

Getting Started

Server Development

Extensions

Using Extensions

Creating Extensions

Writing Extension Functions

Exception Safety

Argument and Result Types

Aggregate VDFs

Per-Statement State (Prerun and Postrun)

Varargs VDFs

Extension Load and Unload Hooks

VEF_GENERATE_REGISTRATION

Custom Type Operations

Preview Capabilities

Inspecting Extension Registration Metadata

See Also

​Writing Extension Functions

​Exception Safety

​Argument and Result Types

​Aggregate VDFs

​Per-Statement State (Prerun and Postrun)

​Varargs VDFs

​Extension Load and Unload Hooks

​VEF_GENERATE_REGISTRATION

​Custom Type Operations

​Preview Capabilities

​Inspecting Extension Registration Metadata

​See Also

Writing Extension Functions

Exception Safety

Argument and Result Types

Aggregate VDFs

Per-Statement State (Prerun and Postrun)

Varargs VDFs

Extension Load and Unload Hooks

VEF_GENERATE_REGISTRATION

Custom Type Operations

Preview Capabilities

Inspecting Extension Registration Metadata

See Also