Type Operations

This page is the reference for the C++ type operation builders. For the tutorial-level introduction to custom types, see Custom Types in C++. Custom types require three operations the engine calls internally: encode (string to binary), decode (binary to string), and compare. Hash is optional. Implement them against these C++ signatures (all available via <villagesql/vsql.h>):

Fixed-Length Types

// Encode: string -> binary. Write the encoded bytes via out.buffer() and
// out.set_length(n); call out.set_null() for SQL NULL, out.warning(msg) for
// recoverable bad input, or out.error(msg) to abort the statement. Returning
// without calling any of these surfaces a default warning.
using TypeEncodeFunc = void (*)(std::string_view from, vsql::CustomResult out);

// Decode: binary -> string. Report the outcome by calling
// out.set_length(n), out.set(sv), out.set_null(), out.warning(msg), or
// out.error(msg). If none is called the SDK falls back to a default
// "failed to decode value" ERROR.
using TypeDecodeFunc = void (*)(vsql::CustomArg in, vsql::StringResult out);

// Compare: returns -1, 0, or 1 (used for ORDER BY and indexes).
using TypeCompareFunc = int (*)(vsql::CustomArg a, vsql::CustomArg b);

// Hash: returns hash code (used for hash joins).
using TypeHashFunc = size_t (*)(vsql::CustomArg in);

Register these operations using vsql::make_type<kTypeName>(). The type name is passed as a non-type template parameter (NTTP) — a static constexpr const char[] array. The builder auto-generates VDF names in the TYPE::method format (e.g., "MYTYPE::from_string") from this NTTP, so no manual string matching is required. Pass the built type object to .type() on the extension builder; separate .func() calls for type operations are not needed.

The type name must be a static constexpr const char[] variable — a string literal cannot be used as a non-type template parameter. Passing "MYTYPE" directly produces a compiler error like:

error: '"MYTYPE"' is not a valid template argument for type 'const char*'

Declare the name as a named array as shown below.

#include <villagesql/vsql.h>

using namespace vsql;

static constexpr const char kMyTypeName[] = "MYTYPE";

constexpr auto MYTYPE =
    vsql::make_type<kMyTypeName>()
        .persisted_length(8)
        .max_decode_buffer_length(64)
        .from_string<&my_encode>()   // auto: "MYTYPE::from_string"
        .to_string<&my_decode>()     // auto: "MYTYPE::to_string"
        .compare<&my_compare>()      // auto: "MYTYPE::compare"
        .hash<&my_hash>()            // optional, auto: "MYTYPE::hash"
        .intrinsic_default_str("0")  // string-literal intrinsic default
        .build();

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .type(MYTYPE))

build() fails at compile time if from_string, to_string, or compare is missing. Each template method checks the function pointer signature via static_assert.

Intrinsic Default

When a NOT NULL custom-type column receives NULL under IGNORE mode (e.g., INSERT IGNORE or UPDATE IGNORE), the server calls the intrinsic default to produce a fallback value rather than raising an error. The intrinsic default provides a string representation; the server converts it to binary using the type’s from_string function.

If you omit both .intrinsic_default_str() and .intrinsic_default_vdf(), the server calls from_string("") as a fallback. This happens when the type is first used (at table creation), not at INSTALL EXTENSION. If your encode function rejects empty string — or encodes it to the wrong number of bytes — type initialization fails with an error visible in the SQL client:

Type 'MYTYPE' failed to initialize: from_string VDF encoded intrinsic
default input '' to N bytes, expected persisted_length=M

For fixed-length types, the default string must encode to exactly persisted_length bytes. Set an explicit default for any type where empty string is not a valid input.

String literal: .intrinsic_default_str() For a constant default, pass the string directly on the type builder (as shown in the fixed-length example above with .intrinsic_default_str("0")). VDF-based: .intrinsic_default_vdf() + make_intrinsic_default When the default value depends on type parameters, implement a function against one of these signatures (available via <villagesql/vsql.h>):

Breaking change: IntrinsicDefaultFunc and IntrinsicDefaultWithParamsFunc return std::string instead of const char*. Update any existing intrinsic default implementations to return std::string directly.

// Fixed (no type parameters):
using IntrinsicDefaultFunc = std::string (*)(char *error_msg);

// Parameterized (receives cached parsed params):
template <typename P>
using IntrinsicDefaultWithParamsFunc = std::string (*)(const P &,
                                                       char *error_msg);

Return a std::string representation of the default value. On error, write a message to error_msg and return any value (the SDK checks error_msg[0] != '\0' to detect errors). Register with make_intrinsic_default<&fn>("vdf_name") (one argument: the VDF name) and reference that name on the type builder with .intrinsic_default_vdf(). The parameterized types example below shows the full registration pattern.

std::string mytype_default(const MyTypeParams &p, char * /*error_msg*/) {
  return /* build string representation based on p */;
}

Parameterized Types

Parameterized types need the column’s declared parameters at encode, decode, compare, and hash time to determine allocation sizes and layout. Define a params struct with a parse function and an inverse to_strings function, register both on the type builder with .params<P, &ParseFunc, &ToStringsFunc>(), and use const P& as the first argument of your type operation functions. The SDK caches the parse result per unique parameter combination, so the parse function runs at most once per type instantiation. The to_strings function is the inverse of parse: it writes a typed P back into the canonical key/value string form so the server can publish inferred params in the same shape parse consumes.

struct MyTypeParams {
  int64_t dimension;
  static MyTypeParams parse(const std::map<std::string, std::string> &p) {
    return {.dimension = stoll(p.at("dimension"))};
  }
  static void to_strings(const MyTypeParams &p,
                         std::map<std::string, std::string> &out) {
    out["dimension"] = std::to_string(p.dimension);
  }
};

void mytype_encode(vsql::MaybeParams<MyTypeParams> &params,
                   std::string_view from, vsql::CustomResult out) {
  const MyTypeParams &p = params.value();  // is_known() is always true at runtime
  size_t bytes = (size_t)p.dimension * 4;
  auto buf = out.buffer();
  if (buf.size() < bytes) { out.error("MYTYPE: buffer too small"); return; }
  // ... parse from, write to buf ...
  out.set_length(bytes);
}

void mytype_decode(vsql::CustomArgWith<MyTypeParams> in,
                   vsql::StringResult out) {
  const MyTypeParams &p = in.params();
  // ... read p.dimension floats from in.value(), write to out.buffer() ...
  out.set_length(bytes_written);
}

int mytype_compare(vsql::CustomArgWith<MyTypeParams> a,
                   vsql::CustomArgWith<MyTypeParams> b) {
  // Returns -1, 0, or 1.
}

size_t mytype_hash(vsql::CustomArgWith<MyTypeParams> in) {
  // Returns hash code.
}

// Converts MYTYPE(N) integer syntax to a parameter map.
// Signature: IntToTypeParamsFunc from <villagesql/vsql.h>.
bool mytype_int_to_params_fn(int64_t value,
                             std::map<std::string, std::string> &params,
                             char *error_msg) {
  if (value <= 0) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: dimension must be a positive integer");
    return true;
  }
  params["dimension"] = std::to_string(value);
  return false;  // success
}

// Validates parameters and computes storage sizes.
// Signature: ResolveTypeParamsFunc from <villagesql/vsql.h>.
bool mytype_resolve_params_fn(const std::map<std::string, std::string> &params,
                              vsql::ResolvedTypeParams *result,
                              char *error_msg) {
  int64_t dim = std::stoll(params.at("dimension"));
  result->persisted_length = dim * 4;
  result->max_decode_buffer_length = 64;
  return false;  // success
}

Register .params<>() on the type builder. Use .int_to_params<&mytype_int_to_params_fn>() to handle MYTYPE(N) integer syntax and .resolve_params<&mytype_resolve_params_fn>() to validate parameters and compute storage sizes. Call .max_persisted_length(N) with an upper bound on the persisted byte size across all valid parameterizations; the server uses this only on the type parameter inference path, where it has not yet inferred the params and so cannot consult resolve_params to size the encode buffer. For a VDF-based intrinsic default, use .intrinsic_default_vdf() with the VDF name and register the VDF separately via make_intrinsic_default<&mytype_default>().

.max_persisted_length() requires VEF Protocol 3 or higher. A type using it cannot be loaded by pre-Protocol-3 servers.

static constexpr const char kMyTypeName[] = "MYTYPE";

// Maximum valid dimension for MYTYPE.
constexpr int64_t kMyTypeMaxDimension = 1024;  // your max valid dimension
// Upper bound on MYTYPE's persisted byte size across all valid params.
constexpr int64_t kMyTypeMaxPersistedLength = kMyTypeMaxDimension * 4;

constexpr auto MYTYPE =
    vsql::make_type<kMyTypeName>()
        .variable_length_type()  // Protocol 4; use instead of persisted_length(-1)
        .max_decode_buffer_length(16)
        .max_persisted_length(kMyTypeMaxPersistedLength)
        .params<MyTypeParams, &MyTypeParams::parse, &MyTypeParams::to_strings>()
        .int_to_params<&mytype_int_to_params_fn>()
        .resolve_params<&mytype_resolve_params_fn>()
        .from_string<&mytype_encode>()
        .to_string<&mytype_decode>()
        .compare<&mytype_compare>()
        .intrinsic_default_vdf("mytype_intrinsic_default")
        .build();

using namespace vsql;

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .type(MYTYPE)
        .func(make_intrinsic_default<&mytype_default>(
            "mytype_intrinsic_default")))

The parameterized variants — TypeEncodeWithParamsFunc, TypeDecodeWithParamsFunc, TypeCompareWithParamsFunc, and TypeHashWithParamsFunc — together with ParamsToStringsFunc (void fn(const P&, std::map<std::string,std::string>&)) are available via <villagesql/vsql.h>. The vsql::make_type template methods detect the params argument and route through the params cache automatically. Encode functions take vsql::MaybeParams & as the first argument; is_known() is always true at runtime, and value() returns const P&. Decode, compare, and hash variants take vsql::CustomArgWith, whose params() accessor returns const P&. Supplying parameters in SQL. Two syntaxes reach resolve_params:

Integer — MYTYPE(N). The server routes N through int_to_params to build the parameter map. Requires .int_to_params<>().
String — MYTYPE('key=value,...'). The server normalizes the string and calls resolve_params directly; int_to_params is not involved. Available whenever .resolve_params<>() is registered — no extra builder call.

A type that registers only .resolve_params<>() accepts the string form and rejects MYTYPE(N). SHOW CREATE TABLE preserves whichever form was written.

CREATE TABLE t (v ext.MYTYPE(8));              -- integer form (.int_to_params)
CREATE TABLE t2 (v ext.MYTYPE('dimension=8')); -- string form (.resolve_params only)

The serialized key=value,... parameter string that int_to_params produces and resolve_params consumes is capped at VEF_MAX_TYPE_PARAMS_STRING_LEN (1024 bytes). A parameterization whose canonical string would exceed that limit is rejected with a defined error rather than truncated silently — keep the combined parameter names and values for a single type within 1024 bytes.

Rewriting parameters and supplying defaults

resolve_params has a second, mutating overload: it takes the parameter map by non-const reference so the type can rewrite it — typically to fill defaults the author omitted. Register it the same way (.resolve_params<&fn>() accepts either form; register only one):

bool mytype_resolve_params_fn(std::map<std::string, std::string> &params,
                              vsql::ResolvedTypeParams *result, char *error_msg) {
  if (params.find("dimension") == params.end())
    params["dimension"] = "128";                 // supply a default
  int64_t dim = std::stoll(params.at("dimension"));
  result->persisted_length = dim * 4;
  result->max_decode_buffer_length = 64;
  return false;                                  // success
}

The rewritten map becomes the canonical parameter string the server persists and SHOW CREATE TABLE prints, so the rewrite must be idempotent. A bare declaration (MYTYPE, no length or params) now calls resolve_params with an empty map instead of skipping it, so a type that supplies defaults gives every column explicit params — vsql_bitfield_test’s BITFIELD resolves a bare column to max_number_of_bits=4096:

INSTALL EXTENSION vsql_bitfield_test;
CREATE TABLE bits (id INT PRIMARY KEY, b vsql_bitfield_test.BITFIELD);
SHOW CREATE TABLE bits;   -- b persists as BITFIELD('max_number_of_bits=4096')

Variable-Length Types

A variable-length custom type decides its persisted size per value rather than using a single fixed footprint. Declare one by calling .variable_length_type() on the type builder, which sets the type’s variable_length flag.

.variable_length_type() raises the type’s required protocol to VEF Protocol 4. The server reads the variable_length flag only at protocol 4 or higher. Build against the opt-in dev ABI headers (-DVSQL_USE_DEV_ABI=ON); older servers do not read the flag.

Variable-length types must also call .max_persisted_length(N). build() fails at compile time if it is omitted — the server needs the upper bound to allocate a buffer for the backing field. .variable_length_type() is monotonic: calling it before or after the Protocol 3 setters (max_persisted_length(), params(), int_to_params()) does not lower the protocol requirement back below Protocol 4.

#include <villagesql/vsql.h>
using namespace vsql;

static constexpr const char kMyTypeName[] = "MYTYPE";
constexpr int64_t kMyTypeMaxPersistedLength = 4096;

constexpr auto MYTYPE =
    vsql::make_type<kMyTypeName>()
        .variable_length_type()  // per-value sizing; requires Protocol 4
        .max_persisted_length(kMyTypeMaxPersistedLength)
        .max_decode_buffer_length(64)
        .from_string<&my_encode>()
        .to_string<&my_decode>()
        .compare<&my_compare>()
        .build();

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .type(MYTYPE))

Like every custom type, a variable-length type must produce a usable intrinsic default. The default is encoded into the field’s maximum capacity, and any non-empty result of 1 to max_persisted_length bytes is accepted. A type whose empty-string encode produces zero bytes — an empty array or bit set, say — has no usable default, so declare an explicit one that encodes to a non-empty value:

        .max_persisted_length(kMyTypeMaxPersistedLength)
        .intrinsic_default_str("[0]")  // empty "[]" would encode to zero bytes

Otherwise the type fails to initialize the first time a NOT NULL column references it, at CREATE TABLE — the same as a fixed-length type that cannot encode from_string("").

Custom Types in Stored Procedures

Custom extension types can be used as stored procedure parameter types and in DECLARE variable declarations. The server resolves the custom type at routine execution time using the installed extension’s type metadata.

DELIMITER //
CREATE PROCEDURE insert_complex(IN val COMPLEX)
BEGIN
  DECLARE tmp COMPLEX;
  SET tmp = val;
  INSERT INTO t1 VALUES (tmp);
END //
DELIMITER ;

Getting Started

Server Development

Extensions

Using Extensions

Creating Extensions

Fixed-Length Types

Intrinsic Default

Parameterized Types

Rewriting parameters and supplying defaults

Variable-Length Types

Custom Types in Stored Procedures

See Also

​Fixed-Length Types

​Intrinsic Default

​Parameterized Types

​Rewriting parameters and supplying defaults

​Variable-Length Types

​Custom Types in Stored Procedures

​See Also

Fixed-Length Types

Intrinsic Default

Parameterized Types

Rewriting parameters and supplying defaults

Variable-Length Types

Custom Types in Stored Procedures

See Also