> ## Documentation Index
> Fetch the complete documentation index at: https://villagesql.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Type Operations

> Deep reference for the C++ type operation builders — encode, decode, compare, hash, intrinsic defaults, and parameterized types.

This page is the reference for the C++ type operation builders. For the tutorial-level introduction to custom types, see [Custom Types in C++](https://villagesql.com/docs/en/sdk/cpp/custom-types.md).

Custom types require three operations the engine calls internally: encode (string to binary), decode (binary to string), and compare. Hash is optional. Implement them against these C++ signatures (all available via `<villagesql/vsql.h>`):

## Fixed-Length Types

```cpp theme={null}
// Encode: string -> binary. Write the encoded bytes via out.buffer() and
// out.set_length(n). out.set_null(), out.warning(msg), out.error(msg), or no
// call at all reports the value as invalid: ERROR 1366 in strict mode, a
// warning under IGNORE. msg is discarded.
using TypeEncodeFunc = void (*)(std::string_view from, vsql::CustomResult out);

// Decode: binary -> string. Report the result by calling out.set_length(n)
// or out.set(sv). out.set_null(), out.warning(msg), out.error(msg), or no call
// at all fails the read with ERROR 3219; msg goes to the server log, and the
// default message is "failed to decode value".
using TypeDecodeFunc = void (*)(vsql::CustomArg in, vsql::StringResult out);

// Compare: returns -1, 0, or 1 (used for ORDER BY and indexes).
using TypeCompareFunc = int (*)(vsql::CustomArg a, vsql::CustomArg b);

// Hash: returns hash code (used for GROUP BY, DISTINCT and UNION).
using TypeHashFunc = size_t (*)(vsql::CustomArg in);
```

> [!NOTE]
>
> **Changed in 0.0.8-dev.** In an encode function, `out.set_null()` stores SQL `NULL`, and `out.error(msg)` with a non-empty message aborts the statement with that message, whatever the SQL mode. `out.warning(msg)`, or no call, still reports the value as invalid.

Omitting a custom hash function falls back to hashing the type's raw stored
bytes. This is unsafe for a type whose encode function can produce more than
one binary representation of an equal logical value — the common case is a
floating-point field, where `-0.0` and `+0.0` compare equal but differ
bit-for-bit, so `GROUP BY` and `SELECT DISTINCT` would treat them as different.

Register these operations using `vsql::make_type<kTypeName>()`. The type name
is passed as a non-type template parameter (NTTP) — a `static constexpr const char[]`
array. The builder auto-generates VDF names in the `TYPE::method` format
(e.g., `"MYTYPE::from_string"`) from this NTTP, so no manual string matching
is required. Pass the built type object to `.type()` on the extension builder;
separate `.func()` calls for type operations are not needed.

> [!WARNING]
>
> The type name must be a `static constexpr const char[]` variable — a string literal cannot be used as a non-type template parameter. Passing `"MYTYPE"` directly produces a compiler error like:
>
> ```
> error: '"MYTYPE"' is not a valid template argument for type 'const char*'
> ```
>
> Declare the name as a named array as shown below.

```cpp theme={null}
#include <villagesql/vsql.h>

using namespace vsql;

static constexpr const char kMyTypeName[] = "MYTYPE";

constexpr auto MYTYPE =
    vsql::make_type<kMyTypeName>()
        .persisted_length(8)
        .max_decode_buffer_length(64)
        .from_string<&my_encode>()   // auto: "MYTYPE::from_string"
        .to_string<&my_decode>()     // auto: "MYTYPE::to_string"
        .compare<&my_compare>()      // auto: "MYTYPE::compare"
        .hash<&my_hash>()            // optional, auto: "MYTYPE::hash"; set it for types that store floating-point values
        .intrinsic_default_str("0")  // string-literal intrinsic default
        .build();

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .type(MYTYPE))
```

`build()` fails at compile time if `from_string`, `to_string`, or `compare`
is missing. Each template method checks the function pointer signature via
`static_assert`.

A custom type may declare at most **65532 bytes** of storage per value. A custom
column is backed by a `VARBINARY` field, and MySQL's 65535-byte per-row budget
also pays for that field's two length bytes and the row's null byte; because
whether a column is nullable is decided per column at `CREATE TABLE`, and
nullable is the default, the cap has to leave room for the null byte. Both
`persisted_length` and `max_persisted_length` are checked against it, and the
check runs at `INSTALL EXTENSION` rather than at `CREATE TABLE`, so a type that
is too wide is rejected with the whole extension:

```text theme={null}
ERROR 3219 (HY000): Failed to install extension 'vsql_huge_fixed_test': type 'HUGE_FIELD' declares 70000 bytes of storage per value, which exceeds the maximum of 65532 bytes supported for custom-type columns today
```

Staying under the cap does not guarantee a column fits. Every other column in
the table draws on the same 65535-byte row budget, which only `CREATE TABLE` can
weigh, so a type at or near the cap can still fail there with
`ER_TOO_BIG_ROWSIZE`. A plain `VARBINARY` of the same width behaves the same way.

## Intrinsic Default

When a `NOT NULL` custom-type column receives `NULL` under `IGNORE` mode
(e.g., `INSERT IGNORE` or `UPDATE IGNORE`), the server calls the intrinsic
default to produce a fallback value rather than raising an error. The
intrinsic default provides a string representation; the server converts it
to binary using the type's `from_string` function.

> [!NOTE]
>
> If you omit both `.intrinsic_default_str()` and `.intrinsic_default_vdf()`,
> the server calls `from_string("")` as a fallback. This happens when the type
> is **first used** (at table creation), not at `INSTALL EXTENSION`. If your
> encode function rejects empty string, type initialization fails with an error
> visible in the SQL client:
>
> ```
> ERROR 3219 (HY000): Type 'my_ext.MYTYPE' failed to initialize: from_string VDF failed to encode intrinsic default input '' (expected persisted_length=M)
> ```
>
> If it encodes empty string to the wrong number of bytes, the error is:
>
> ```
> ERROR 3219 (HY000): Type 'my_ext.MYTYPE' failed to initialize: from_string VDF encoded intrinsic default input '' to N bytes, expected persisted_length=M
> ```
>
> For fixed-length types, the default string must encode to exactly
> `persisted_length` bytes. Set an explicit default for any type where
> empty string is not a valid input.

Provide the intrinsic default one of two ways. The two are mutually exclusive:
declaring both `.intrinsic_default_str()` and `.intrinsic_default_vdf()` is a
compile-time error with the dev ABI headers (`-DVSQL_USE_DEV_ABI=ON`); with the
stable headers the server rejects the type at `INSTALL EXTENSION`.

**String literal: `.intrinsic_default_str()`**

For a constant default, pass the string directly on the type builder (as
shown in the fixed-length example above with `.intrinsic_default_str("0")`).

**VDF-based: `.intrinsic_default_vdf()` + `make_intrinsic_default`**

When the default value depends on type parameters, implement a function
against one of these signatures (available via `<villagesql/vsql.h>`):

```cpp theme={null}
// Fixed (no type parameters):
using IntrinsicDefaultFunc = std::string (*)(char *error_msg);

// Parameterized (receives cached parsed params):
template <typename P>
using IntrinsicDefaultWithParamsFunc = std::string (*)(const P &,
                                                       char *error_msg);
```

Return a `std::string` representation of the default value. On error,
write a message to `error_msg` and return any value (the SDK checks
`error_msg[0] != '\0'` to detect errors). Register with
`make_intrinsic_default<&fn>("vdf_name")` (one argument: the VDF name) and
reference that name on the type builder with `.intrinsic_default_vdf()`.
The parameterized types example below shows the full registration pattern.

```cpp theme={null}
std::string mytype_default(const MyTypeParams &p, char * /*error_msg*/) {
  return /* build string representation based on p */;
}
```

## Parameterized Types

Parameterized types need the column's declared parameters at encode,
decode, compare, and hash time to determine allocation sizes and layout.
Define a params struct with a parse function and an inverse `to_strings`
function, register both on the type builder with
`.params<P, &ParseFunc, &ToStringsFunc>()`, and take `vsql::MaybeParams<P> &` (encode) or
`vsql::CustomArgWith<P>` (decode, compare, hash) as the first argument of your type operation functions. The SDK caches the parse result
per unique parameter combination, so the parse function runs at most once
per type instantiation. The `to_strings` function is the inverse of `parse`:
it writes a typed `P` back into the canonical key/value string form so the
server can publish inferred params in the same shape `parse` consumes.

The parse cache is keyed on the params struct itself: `TypeParamsCache<P>` is
one instance per `P`, shared by every custom type declared with that `P`. If
more than one of your types uses the same params struct, each of them must
register the same parse function; a type that needs to parse differently needs
its own params struct.

```cpp theme={null}
struct MyTypeParams {
  int64_t dimension;
  static MyTypeParams parse(const std::map<std::string, std::string> &p) {
    return {.dimension = stoll(p.at("dimension"))};
  }
  static void to_strings(const MyTypeParams &p,
                         std::map<std::string, std::string> &out) {
    out["dimension"] = std::to_string(p.dimension);
  }
};

void mytype_encode(vsql::MaybeParams<MyTypeParams> &params,
                   std::string_view from, vsql::CustomResult out) {
  const MyTypeParams &p = params.value();  // is_known() is false when the server infers the params from a constant string
  size_t bytes = (size_t)p.dimension * 4;
  auto buf = out.buffer();
  if (buf.size() < bytes) { out.error("MYTYPE: buffer too small"); return; }
  // ... parse from, write to buf ...
  out.set_length(bytes);
}

void mytype_decode(vsql::CustomArgWith<MyTypeParams> in,
                   vsql::StringResult out) {
  const MyTypeParams &p = in.params();
  // ... read p.dimension floats from in.value(), write to out.buffer() ...
  out.set_length(bytes_written);
}

int mytype_compare(vsql::CustomArgWith<MyTypeParams> a,
                   vsql::CustomArgWith<MyTypeParams> b) {
  // Returns -1, 0, or 1.
}

size_t mytype_hash(vsql::CustomArgWith<MyTypeParams> in) {
  // Returns hash code.
}

// Converts MYTYPE(N) integer syntax to a parameter map.
// Signature: IntToTypeParamsFunc from <villagesql/vsql.h>.
bool mytype_int_to_params_fn(int64_t value,
                             std::map<std::string, std::string> &params,
                             char *error_msg) {
  if (value <= 0) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: dimension must be a positive integer");
    return true;
  }
  params["dimension"] = std::to_string(value);
  return false;  // success
}

// Validates parameters and computes storage sizes.
// Signature: ResolveTypeParamsFunc from <villagesql/vsql.h>.
bool mytype_resolve_params_fn(const std::map<std::string, std::string> &params,
                              vsql::ResolvedTypeParams *result,
                              char *error_msg) {
  auto it = params.find("dimension");
  if (it == params.end()) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN, "MYTYPE: dimension parameter is required");
    return true;
  }
  char *endptr = nullptr;
  int64_t dim = std::strtoll(it->second.c_str(), &endptr, 10);
  if (endptr == it->second.c_str() || *endptr != '\0') {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: invalid dimension value '%s'", it->second.c_str());
    return true;
  }
  if (dim <= 0) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: dimension must be a positive integer");
    return true;
  }
  if (dim > INT64_MAX / 4) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: dimension %" PRId64 " is too large", dim);
    return true;
  }
  result->persisted_length = dim * 4;
  result->max_decode_buffer_length = 64;
  return false;  // success
}
```

Register `.params<>()` on the type builder. Use `.int_to_params<&mytype_int_to_params_fn>()`
to handle `MYTYPE(N)` integer syntax and `.resolve_params<&mytype_resolve_params_fn>()` to
validate parameters and compute storage sizes. Call `.max_persisted_length(N)` with an
upper bound on the persisted byte size across all valid parameterizations; the server
uses it to size the encode buffer on the type parameter inference path, where it has not yet inferred
the params and so cannot consult `resolve_params`, and rejects a resolved fixed `persisted_length` above it.
For a VDF-based intrinsic default, use `.intrinsic_default_vdf()` with the VDF name and
register the VDF separately via `make_intrinsic_default<&mytype_default>("vdf_name")`.

`build()` requires `.resolve_params<>()` whenever `.params<>()` or `.int_to_params<>()` is
registered when you build against the dev ABI headers (`-DVSQL_USE_DEV_ABI=ON`): `int_to_params` only rewrites the
`MYTYPE(N)` shorthand into a parameter string, while `resolve_params` is the
gate that validates the parameters and resolves storage size. Omitting it is a
compile-time error there; with the stable headers the server rejects the type at `INSTALL EXTENSION`.

> [!WARNING]
>
> `.max_persisted_length()` requires VEF Protocol 3 or higher. A type using it
> cannot be loaded by pre-Protocol-3 servers. It is valid only for variable-length
> or parameterized types; a fixed non-parameterized type that sets it fails to
> compile with the dev ABI headers (`-DVSQL_USE_DEV_ABI=ON`), and the server rejects
> it at `INSTALL EXTENSION` when built with the stable headers.

The sizes `resolve_params` reports are also validated at `CREATE TABLE` time,
once a column's parameters are known. For a fixed-length type, the resolved
`persisted_length` must not exceed the declared `max_persisted_length`: a
parameterization whose computed footprint (for example a `dimension * 4` sizing)
overruns that bound is rejected with a syntax error. Independently of length
kind, `resolve_params` must resolve a `max_decode_buffer_length` greater than
zero, since that value sizes the decode scratch buffer; a non-positive result is
rejected with the same class of error.

```cpp theme={null}
static constexpr const char kMyTypeName[] = "MYTYPE";

// Maximum valid dimension for MYTYPE.
constexpr int64_t kMyTypeMaxDimension = 1024;  // your max valid dimension
// Upper bound on MYTYPE's persisted byte size across all valid params.
constexpr int64_t kMyTypeMaxPersistedLength = kMyTypeMaxDimension * 4;

constexpr auto MYTYPE =
    vsql::make_type<kMyTypeName>()
        .persisted_length(-1)    // fixed length, sized per column by resolve_params
        .max_decode_buffer_length(16)
        .max_persisted_length(kMyTypeMaxPersistedLength)
        .params<MyTypeParams, &MyTypeParams::parse, &MyTypeParams::to_strings>()
        .int_to_params<&mytype_int_to_params_fn>()
        .resolve_params<&mytype_resolve_params_fn>()
        .from_string<&mytype_encode>()
        .to_string<&mytype_decode>()
        .compare<&mytype_compare>()
        .intrinsic_default_vdf("mytype_intrinsic_default")
        .build();

using namespace vsql;

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .type(MYTYPE)
        .func(make_intrinsic_default<&mytype_default>(
            "mytype_intrinsic_default")))
```

The parameterized variants — `TypeEncodeWithParamsFunc<P>`,
`TypeDecodeWithParamsFunc<P>`, `TypeCompareWithParamsFunc<P>`, and
`TypeHashWithParamsFunc<P>` — together with `ParamsToStringsFunc<P>`
(`void fn(const P&, std::map<std::string,std::string>&)`) are available via
`<villagesql/vsql.h>`.
The `vsql::make_type` template methods detect the params argument and route
through the params cache automatically. Encode functions take
`vsql::MaybeParams<P> &` as the first argument; `is_known()` is false when the server infers
the params from a constant string, and `value()` returns `const P&` once they are known. Decode, compare, and hash
variants take `vsql::CustomArgWith<P>`, whose `params()` accessor returns
`const P&`.

**Supplying parameters in SQL.** Two syntaxes reach `resolve_params`:

* **Integer** — `MYTYPE(N)`. The server routes `N` through `int_to_params` to
  build the parameter map. Requires `.int_to_params<>()`.
* **String** — `MYTYPE('key=value,...')`. The server normalizes the string and
  calls `resolve_params` directly; `int_to_params` is not involved. Available
  whenever `.resolve_params<>()` is registered — no extra builder call.

A type that registers only `.resolve_params<>()` accepts the string form and
rejects `MYTYPE(N)`.

`SHOW CREATE TABLE` does not repeat what the author wrote. The server keeps the
resolved parameter map and rebuilds a display form from it, preferring
`MYTYPE(N)` when `int_to_params(N)` reproduces the whole map and falling back to
the canonical `MYTYPE('key=value,...')` with the keys sorted. A column written as
`MYTYPE('dimension=8')` therefore reads back as `MYTYPE(8)`.

```sql theme={null}
CREATE TABLE t (v ext.MYTYPE(8));              -- integer form (.int_to_params)
CREATE TABLE t2 (v ext.MYTYPE('dimension=8')); -- string form (.resolve_params only)
```

> [!NOTE]
>
> The serialized `key=value,...` parameter string that `int_to_params` produces
> and `resolve_params` consumes is capped at `VEF_MAX_TYPE_PARAMS_STRING_LEN`
> (1024 bytes). A parameterization whose canonical string would exceed that limit
> is rejected with a defined error rather than truncated silently — keep the
> combined parameter names and values for a single type within 1024 bytes.

Whichever syntax supplies the parameters, the server validates the resulting
map before calling `resolve_params`: every key must be non-empty, appear only
once, and have a non-empty value. A violation is a syntax error naming the
type. Except for an empty parameter name, it also names the offending key:

```sql theme={null}
-- vsql_tvector ships in the villagesql/server Docker image; with a tarball install, build it from villagesql/examples/vsql-tvector in the server source
INSTALL EXTENSION vsql_tvector;
CREATE TABLE t3 (v vsql_tvector.TVECTOR('dimension=3,dimension=4'));
-- ERROR 1064 (42000) at line 1: Type 'vsql_tvector.TVECTOR': duplicate parameter 'dimension' near 'vsql_tvector.TVECTOR('dimension=3,dimension=4'))' at line 1
```

> [!NOTE]
>
> **Changed in 0.0.8-dev.** Type parameter text is converted from the connection character set to UTF-8, and text that is not valid UTF-8 or that contains a NUL byte is rejected. Duplicate parameter names are compared without regard to case or accents, so `café` and `cafe` are the same name.

### Rewriting parameters and supplying defaults

`resolve_params` has a second, mutating overload, declared only in the opt-in dev ABI
headers (`-DVSQL_USE_DEV_ABI=ON`): it takes the parameter map by non-const reference so
the type can rewrite it — typically to fill defaults the author omitted. Register it the same
way (`.resolve_params<&fn>()` accepts either form; register only one):

```cpp theme={null}
bool mytype_resolve_params_fn(std::map<std::string, std::string> &params,
                              vsql::ResolvedTypeParams *result, char *error_msg) {
  if (params.find("dimension") == params.end())
    params["dimension"] = "128";                 // supply a default
  const std::string &dim_str = params.at("dimension");
  char *endptr = nullptr;
  int64_t dim = std::strtoll(dim_str.c_str(), &endptr, 10);
  if (endptr == dim_str.c_str() || *endptr != '\0') {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: invalid dimension value '%s'", dim_str.c_str());
    return true;
  }
  if (dim <= 0) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: dimension must be a positive integer");
    return true;
  }
  if (dim > INT64_MAX / 4) {
    snprintf(error_msg, VEF_MAX_ERROR_LEN,
             "MYTYPE: dimension %" PRId64 " is too large", dim);
    return true;
  }
  result->persisted_length = dim * 4;
  result->max_decode_buffer_length = 64;
  return false;                                  // success
}
```

A mutating `resolve_params` callback may add or change parameters, but
removing one the caller wrote is an error: `Type '<extension>.<type>':
resolve_params removed parameter '<key>'; parameters may be added or changed
but not removed`.

The rewritten map becomes the canonical parameter string the server persists and
`SHOW CREATE TABLE` prints, so the rewrite must be idempotent. A **bare**
declaration (`MYTYPE`, no length or params) calls `resolve_params` with an empty
map when the type registers no `.int_to_params<>()`, so a type that supplies defaults gives every column explicit params —
`vsql_bitfield_test`'s `BITFIELD` resolves a bare column to
`max_number_of_bits=4096`:

```sql theme={null}
-- vsql_bitfield_test is built from villagesql/test-extensions/vsql-bitfield-test in the server source; the release does not ship it
INSTALL EXTENSION vsql_bitfield_test;
CREATE TABLE bits (id INT PRIMARY KEY, b vsql_bitfield_test.BITFIELD);
SHOW CREATE TABLE bits;   -- b persists as BITFIELD('max_number_of_bits=4096')
```

## Variable-Length Types

A variable-length custom type decides its persisted size per value rather than
using a single fixed footprint. Declare one by calling `.variable_length_type()`
on the type builder, which sets the type's `variable_length` flag.

> [!WARNING]
>
> `.variable_length_type()` raises the type's required protocol to VEF Protocol 4.
> The server reads the `variable_length` flag only at protocol 4 or higher. Build
> against the opt-in dev ABI headers (`-DVSQL_USE_DEV_ABI=ON`); older servers do
> not read the flag.

Variable-length types must also call `.max_persisted_length(N)`. `build()` fails
at compile time if it is omitted — the server needs the upper bound to allocate
a buffer for the backing field. A variable-length type must not also call
`.persisted_length()`: its size is decided per value, so `build()` rejects the
combination at compile time.

`.variable_length_type()` is monotonic: calling it before or after the Protocol 3
setters (`max_persisted_length()`, `params()`, `int_to_params()`) does not lower
the protocol requirement back below Protocol 4.

```cpp theme={null}
#include <villagesql/vsql.h>
using namespace vsql;

static constexpr const char kMyTypeName[] = "MYTYPE";
constexpr int64_t kMyTypeMaxPersistedLength = 4096;

constexpr auto MYTYPE =
    vsql::make_type<kMyTypeName>()
        .variable_length_type()  // per-value sizing; requires Protocol 4
        .max_persisted_length(kMyTypeMaxPersistedLength)
        .max_decode_buffer_length(64)
        .from_string<&my_encode>()
        .to_string<&my_decode>()
        .compare<&my_compare>()
        .build();

VEF_GENERATE_ENTRY_POINTS(
    make_extension()
        .type(MYTYPE))
```

> [!NOTE]
>
> Like every custom type, a variable-length type must produce a usable
> [intrinsic default](#intrinsic-default). The default is encoded into the
> field's maximum capacity, and any non-empty result of 1 to
> `max_persisted_length` bytes is accepted. A type whose empty-string encode
> produces **zero** bytes — an empty array or bit set, say — has no usable
> default, so declare an explicit one that encodes to a non-empty value:
>
> ```cpp theme={null}
>         .max_persisted_length(kMyTypeMaxPersistedLength)
>         .intrinsic_default_str("[0]")  // empty "[]" would encode to zero bytes
> ```
>
> Otherwise the type fails to initialize the first time a column
> references it, at `CREATE TABLE` — the same as a fixed-length type that cannot
> encode `from_string("")`.

## Custom Types in Stored Procedures

Custom extension types can be used as stored procedure parameter types and
in `DECLARE` variable declarations. The server resolves the custom type at
routine execution time using the installed extension's type metadata.

```sql theme={null}
DELIMITER //
CREATE PROCEDURE insert_complex(IN val COMPLEX)
BEGIN
  DECLARE tmp COMPLEX;
  SET tmp = val;
  INSERT INTO t1 VALUES (tmp);
END //
DELIMITER ;
```

## See Also

* [Custom Types in C++](https://villagesql.com/docs/en/sdk/cpp/custom-types.md) — tutorial introduction to custom types
* [C++ API Reference](https://villagesql.com/docs/en/sdk/cpp/extension-api-reference.md) — VDF contracts, null handling, and buffer sizing
* [C++ Development](https://villagesql.com/docs/en/sdk/cpp/development.md) — VDF authoring depth, argument and result types, aggregates, varargs


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.