Aligning MySQL Extensions with Existing Component Services

When something goes wrong inside MySQL, the server says so in the error log. It's what administrators already watch. Every entry carries a severity and an error code, and you can query it from performance_schema.error_log like any other table. It is the server's place for communicating "something here needs your attention." This is an example of a service that MySQL provides out of the box. Since 8.0, MySQL's component framework has been able to leverage these services, and now the VillageSQL Extension Framework can too.

VillageSQL is a drop-in replacement for MySQL that lets you run your own code inside the server. The VillageSQL Extension Framework (VEF) gives you a channel to expand MySQL via portable extensions written in C++ or Rust, without waiting for that functionality to land upstream. There are quite a few extensions already, doing everything from cryptography to HTTP calls. You can use one of these or build your own in C++ or Rust.

Going back to the error-log example, until now, VEF didn't expose a way to file an error-log entry. A proper entry, with a severity and an error code that filters and queries can act on, goes through a MySQL component service, and component services were reachable only from inside MySQL's own component system. VEF’s new preview capability, MySQL Services, lets VillageSQL extensions consume MySQL's component services directly.

Why MySQL has a service registry

Taking a step back, it's important to understand why MySQL has a service registry. A database server is a complex collection of facilities that need to cooperate without being welded together. Error logging should be filterable and routable without recompiling the server. Password validation policies differ between users. Encryption keys might live in a file on one server and in a vault on another. MySQL's answer is its component infrastructure: each of these facilities is a "component," and components interact with each other through services, which are named interfaces registered in a central registry. For example, when something needs the keyring, it doesn't call keyring code. It asks the registry for the keyring service by name and gets whatever implementation is installed. Swap one keyring component for another and every consumer keeps working, assuming the name and the interface didn't change.

In MySQL's own architecture (from 8.0 onwards) the server core is just another component and the server source tree ships with over a hundred services: for example, the keyring, the error log, session information, system variables, and the rest of what MySQL's facilities use to talk to each other.

What the registry doesn't have is an external-facing client API. Internally, components and plugins consume these C/C++ services via a central registry.

VillageSQL extensions can now leverage that same set of services.

What this changes

In a post this June, we described how we build extensibility APIs in three beats: a typed, ergonomic happy path where the framework does the work; a deliberate space between, where reasonable use cases wait for a signal to guide us to build the right abstraction instead of implementing the wrong one forever; and an escape hatch, a path that unblocks you today.

MySQL services are an escape hatch in action. VEF supports a growing set of functionality, but MySQL components are an established framework already adopted by the community and there's no good reason not to be able to reuse them within the extension framework. This change opens territory the SDKs don't yet cover. A happy-path version would wrap each service in typed VEF vocabulary and smooth over any rough edges. Designing those wrappers so they feel native to a MySQL user takes time. Instead, you get the raw registry and MySQL's own headers today, and the ergonomics can come later, informed by what people actually want.

Going back to our example, an extension can now write to the server's error log, with a severity and an error code, and administrators can query those entries from performance_schema.error_log like any other server message, in the place they already watch. It can read attributes off the calling session, like which SQL command is running.

There are a variety of services in the registry. Its headers describe services for checking the calling user's security context, registering dynamic privileges of your own for DBAs to GRANT, exposing your extension's internal state as performance_schema tables, and emitting traces and metrics into the server's telemetry, among others. If a service is in the registry, your extension can now ask for it.

How you use it

You declare what services you need with the C++ SDK. The extensino framework acquires the services when your extension loads and releases them when it unloads. Following is an example extension that composes two services. The first to get the calling session's thread descriptor (THD) and one to read a named attribute off it:

#include <cstddef>

#include <mysql/components/services/defs/mysql_string_defs.h>
#include <mysql/components/services/mysql_current_thread_reader.h>
#include <mysql/components/services/mysql_thd_attributes.h>
#include <villagesql/preview/mysql_services.h>
#include <villagesql/vsql.h>

using namespace vsql;

static preview_mysql_services::MysqlServices services;
VSQL_REQUIRE_SERVICE(services, mysql_current_thread_reader, thd_reader);
VSQL_REQUIRE_SERVICE(services, mysql_thd_attributes, attrs);

// session_sql_command() -> STRING: the name of the SQL command running on
// the calling session, or NULL when the THD or attribute cannot be read.
void session_sql_command(StringResult out) {
  if (!thd_reader.valid() || !attrs.valid()) {
    out.error("MySQL session services are not available");
    return;
  }

  MYSQL_THD thd = nullptr;
  // MySQL convention: a false return means success.
  if (thd_reader->get(&thd) || thd == nullptr) {
    out.set_null();
    return;
  }

  mysql_cstring_with_length value{nullptr, 0};
  if (attrs->get(thd, "sql_command", &value) || value.str == nullptr) {
    out.set_null();
    return;
  }

  out.set(std::string_view(value.str, value.length));
}

VEF_GENERATE_ENTRY_POINTS(make_extension().with(services).func(
    make_func<&session_sql_command>("session_sql_command")
        .returns(STRING)
        .no_params()
        .build()))

SELECT session_sql_command() AS sql_command;

+-------------+
| sql_command |
+-------------+
| select      |
+-------------+

The VEF SDK's job here is deliberately small. It handles acquisition and lifetime, and gives you valid() to check that acquisition succeeded. Acquisition happens at install time, so an extension that requires a service the server doesn't have fails at INSTALL EXTENSION, with the missing service named in the error. Because VEF doesn't wrap these services, you are interacting directly with native MySQL code. The arguments, return values, and any quirky conventions are dictated entirely by MySQL, not VillageSQL. To learn how to use a specific service, you should read MySQL's own header for it, at include/mysql/components/services/NAME.h. Whichever service you need, the VEF workflow is the same three steps: require the service, check valid(), and call through to the raw MySQL implementation.

Three practical notes. First, naming: vsql::preview::mysql_services is the capability's name, the string the server sees. The C++ namespace is vsql::preview_mysql_services, the same convention every preview capability's header follows. Second, building: MySQL's service headers belong to its component framework and don't ship with the extension SDK, so an extension that consumes services builds against a VillageSQL server source tree. The preview capabilities reference covers the include paths. Third, language: this capability is C++ only today. MySQL's service definitions are C headers meant to be consumed from C++, and the Rust SDK has no binding for them yet.

What about providing services from extensions

Consuming services is half of the full design. The other half is extensions providing services - registering their own implementations into the registry for the rest of the server to use. That half isn't implemented yet, on purpose. We want to hear from you first. Providing raises a design question we don't want to answer by guessing: what happens when an extension that provides a service gets uninstalled while something still depends on it? The wrong abstraction design now means a bad API. The consumer side stands on its own and unblocks people today, so it has shipped, and the provider side waits until there's a clearer signal of a right answer. If you have a use case for providing services, let us know.

How will this evolve

Feedback from real use is what will settle this feature's shape, decide when it graduates into the stable SDK, and tell us whether we should build the provider side. So when you build against it, tell us why and how please. Thanks to MySQL community members at AWS and their investment in the component system, we expect the number of available services to grow over time.

To try it: run SET PERSIST vsql_allow_preview_extensions = ON; on your server and include <villagesql/preview/mysql_services.h> in your extension. The preview capabilities reference explains how preview capabilities work.

Get started at villagesql.com and star the GitHub repo.