C++ Testing

This guide covers the test-and-iterate loop for a C++ extension: setting up a local server, installing .veb files, running regression tests, and debugging failures. It is the companion to Creating Extensions in C++, which covers the initial build, and C++ Development, which covers VDF authoring depth.

If you are contributing to the VillageSQL server itself (not building an extension), see Build from Source, which covers the full server developer workflow including running tests with mysql-test-run.pl directly.

Setting Up Your Environment

To develop and test extensions, you need a built VillageSQL server. Follow the Clone and Build from Source guide to compile the server binaries. Once you have a build, use the villagesql CLI to manage a local dev server instance. Run all commands from the directory where VillageSQL was installed.

Starting a Local Dev Server

Initialize and start a server instance:

./villagesql init    # initialize database and seed bundled extensions
./villagesql start   # start the server (default port 3307)
./villagesql status  # check the server is running
./villagesql connect # open a mysql shell
./villagesql stop    # stop the server

To set a root password on init:

./villagesql init --password
./villagesql start

To enable or disable mysqld features for a test run, pass flags after a -- separator; everything after it is forwarded verbatim to mysqld:

./villagesql start -- --skip-name-resolve --general-log

Any unrecognized argument before the -- is rejected, so use -- for every flag you intend for mysqld. Pass --dir <path> before any command to manage multiple independent instances, or use --here to create a server directory in the current working directory:

./villagesql --here init
./villagesql --here start

Managing Extension Files

Before installing an extension via SQL, its .veb file must be present on the server. The CLI manages the server’s lib/veb/ directory:

./villagesql veb add /path/to/my_extension.veb  # copy a .veb to the server
./villagesql veb ls                              # list available .veb files
./villagesql veb rm my_extension                # remove a .veb file

.veb files placed in lib/veb/ before init are seeded automatically. After adding a file, install the extension via SQL:

INSTALL EXTENSION my_extension;

Running Regression Tests

Run extension regression tests using the MySQL Test Runner from your VillageSQL build directory.

Running the Full Suite

To run all tests for your extension:

cd $BUILD_HOME
./mysql-test/mysql-test-run.pl --suite=/path/to/your/extension/mysql-test --parallel=auto

Running Individual Tests

To run a single test case, specify the suite path and test name:

cd $BUILD_HOME
./mysql-test/mysql-test-run.pl --suite=/path/to/your/extension/mysql-test my_test_name

Creating New Tests

When adding new features or fixing bugs, you should add corresponding regression tests.

Test Location

Extension tests live in the extension’s own repository under a mysql-test/ directory — not in the VillageSQL server’s mysql-test/suite/ tree.

Test files end with .test and go in mysql-test/t/.
Expected result files end with .result and go in mysql-test/r/.

For example, for an extension named my_extension:

mysql-test/t/my_new_test.test
mysql-test/r/my_new_test.result

Test File Conventions

A typical extension test installs the extension, runs SQL, and uninstalls:

# Description of the test

INSTALL EXTENSION my_extension;

# ... Your Test Code Here ...
CREATE TABLE t1 (val MYTYPE);
INSERT INTO t1 VALUES ('some_value');
SELECT * FROM t1;
DROP TABLE t1;

UNINSTALL EXTENSION my_extension;

When your test output includes paths from the test runner’s temp directory, add this directive inside your .test file to normalize them — without it, recorded results contain absolute paths that break on other machines:

--replace_result $MYSQLTEST_VARDIR MYSQLTEST_VARDIR

Steps to Add a Test

Create the .test file in your extension’s mysql-test/t/ directory.
Create an empty .result file in your extension’s mysql-test/r/ directory.

Run the test with --record to generate the expected output:

cd $BUILD_HOME
./mysql-test/mysql-test-run.pl --suite=/path/to/your/extension/mysql-test --record my_new_test

Verify the output in the generated .result file to ensure it matches your expectations.

Debugging Tests

If a test fails, the test framework provides detailed logs.

Test output: Check mysql-test/var/log/mysqltest.log (combined) or mysql-test/var/log/<test_name>/ (per-test directory).
Server error log: Check mysql-test/var/log/mysqld.1.err. VillageSQL-specific log messages (emitted via LogVSQL()) only appear when the server runs with --log-error-verbosity=3.
Diff: The framework outputs a diff between the actual output and the expected .result file.

To run a test with extra debug information:

cd $BUILD_HOME
./mysql-test/mysql-test-run.pl --verbose --suite=/path/to/your/extension/mysql-test my_new_test

# To surface LogVSQL() messages in the error log:
./mysql-test/mysql-test-run.pl --mysqld=--log-error-verbosity=3 \
    --suite=/path/to/your/extension/mysql-test my_new_test

Getting Started

Server Development

Extensions

Using Extensions

Creating Extensions

Setting Up Your Environment

Starting a Local Dev Server

Managing Extension Files

Running Regression Tests

Running the Full Suite

Running Individual Tests

Creating New Tests

Test Location

Test File Conventions

Steps to Add a Test

Debugging Tests

See Also

​Setting Up Your Environment

​Starting a Local Dev Server

​Managing Extension Files

​Running Regression Tests

​Running the Full Suite

​Running Individual Tests

​Creating New Tests

​Test Location

​Test File Conventions

​Steps to Add a Test

​Debugging Tests

​See Also

Setting Up Your Environment

Starting a Local Dev Server

Managing Extension Files

Running Regression Tests

Running the Full Suite

Running Individual Tests

Creating New Tests

Test Location

Test File Conventions

Steps to Add a Test

Debugging Tests

See Also