GRDB with custom SQLite build -> How to actually load an extension?

[SwiftedMind]

Update (April 26th 2025)

I think I have found a way to load extensions. For a fully working example, see my reply below .

This approach (and I believe any approach that uses SQLite extensions) does require some small changes to be made to the GRDBCustom project in GRDB. If that's possible and doesn't break anything, it would be awesome, I think.

Original Post

Hey,

I've been trying to set up GRDB with a custom SQLite build so that I can load an SQLite extension that enables efficient vector search: https://github.com/asg017/sqlite-vec

I have found several threads trying to do this or a similar thing, but none actually concluded anything, as far as I can see. I'm not sure this is because it is still unclear how to do this, or if it is clear and I'm just not capable of understanding it 😅.

I'd love to get it working and provide clear instructions for others to reproduce because I think it would be awesome to have support for vector search directly in GRDB!

What I've Done So Far

I started a new project and followed the instructions in the "Custom SQLite Builds" guide, which worked well.

However, I did need to come up with another solution to make it possible for a local Swift package (living inside my app repo) to also access GRDB. So I created a script that compiles GRDB with the custom SQLite build into a .xcframework which I can then add to the package as binary target. That also worked! I can build and use GRDB in my Swift package.

I have uploaded the example project, including a detailed list with all the steps I took to get it to work:
https://github.com/SwiftedMind/GRDBCustomExample

What's Missing

Where I am completely lost now is adding the sqlite-vec SQLite extension. It's just two files: sqlite-vec.c and sqlite-vec.h but I have no idea how to include it in the SQLite build and then initialize it.

I've tried adding it to the amalgamation, by modifying the buildAmalgamation.sh file:

EXT_DIR="${PROJECT_DIR}/sqlite-vec"
if [ -f "${EXT_DIR}/sqlite-vec.c" ]; then
  echo " Merging sqlite-vec into sqlite3.c…"
  cat "${EXT_DIR}/sqlite-vec.c" >> sqlite3.c
fi

That does compile, but I have no way of loading the extension. This code

sqlite3_auto_extension(sqlite3_vec_init)

just tells me "Cannot find 'sqlite3_vec_init' in scope". Unfortunately, I have zero experience with Swift and C interactions and how to expose stuff to Swift, when it's part of this SQLite amalgamation.

---

I'd super appreciate any help with this. I've been trying for the past 3 days to find a way to load the extension, but nothing works and I think I'm lacking a lot of fundamental understanding. I'm basically poking around in the dark.

I'm not entirely sure if this is out-of-scope for GRDB (maybe it's more of a topic for SQLiteLib?) but I do think it would be amazing to have this support in GRDB, even if it involves some custom work.

---

Previous Related Threads

• Can I load an SQLite extension in GRDB?
• Support for removing compilation options
• DRAFT: Embed SQLiteLb Build With Loading Extensions Enabled

[SwiftedMind]

Today, I've made some amount of progress. I've managed to load the extension, create a virtual table using it, insert some vectors and then search for them via the special MATCH syntax. I have written down how I managed to get it to work below.

However, this currently only works by modifying the GRDBCustom project and adding some files to the GRDBCustom target, which is not ideal because it can cause merge conflicts whenever I try to update GRDB.

I'd appreciate anyone with more experience and knowledge about C and how to bundle these extensions into GRDB without having to touch the GRDBCustom project directly. I'm unsure how to do that. Xcode kind of needs to know about these files to build everything.

---

Steps to Load sqlite-vec Extension

You can also check out the example project using the "working-extension" branch: https://github.com/SwiftedMind/GRDBCustomExample/tree/working-extension

Prepare Extension Files

Download the amalgamation of the sqlite-Vec SQLite extension by going to the Releases and download sqlite-vec-x.x.x-alpha.2-amalgamation.zip from the Assets of the latest release. The file contains two files: sqlite-vec.c and sqlite-vec.h. Copy both into the "GRDB/SQLiteCustom" directory (that's not the "GRDBCustomSQLite" directory that contains the configuration files from the GRDB Guide).

Prepare Initialization Code

Also in the "GRDB/SQLiteCustom" directory, create two files:

// File: GRDB/SQLiteCustom/extinit.c

#define SQLITE_CORE 1

#include "extinit.h"
#include "sqlite3.h"
#include "sqlite-vec.h"

int core_vec_init(void) {
  return sqlite3_auto_extension((void *)sqlite3_vec_init);
}

// File: GRDB/SQLiteCustom/extinit.h

#ifndef EXTINIT_H
#define EXTINIT_H

#include <stdio.h>
#include "sqlite3.h"

#ifdef __cplusplus
extern "C" {
#endif

int core_vec_init(void);

#ifdef __cplusplus
}  /* end of the 'extern "C"' block */
#endif

#endif /* extinit_h */

I've taken those files from the SQLiteVec Swift package. I can't entirely explain everything that's going on there with the imports and stuff, but it essentially exposes a core_vec_init function that we can call in our app project to load the extension.

Add Files to GRDBCustom target

Open GRDBCustom.xcodeproj and drag the 4 files we have added into the project, into the "GRDBCustomSQLite" group. Choose "Reference files in place" as action and select "GRDBCustom" as target.

Then, we also need to drag in the sqlite3ext.h file into the same group, also as reference and added to the "GRDBCustom" target. You can find that file in "GRDB/SQLiteCustom/src/sqlite/src". This is needed for the sqlite-vec extension, so its header must be visible in the GRDBCustom project.

Next, open the project settings and go to the "Build Phases" of the "GRDBCustom" target. There should be a "Header" phase. Open that and drag extinit.h into the "Public" section of the Headers.

Finally, open GRDB.h and add the following line:

#import "extinit.h"

I don't understand exactly why/how, but this will expose the core_vec_init function from extinit.c to the application using GRDB.

Build the Binary

Run the script mentioned in the Readme of my example project to build the .xcframework.

Load the Extension

In the Swift Package that has the GRDB framework as binary target, you can now call core_vec_init() wherever GRDB is imported. Place that call before initializing the database, for example:

import Foundation
import GRDB

extension AppDatabase {
  /// The database for the application
  static let shared = makeShared()

  private static func makeShared() -> AppDatabase {
    do {

      // Load the sqlite-vec extension 
      core_vec_init()

      let fileManager = FileManager.default
      let appSupportURL = try fileManager.url(
        for: .applicationSupportDirectory, in: .userDomainMask,
        appropriateFor: nil, create: true
      )
      let directoryURL = appSupportURL.appendingPathComponent("Database", isDirectory: true)

      
      // Create the database folder if needed
      try fileManager.createDirectory(at: directoryURL, withIntermediateDirectories: true)

      // Open or create the database
      let databaseURL = directoryURL.appendingPathComponent("db.sqlite")
      let dbPool = try DatabasePool(
        path: databaseURL.path,
        // Use default AppDatabase configuration
        configuration: AppDatabase.makeConfiguration()
      )

      // Create the AppDatabase
      let appDatabase = try AppDatabase(dbPool)

      return appDatabase
    } catch {
      fatalError("Critical error: \(error)")
    }
  }
}

Now, you can start using it:

// Run migration to create the table
try db.execute(
  sql: """
    create virtual table vec_examples using vec0(
    sample_embedding float[8]
    );
    """
)

// Insert data
try db.execute(
  sql: """
    insert into vec_examples(rowid, sample_embedding)
    values
    (1, '[-0.200, 0.250, 0.341, -0.211, 0.645, 0.935, -0.316, -0.924]'),
    (2, '[0.443, -0.501, 0.355, -0.771, 0.707, -0.708, -0.185, 0.362]'),
    (3, '[0.716, -0.927, 0.134, 0.052, -0.669, 0.793, -0.634, -0.162]'),
    (4, '[-0.710, 0.330, 0.656, 0.041, -0.990, 0.726, 0.385, -0.958]');
    """
)

// Query Data
let result = try Row.fetchAll(db, sql: """
  select rowid, distance from vec_examples
    where sample_embedding match '[0.890, 0.544, 0.825, 0.961, 0.358, 0.0196, 0.521, 0.175]'
    order by distance
    limit 2;
  """)

print(result)

[SwiftedMind]

I have been thinking about ways to load the extension without having to manually open and modify the GRDBCustom project. It might be possible doing that through a simple Swift package target in the app (so completely outside of GRDBCustom). The SQLiteVec package does this: https://github.com/jkrukowski/SQLiteVec/tree/main/Sources/CSQLiteVec

The only problem is that this package includes its own implementation of sqlite3.c, so the imports for the extension file work fine. To make it work with GRDB, we somehow need to import sqlite3.h from GRDB, but it always defaults to the iOS version of sqlite. If there is a way to redirect that import, then loading sqlite extensions could potentially become very easy!

---

What We Need

• GRDBCustom must expose sqlite3ext.h as part of its public headers/module (maybe it's okay to simply always do that by default)
• In our app, we need a way to import sqlite3.h and sqlite3ext.h from GRDB. Simply writing #include "sqlite3.h" will cause it to use the iOS default and then not work.

Exposing sqlite3ext.h

When I try to add sqlite3ext.h to the public headers (as well as adding it to the GRDB.h file to re-export it), GRDBCustom stops compiling because grdb_config.h has an error:

Error: Use of undeclared identifier 'sqlite3_api'

That error goes away when I add the SQLITE_EXTENSION_INIT1 macro in either GRDB.h or grdb_config.h. But if I do that, I get an error in my sqlite-vec.c file, where it calls that macro itself:

Error: Redefinition of 'sqlite3_api'

And if I remove it, I get errors like:

Error: Declaration of 'sqlite3_api' must be imported from module 'GRDB' before it is required

But I have no idea how to "import" that macro call or whatever.

Importing the correct SQLite build

If I replace all imports in all the Swift package C files like this:

#import "sqlite3.h" // Before
#import <GRDB/sqlite3.h> // After

#import "sqlite3ext.h" // Before
#import <GRDB/sqlite3ext.h> // After

Then it seems to find the correct SQLite build! Not ideal, because it means I have to modify the extension files, but that is more doable (via script, for example) than modifying Xcode project files, I think.

[SwiftedMind]

I think I have found a way to make this work. It still requires to make a small change in the GRDBCustom project to expose sqlite3ext.h but maybe that can be incorporated directly into GRDB if it doesn't hurt anything.

To keep the "checkpoints" alive in the thread above, I have created a new repository with a fully working set up that uses the sqlite-vec SQLite extension:

https://github.com/SwiftedMind/GRDBCustomSQLiteBuild

The readme there contains a full step-by-step guide on how to set this up in any project. I hope my approach doesn't break anything, I'm not very experienced with C and stuff. But in my app, everything seems to be working.

I'd love to hear if this would be possible to change in GRDB!

---

What needs to change in the GRDBCustom project

1. To support loading extensions, the GRDBCustom project must expose the sqlite3ext.h file.

   • Open the GRDBCustom project.

   • Drag the sqlite3ext.h file from GRDB/SQLiteCustom/src/sqlite/src into the project as file reference into the project, where sqlite3.h is already references (in the GRDBCustomSQLite group). Add it to the GRDBCustom target.

   • Go to the project settings, choose the GRDBCustom target and go to the "Build Phases" tab. There is a section "Headers". Drag the sqlite3ext.h file from the project navigator to the "Public" headers list.

   • Got to GRDB.h and replace the contents with this:

     #ifdef __OBJC__
     @import Foundation;
      
     //! Project version number for GRDB.
     FOUNDATION_EXPORT double GRDB_VersionNumber;
      
     //! Project version string for GRDB.
     FOUNDATION_EXPORT const unsigned char GRDB_VersionString[];
     #endif
      
     #ifndef SQLITE_CORE
     #define SQLITE_CORE 1
     #define _GRDB_UNDEF_SQLITE_CORE
     #endif
      
     #import <GRDB/GRDBCustomSQLite-USER.h>
     #import <GRDB/sqlite3.h>
     #import <GRDB/sqlite3ext.h>
     #import "GRDB-Bridging.h"
      
     #ifdef _GRDB_UNDEF_SQLITE_CORE
     #undef SQLITE_CORE
     #undef _GRDB_UNDEF_SQLITE_CORE
     #endif

     This adds the sqlite3ext.h to the umbrella header for the GRDB module. It also makes sure that the Objective-C code in the file is not used when this is imported from a pure C SQlite extension. Finally, it temporarily defines SQLITE_CORE to declare the imports. If this is not present, then grdb-config.h will fail to compile because the sqlite3ext.h messes with the sqlite3_api it sees. This should not affect SQLite at all, it should makes it possible to expose the sqlite3ext.h header.