Skip to content

Commit

Permalink
Feat: seperate implementation from interface (#17)
Browse files Browse the repository at this point in the history
* feat: seperate interface from implementation, cplusplus compat

* feat: extism_log_sz
  • Loading branch information
G4Vi committed Dec 18, 2023
1 parent 5add50e commit 01064c0
Show file tree
Hide file tree
Showing 15 changed files with 156 additions and 41 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
*.wasm
*.o
wasi-sdk
25 changes: 23 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ This project contains a tool that can be used to create [Extism Plug-ins](https:

## Installation

Since the Extism host provides these functions, all that's required to build a plugin with the Extism C PDK is the [extism-pdk.h](https://github.com/extism/c-pdk/blob/main/extism-pdk.h) header file. This can be copied into
your project, or you can add the repo as a Git submodule:
The Extism C PDK is a single header library. Just copy [extism-pdk.h](https://github.com/extism/c-pdk/blob/main/extism-pdk.h) into your project or add this repo as a Git submodule:

```shell
git submodule add https://github.com/extism/c-pdk extism-pdk
Expand All @@ -21,6 +20,7 @@ The first thing you should understand is creating an export.
Let's write a simple program that exports a `greet` function which will take a name as a string and return a greeting string. Paste this into a file `plugin.c`:

```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdint.h>

Expand Down Expand Up @@ -79,6 +79,7 @@ extism call plugin.wasm greet --input="Benjamin"
We catch any exceptions thrown and return them as errors to the host. Suppose we want to re-write our greeting module to never greet Benjamins:

```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdbool.h>
#include <stdint.h>
Expand Down Expand Up @@ -145,6 +146,7 @@ Configs are key-value pairs that can be passed in by the host when creating a
plug-in. These can be useful to statically configure the plug-in with some data that exists across every function call. Here is a trivial example using `extism_config_get`:

```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdint.h>
#include <stdlib.h>
Expand Down Expand Up @@ -204,6 +206,7 @@ host has loaded and not freed the plug-in.
You can use `extism_var_get`, and `extism_var_set` to manipulate vars:

```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdint.h>

Expand Down Expand Up @@ -242,6 +245,7 @@ int32_t EXTISM_EXPORTED_FUNCTION(count) {
The `extism_log*` functions can be used to emit logs:
```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdint.h>
Expand Down Expand Up @@ -273,6 +277,7 @@ extism call plugin.wasm log_stuff --log-level=info
HTTP calls can be made using `extism_http_request`:

```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdint.h>
#include <string.h>
Expand Down Expand Up @@ -327,6 +332,7 @@ IMPORT("my_module", "a_python_func") extern ExtismPointer a_python_func(ExtismPo
To call this function, we pass an Extism pointer and receive one back:
```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
#include <stdint.h>
Expand Down Expand Up @@ -388,6 +394,21 @@ python3 app.py
# => An argument to send to Python!
```

## Building

One source file must contain the implementation:

```c
#define EXTISM_IMPLEMENTATION
#include "extism-pdk.h"
```

All other source files using the pdk must include the header without `#define EXTISM_IMPLEMENTATION`

The C PDK does not require building with `libc`, but additional functions can be enabled when `libc` is available. `#define EXTISM_USE_LIBC` in each file before including the pdk (everywhere it is included) or, when compiling, pass it as a flag to clang: `-D EXTISM_USE_LIBC`

The C PDK may be used from C++, however, the implementation must be built with a C compiler. See `cplusplus` in `tests/Makefile` for an example.

## Exports (details)

The `EXTISM_EXPORTED_FUNCTION` macro is not essential to create a plugin function and export it to the host. You may instead write a function and then export it when linking. For example, the first example may have the following signature instead:
Expand Down
1 change: 1 addition & 0 deletions examples/count-vowels/count-vowels.c
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#define EXTISM_IMPLEMENTATION
#include "../../extism-pdk.h"

#include <stdio.h>
Expand Down
1 change: 1 addition & 0 deletions examples/globals/globals.c
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#define EXTISM_IMPLEMENTATION
#include "../../extism-pdk.h"

#include <stdio.h>
Expand Down
3 changes: 2 additions & 1 deletion examples/host-functions/host-functions.c
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
#define EXTISM_IMPLEMENTATION
#include "../../extism-pdk.h"

#include <stdio.h>

IMPORT("extism:host/user", "hello_world")
EXTISM_IMPORT("extism:host/user", "hello_world")
extern uint64_t hello_world(uint64_t);

int32_t EXTISM_EXPORTED_FUNCTION(count_vowels) {
Expand Down
132 changes: 98 additions & 34 deletions extism-pdk.h
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
#pragma once
#ifndef extism_pdk_h
#define extism_pdk_h

#ifdef __cplusplus
extern "C" {
#endif

#include <stdbool.h>
#include <stdint.h>
Expand Down Expand Up @@ -76,9 +81,74 @@ extern void extism_log_warn(const ExtismPointer);
EXTISM_IMPORT_ENV("log_error")
extern void extism_log_error(const ExtismPointer);

// Load data from Extism memory
// Does not verify load is in bounds
static void extism_load(const ExtismPointer offs, void *dest, const size_t n) {
// Load data from Extism memory, does not verify load is in bounds
void extism_load(const ExtismPointer offs, void *dest, const size_t n);

// Load data from input buffer, verifies load is inbounds
bool extism_load_input(void *dest, const size_t n);

// Load n-1 bytes from input buffer and zero terminate
// Verifies load is inbounds
bool extism_load_input_sz(char *dest, const size_t n);

// Copy data into Extism memory
void extism_store(ExtismPointer offs, const void *buffer, const size_t length);

// Allocate a buffer in Extism memory and copy into it
ExtismPointer extism_alloc_buf(const void *src, const size_t n);

__attribute__((
deprecated("Use extism_alloc_buf instead."))) static inline ExtismPointer
extism_alloc_string(const char *s, const size_t n) {
return extism_alloc_buf(s, n);
}

#ifdef EXTISM_USE_LIBC
// get the input length (n) and malloc(n), load n bytes from Extism memory
// into it. If outSize is provided, set it to n
void *extism_load_input_dup(size_t *outSize);

// get the input length, add 1 to it to get n. malloc(n), load n - 1 bytes
// from Extism memory into it. Zero terminate. If outSize is provided, set it
// to n
void *extism_load_input_sz_dup(size_t *outSize);

#endif // EXTISM_USE_LIBC

// Allocate a buffer in Extism memory and copy string data into it
// copied string is NOT null terminated
ExtismPointer extism_alloc_buf_from_sz(const char *sz);

typedef enum {
ExtismLogInfo,
ExtismLogDebug,
ExtismLogWarn,
ExtismLogError,
} ExtismLog;

// Write to Extism log
void extism_log(const char *s, const size_t len, const ExtismLog level);

// Write zero-terminated string to Extism log
void extism_log_sz(const char *s, const ExtismLog level);

#ifdef __cplusplus
}
#endif
#endif // extism_pdk_h

// avoid greying out the implementation section
#if defined(Q_CREATOR_RUN) || defined(__INTELLISENSE__) || \
defined(_CDT_PARSER__)
#define EXTISM_IMPLEMENTATION
#endif

#ifdef EXTISM_IMPLEMENTATION
#ifndef extism_pdk_c
#define extism_pdk_c

// Load data from Extism memory, does not verify load is in bounds
void extism_load(const ExtismPointer offs, void *dest, const size_t n) {
const size_t chunk_count = n >> 3;
uint64_t *i64_buffer = dest;
for (size_t chunk_idx = 0; chunk_idx < chunk_count; chunk_idx++) {
Expand All @@ -93,8 +163,7 @@ static void extism_load(const ExtismPointer offs, void *dest, const size_t n) {
}
}

// Load data from input buffer
// Does not verify load is inbounds
// Load data from input buffer, does not verify load is inbounds
static void extism_load_input_unsafe(void *dest, const size_t n) {
const size_t chunk_count = n >> 3;
uint64_t *i64_buffer = dest;
Expand All @@ -110,9 +179,8 @@ static void extism_load_input_unsafe(void *dest, const size_t n) {
}
}

// Load data from input buffer
// Verifies load is inbounds
static bool extism_load_input(void *dest, const size_t n) {
// Load data from input buffer, verifies load is inbounds
bool extism_load_input(void *dest, const size_t n) {
const uint64_t input_len = extism_input_length();
if (n > input_len) {
return false;
Expand All @@ -130,7 +198,7 @@ static void extism_load_input_sz_unsafe(char *dest, const size_t n) {

// Load n-1 bytes from input buffer and zero terminate
// Verifies load is inbounds
static bool extism_load_input_sz(char *dest, const size_t n) {
bool extism_load_input_sz(char *dest, const size_t n) {
const uint64_t input_len = extism_input_length();
if ((n - 1) > input_len) {
return false;
Expand All @@ -140,8 +208,7 @@ static bool extism_load_input_sz(char *dest, const size_t n) {
}

// Copy data into Extism memory
static void extism_store(ExtismPointer offs, const void *buffer,
const size_t length) {
void extism_store(ExtismPointer offs, const void *buffer, const size_t length) {
const size_t chunk_count = length >> 3;
const uint64_t *i64_buffer = buffer;
for (size_t chunk_idx = 0; chunk_idx < chunk_count; chunk_idx++) {
Expand All @@ -157,27 +224,21 @@ static void extism_store(ExtismPointer offs, const void *buffer,
}

// Allocate a buffer in Extism memory and copy into it
static ExtismPointer extism_alloc_buf(const void *src, const size_t n) {
ExtismPointer extism_alloc_buf(const void *src, const size_t n) {
ExtismPointer ptr = extism_alloc(n);
extism_store(ptr, src, n);
return ptr;
}

__attribute__((
deprecated("Use extism_alloc_buf instead."))) static inline ExtismPointer
extism_alloc_string(const char *s, const size_t n) {
return extism_alloc_buf(s, n);
}

#ifdef EXTISM_USE_LIBC
#include <stdlib.h>
#include <string.h>

#define extism_strlen strlen

// get the input length (n) and malloc(n), load n bytes from Extism memory into
// it. If outSize is provided, set it to n
static void *extism_load_input_dup(size_t *outSize) {
// get the input length (n) and malloc(n), load n bytes from Extism memory
// into it. If outSize is provided, set it to n
void *extism_load_input_dup(size_t *outSize) {
const uint64_t n = extism_input_length();
if (n > SIZE_MAX) {
return NULL;
Expand All @@ -193,9 +254,10 @@ static void *extism_load_input_dup(size_t *outSize) {
return buf;
}

// get the input length, add 1 to it to get n. malloc(n), load n - 1 bytes from
// Extism memory into it. Zero terminate. If outSize is provided, set it to n
static void *extism_load_input_sz_dup(size_t *outSize) {
// get the input length, add 1 to it to get n. malloc(n), load n - 1 bytes
// from Extism memory into it. Zero terminate. If outSize is provided, set it
// to n
void *extism_load_input_sz_dup(size_t *outSize) {
uint64_t n = extism_input_length();
if (n > (SIZE_MAX - 1)) {
return NULL;
Expand Down Expand Up @@ -223,19 +285,12 @@ static size_t extism_strlen(const char *sz) {

// Allocate a buffer in Extism memory and copy string data into it
// copied string is NOT null terminated
static ExtismPointer extism_alloc_buf_from_sz(const char *sz) {
ExtismPointer extism_alloc_buf_from_sz(const char *sz) {
return extism_alloc_buf(sz, extism_strlen(sz));
}

typedef enum {
ExtismLogInfo,
ExtismLogDebug,
ExtismLogWarn,
ExtismLogError,
} ExtismLog;

// Write to Extism log
static void extism_log(const char *s, const size_t len, const ExtismLog level) {
void extism_log(const char *s, const size_t len, const ExtismLog level) {
ExtismPointer ptr = extism_alloc(len);
extism_store(ptr, s, len);
switch (level) {
Expand All @@ -254,3 +309,12 @@ static void extism_log(const char *s, const size_t len, const ExtismLog level) {
}
extism_free(ptr);
}

// Write zero-terminated string to Extism log
void extism_log_sz(const char *s, const ExtismLog level) {
const size_t len = extism_strlen(s);
extism_log(s, len, level);
}

#endif // extism_pdk_c
#endif // EXTISM_IMPLEMENTATION
9 changes: 7 additions & 2 deletions tests/Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
WASI_SDK_PATH?=../wasi-sdk

test: test-alloc test-hello test-load_input_bounds_checks test-strlen test-use_libc test-alloc_buf_from_sz
test: test-alloc test-hello test-load_input_bounds_checks test-strlen test-use_libc test-alloc_buf_from_sz test-cplusplus
run-fail: test-fail
build: alloc hello fail load_input_bounds_checks strlen use_libc alloc_buf_from_sz
build: alloc hello fail load_input_bounds_checks strlen use_libc alloc_buf_from_sz cplusplus

cplusplus:
$(WASI_SDK_PATH)/bin/clang++ --target=wasm32-wasi -c $@.cpp
$(WASI_SDK_PATH)/bin/clang --target=wasm32-wasi -c implementation.c
$(WASI_SDK_PATH)/bin/clang++ --target=wasm32-wasi -o $@.wasm $@.o implementation.o -mexec-model=reactor

strlen use_libc:
$(WASI_SDK_PATH)/bin/clang --target=wasm32-wasi -o $@.wasm $@.c -mexec-model=reactor
Expand Down
1 change: 1 addition & 0 deletions tests/alloc_buf_from_sz.c
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#define EXTISM_IMPLEMENTATION
#include "util.h"

EXTISM_EXPORT_AS("run_test") int32_t run_test(void) {
Expand Down
12 changes: 12 additions & 0 deletions tests/cplusplus.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#include "util.h"
#include <string.h>

EXTISM_EXPORT_AS("run_test") int32_t run_test(void) {
const char *msg = "Hello, world!";
ExtismPointer ptr = extism_alloc(strlen(msg));
assert(ptr > 0);
extism_store(ptr, (const uint8_t *)msg, strlen(msg));
assert(extism_length(ptr) == strlen(msg));
extism_output_set(ptr, strlen(msg));
return 0;
}
5 changes: 3 additions & 2 deletions tests/fail.c
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
#define EXTISM_IMPLEMENTATION
#include "util.h"

int32_t EXTISM_EXPORTED_FUNCTION(run_test) {
const char *msg = "Some error message";
ExtismPointer ptr = extism_alloc(strlen(msg));
extism_store(ptr, (const uint8_t *)msg, strlen(msg));
ExtismPointer ptr = extism_alloc(extism_strlen(msg));
extism_store(ptr, (const uint8_t *)msg, extism_strlen(msg));
extism_error_set(ptr);
return 1;
}
1 change: 1 addition & 0 deletions tests/hello.c
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#define EXTISM_IMPLEMENTATION
#include "util.h"

EXTISM_EXPORT_AS("run_test") int32_t run_test(void) {
Expand Down
2 changes: 2 additions & 0 deletions tests/implementation.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#define EXTISM_IMPLEMENTATION
#include "../extism-pdk.h"
1 change: 1 addition & 0 deletions tests/load_input_bounds_checks.c
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#define EXTISM_IMPLEMENTATION
#include "util.h"

EXTISM_EXPORT_AS("run_test") int32_t run_test(void) {
Expand Down
1 change: 1 addition & 0 deletions tests/strlen.c
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#define EXTISM_IMPLEMENTATION
#include "util.h"
#include <string.h>

Expand Down
2 changes: 2 additions & 0 deletions tests/use_libc.c
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#define EXTISM_USE_LIBC
#define EXTISM_IMPLEMENTATION
#include "util.h"
#include <stdlib.h>

EXTISM_EXPORT_AS("run_test") int32_t run_test(void) {
void *input = extism_load_input_dup(NULL);
Expand Down

0 comments on commit 01064c0

Please sign in to comment.