ARTICLE DETAIL

资讯详情

深耕网站建设与运营推广的一线实战洞察。

首页/资讯中心/资讯详情

测试DuckDB LuaJit插件通过ffi调用c语言动态库性能

发布时间：2026/8/16 17:28:28 拓冰企业网站定制
测试DuckDB LuaJit插件通过ffi调用c语言动态库性能

关键步骤有2步。
1.用c编译器(gcc/tcc等)编译动态库

tcc -shared -fPIC -o libsudoku_c.so sudoku_solve.c

2.在DuckDB中把动态库中的函数注册为luajit udf

SELECT * FROM luajit_module(mode := 'compile', sql_name := 'sudoku_c', source := ' return function(p) local ffi = require("ffi") ffi.cdef[[ int sudoku_solve(const char *p, char *out); ]] local lib = ffi.load("/mnt/c/d/libsudoku_c.so") local buf = ffi.new("char[82]") local rc = lib.sudoku_solve(p, buf) if rc ~= 1 then return NULL end return ffi.string(buf) end ');

然后用luajit_s等函数调用udf。

在Windows中调用不成功。

Microsoft Windows [版本 10.0.26200.8655] (c) Microsoft Corporation。保留所有权利。 C:\Users\lt>cd\d C:\d>duckdb155 DuckDB v1.5.5 (Variegata) Enter ".help" for usage hints. memory D load luajit; memory D C:\d>tcc\tcc -shared -fPIC -o libsudoku_c.so sudoku_solve.c C:\d>duckdb155 DuckDB v1.5.5 (Variegata) Enter ".help" for usage hints. memory D load luajit; memory D SELECT * FROM luajit_module(mode := 'compile', sql_name := 'sudoku_c', source := ' return function(p) local ffi = require("ffi") ffi.cdef[[ int sudoku_solve(const char *p, char *out); ]] local lib = ffi.load("/d/libsudoku_c.so") local buf = ffi.new("char[82]") local rc = lib.sudoku_solve(p, buf) if rc ~= 1 then return NULL end return ffi.string(buf) end '); ┌─────────┬─────────┬─────────┬─────────┬─────────┬──────────┐ │ ok │ mode │ phase │ message │ detail │ sql_name │ │ boolean │ varchar │ varchar │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┼─────────┼─────────┼──────────┤ │ true │ compile │ compile │ OK │ │ sudoku_c │ └─────────┴─────────┴─────────┴─────────┴─────────┴──────────┘ memory D SELECT luajit_s('sudoku_c', '000001002000020030004500600007600050080090006100005800001004000070900003400030020'); ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ luajit_s('sudoku_c', '000001002000020030004500600007600050080090006100005800001004000070900003400030020') │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ NULL │ └───────────────────────────────────────────────────────────────────────────────────────────────────────────┘

在WSL Linux中调用成功。首先用apt install 安装Tcc。

C:\d>wsl root@DESKTOP-59T6U68:/mnt/c/d# tcc Command 'tcc' not found, but can be installed with: apt install tcc root@DESKTOP-59T6U68:/mnt/c/d# apt install tcc Reading package lists... Done Building dependency tree... Done Reading state information... Done The following NEW packages will be installed: tcc 0 upgraded, 1 newly installed, 0 to remove and 48 not upgraded. Need to get 278 kB of archives. After this operation, 824 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 tcc amd64 0.9.27+git20200814.62c30a4a-1 [278 kB] Fetched 278 kB in 1s (205 kB/s) Selecting previously unselected package tcc. (Reading database ... 84088 files and directories currently installed.) Preparing to unpack .../tcc_0.9.27+git20200814.62c30a4a-1_amd64.deb ... Unpacking tcc (0.9.27+git20200814.62c30a4a-1) ... Setting up tcc (0.9.27+git20200814.62c30a4a-1) ... Processing triggers for install-info (6.8-4build1) ... Processing triggers for man-db (2.10.2-1) ... root@DESKTOP-59T6U68:/mnt/c/d# tcc -shared -fPIC -o libsudoku_c2.so sudoku_solve.c root@DESKTOP-59T6U68:/mnt/c/d# ./duckdb155 DuckDB v1.5.5 (Variegata) Enter ".help" for usage hints. memory D .timer on memory D load luajit; Run Time (s): real 0.015 user 0.004387 sys 0.000000 memory D SELECT * FROM luajit_module(mode := 'compile', sql_name := 'sudoku_c', source := ' return function(p) local ffi = require("ffi") ffi.cdef[[ int sudoku_solve(const char *p, char *out); ]] local lib = ffi.load("/mnt/c/d/libsudoku_c2.so") local buf = ffi.new("char[82]") local rc = lib.sudoku_solve(p, buf) if rc ~= 1 then return NULL end return ffi.string(buf) end '); ┌─────────┬─────────┬─────────┬─────────┬─────────┬──────────┐ │ ok │ mode │ phase │ message │ detail │ sql_name │ │ boolean │ varchar │ varchar │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┼─────────┼─────────┼──────────┤ │ true │ compile │ compile │ OK │ │ sudoku_c │ └─────────┴─────────┴─────────┴─────────┴─────────┴──────────┘ Run Time (s): real 0.008 user 0.002350 sys 0.000048 memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 100); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 978654213645231798213879654796518432521347986384926175437182569159463827862795341 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.950 user 0.900484 sys 0.008561 memory D root@DESKTOP-59T6U68:/mnt/c/d# tcc -shared -fPIC -o libsudoku_c3.so dlxdll.c root@DESKTOP-59T6U68:/mnt/c/d# exit exit memory D SELECT * FROM luajit_module(mode := 'compile', sql_name := 'sudoku_c', source := ' return function(p) local ffi = require("ffi") ffi.cdef[[ int sudoku_solve(const char *p, char *out); ]] local lib = ffi.load("/mnt/c/d/libsudoku_c3.so") local buf = ffi.new("char[82]") local rc = lib.sudoku_solve(p, buf) if rc ~= 1 then return NULL end return ffi.string(buf) end '); ┌─────────┬─────────┬─────────┬─────────┬─────────┬──────────┐ │ ok │ mode │ phase │ message │ detail │ sql_name │ │ boolean │ varchar │ varchar │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┼─────────┼─────────┼──────────┤ │ true │ compile │ compile │ OK │ │ sudoku_c │ └─────────┴─────────┴─────────┴─────────┴─────────┴──────────┘ Run Time (s): real 0.019 user 0.019758 sys 0.000000 memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 100); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 978654213645231798213879654796518432521347986384926175437182569159463827862795341 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.075 user 0.031685 sys 0.026326 memory D memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 1000); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 987563214124987653356412987671398542295174836843256179439825761562731498718649325 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.114 user 0.066773 sys 0.012692 memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 10000); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 987652431164739582253481796375824619842916357619375824728163945431597268596248173 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.483 user 0.466168 sys 0.003213

分别使用两种算法的数独求解程序，跳舞链的速度比MRV启发式—dfs快得多。

又尝试mold的作者编写的chibicc编译器。先从源码编译二进制可执行文件。

C:\Users\lt>wsl root@DESKTOP-59T6U68:/mnt/c/Users/lt# docker start gcc gcc root@DESKTOP-59T6U68:/mnt/c/Users/lt# docker exec -it gcc bash root@6ae32a5ffcde:/# cd /par root@6ae32a5ffcde:/par# cd chibicc-main root@6ae32a5ffcde:/par/chibicc-main# make cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o codegen.o codegen.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o hashmap.o hashmap.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o main.o main.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o parse.o parse.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o preprocess.o preprocess.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o strings.o strings.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o tokenize.o tokenize.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o type.o type.c cc -std=c11 -g -fno-common -Wall -Wno-switch -c -o unicode.o unicode.c cc -std=c11 -g -fno-common -Wall -Wno-switch -o chibicc codegen.o hashmap.o main.o parse.o preprocess.o strings.o tokenize.o type.o unicode.o root@6ae32a5ffcde:/par/chibicc-main# cd ..

后面的步骤和tcc一样

root@6ae32a5ffcde:/par# /par/chibicc-main/chibicc -shared -fPIC -o libsudoku_c4.so dlxdll.c ld: warning: /tmp/chibicc-RZBje2: missing .note.GNU-stack section implies executable stack ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker root@6ae32a5ffcde:/par# ./duckdb155 DuckDB v1.5.5 (Variegata) Enter ".help" for usage hints. memory D force install luajit from community; memory D load luajit; memory D SELECT * FROM luajit_module(mode := 'compile', sql_name := 'sudoku_c', source := ' return function(p) local ffi = require("ffi") ffi.cdef[[ int sudoku_solve(const char *p, char *out); ]] local lib = ffi.load("/par/libsudoku_c4.so") local buf = ffi.new("char[82]") local rc = lib.sudoku_solve(p, buf) if rc ~= 1 then return NULL end return ffi.string(buf) end '); ┌─────────┬─────────┬─────────┬─────────┬─────────┬──────────┐ │ ok │ mode │ phase │ message │ detail │ sql_name │ │ boolean │ varchar │ varchar │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┼─────────┼─────────┼──────────┤ │ true │ compile │ compile │ OK │ │ sudoku_c │ └─────────┴─────────┴─────────┴─────────┴─────────┴──────────┘ memory D SELECT luajit_s('sudoku_c', '000001002000020030004500600007600050080090006100005800001004000070900003400030020'); ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ luajit_s('sudoku_c', '000001002000020030004500600007600050080090006100005800001004000070900003400030020') │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ 359461782716829534824573619947682351583197246162345897631254978275918463498736125 │ └───────────────────────────────────────────────────────────────────────────────────────────────────────────┘ memory D .timer on memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 100); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 978654213645231798213879654796518432521347986384926175437182569159463827862795341 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.065 user 0.026620 sys 0.029178 memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 1000); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 987563214124987653356412987671398542295174836843256179439825761562731498718649325 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.140 user 0.125244 sys 0.003991 memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 10000); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 987652431164739582253481796375824619842916357619375824728163945431597268596248173 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 1.157 user 2.327257 sys 0.011661

chibicc编译出来的动态库比tcc编译的动态库执行速度慢一些，使用意义不大。
上述2种编译器都没有优化，再改用gcc -O3优化编译

memory D .system bash root@6ae32a5ffcde:/par# gcc -shared -fPIC -o libsudoku_c5.so dlxdll.c -O3 root@6ae32a5ffcde:/par# exit exit memory D SELECT * FROM luajit_module(mode := 'compile', sql_name := 'sudoku_c', source := ' return function(p) local ffi = require("ffi") ffi.cdef[[ int sudoku_solve(const char *p, char *out); ]] local lib = ffi.load("/par/libsudoku_c5.so") local buf = ffi.new("char[82]") local rc = lib.sudoku_solve(p, buf) if rc ~= 1 then return NULL end return ffi.string(buf) end '); ┌─────────┬─────────┬─────────┬─────────┬─────────┬──────────┐ │ ok │ mode │ phase │ message │ detail │ sql_name │ │ boolean │ varchar │ varchar │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┼─────────┼─────────┼──────────┤ │ true │ compile │ compile │ OK │ │ sudoku_c │ └─────────┴─────────┴─────────┴─────────┴─────────┴──────────┘ Run Time (s): real 0.008 user 0.010392 sys 0.000509 memory D SELECT max(luajit_s('sudoku_c', column0)) from (from read_csv('sudoku17.txt',header=0) limit 10000); ┌───────────────────────────────────────────────────────────────────────────────────┐ │ max(luajit_s('sudoku_c', column0)) │ │ varchar │ ├───────────────────────────────────────────────────────────────────────────────────┤ │ 987652431164739582253481796375824619842916357619375824728163945431597268596248173 │ └───────────────────────────────────────────────────────────────────────────────────┘ Run Time (s): real 0.458 user 0.438527 sys 0.001587

和tcc的结果没有明显差距。瓶颈好像不在动态库这边。

网站建设 企业官网 运营推广
返回列表