In part 2, I went over the creation of a type at run-time by way of System.Reflection.Emit
. We are able to define an interface we want to use, and our handy dandy LoadLibrary<T>
method will take care of all the details for us. That, by itself, is pretty magical.
But we're not done. We need a few quality-of-life improvements.
Returning Strings
Let's go back to our SQLite interface.
interface ISqlite3
{
int Open(string file, out IntPtr database);
int Close(IntPtr database);
}
See how that Open
method takes a string
for one of its parameters? It really shouldn't work because System.String
is a radically different data structure from the const char*
that sqlite3_open
expects. The first problem is that System.Char
is 2 bytes, and the char
in C is 1 byte. The second problem is that strings/arrays in .NET embed their length, and C strings use null-termination.
Thankfully, .NET foresaw this and took care of the issue for us. So, this methods works as you would expect! You can simply call sqlite3.Open("stuff.sqlite", out var db);
without difficulty because your string is automatically converted to a null-terminated C string that your C library can actually handle.
However, what if we wanted a method that returns a string? Let's look at sqlite3_errmsg
.
string Errmsg(IntPtr database);
I have bad news for you. If you call this, your program will crash. If it doesn't crash, it'll behave badly regardless. The nice type marshaling we had for values going in apparently does not apply to values coming out. So, how do we fix this? Well, since the C function returns a pointer (const char*
), we need to capture it as a pointer.
IntPtr Errmsg(IntPtr database);
Great. How do I get a string out of that? Well, if you're using .NET Core, the answer is simple.
public static string GetString(IntPtr cString)
{
return cString == IntPtr.Zero ? null : Marshal.PtrToStringUTF8(cString);
}
If you're not using .NET Core... shame on you. Start using .NET Core immediately! If you absolutely cannot use .NET Core (or a sufficiently high enough version of .NET Standard), you are going to have to parse the string yourself.
public static string GetStringTheHardWay(IntPtr cString)
{
if (cString == IntPtr.Zero)
return null;
var bytes = new List<byte>();
int n = 0;
while (true)
{
byte b = Marshal.ReadByte(cString, n++);
if (b == 0)
break;
bytes.Add(b);
}
return Encoding.UTF8.GetString(bytes.ToArray());
}
OK, so, now we have a way to read strings returned from C functions, but this is still too much work. Now, every time I want a string from my method, I have to call GetString(returnedValue)
. That's no good! I want to be lazier than that! So, let's revisit our code generator.
ILGenerator generator = methodBuilder.GetILGenerator();
for (int i = 0; i < parameters.Length; ++i)
generator.Emit(OpCodes.Ldarg, i + 1); // Arg 0 is 'this'. Don't need that one.
if (Environment.Is64BitProcess)
generator.Emit(OpCodes.Ldc_I8, ptr.ToInt64());
else
generator.Emit(OpCodes.Ldc_I4, ptr.ToInt32());
generator.EmitCalli(
OpCodes.Calli,
CallingConvention.Cdecl,
method.ReturnType,
parameterTypes);
generator.Emit(OpCodes.Ret);
The Calli
instruction places the C function's return value onto the stack, and the subsequent Ret
instruction forwards it onto the caller. There is an opportunity here: we know, ahead of time, what the return type is. Why not just handle the conversion right here? If we know that the method on the interface wants to return string
, we can take care of returning IntPtr
internally and converting it before returning it.
// This should be stored outside the loop iterating over the interface methods.
MethodInfo getStringMethod = typeof(WhateverClassHasTheMethod).GetMethod(nameof(GetString));
ILGenerator generator = methodBuilder.GetILGenerator();
for (int i = 0; i < parameters.Length; ++i)
generator.Emit(OpCodes.Ldarg, i + 1); // Arg 0 is 'this'. Don't need that one.
if (Environment.Is64BitProcess)
generator.Emit(OpCodes.Ldc_I8, ptr.ToInt64());
else
generator.Emit(OpCodes.Ldc_I4, ptr.ToInt32());
bool returnsString = method.ReturnType == typeof(string);
generator.EmitCalli(
OpCodes.Calli,
CallingConvention.Cdecl,
returnsString ? typeof(IntPtr) : method.ReturnType,
parameterTypes);
if (returnsString)
generator.Emit(OpCodes.Call, getStringMethod);
generator.Emit(OpCodes.Ret);
Do you see what we did? The return type sent off to EmitCalli
now has special treatment. If we ultimately want a string
to come back, we capture an IntPtr
first. Since that IntPtr
value is on top of the stack after the Calli
instruction, it is in the right position to serve as an argument for the call to GetString
. How convenient! The call to GetString
will place the string
result onto the stack, and then our Ret
instruction will return it to the caller.
Voila. Methods that return string
work fine now.
Library Cleanup
Remember way back in part 1 when we talked about loading and unloading libraries? Our fancy-shmancy LoadLibrary<T>
happily loads up the library but leaves us without a way to unload the library. That may not matter to many developers. It's quite normal for games to load up libraries, use them for the entire lifetime of the application, and then skip unloading them at the end. Regardless, I think it's best we at least offer up some way to unload these libraries as there are applications out there that need to load and unload libraries over time.
There are several approaches we can take here. I suggest leveraging IDisposable
as it is idiomatic and brings several benefits along with it. The top benefit (in my mind) is the ability to put your library into a using
block.
using (ISqlite3 sqlite3 = LoadLibrary<ISqlite3>("custom_sqlite3.dll", GetSqliteFunctionName))
{
sqlite3.Open("my_data.sqlite", out IntPtr database);
// ...
}
Another perk is that we can enforce this in the LoadLibrary<T>
definition itself. Forgetting to mark your interface as IDisposable
will result in a compiler error.
public interface ISqlite3 : IDisposable
{
int Open(string file, out IntPtr database);
int Close(IntPtr database);
}
public static T LoadLibrary<T>(
string library,
Func<string, string> methodNameToFunctionName)
where T : class, IDisposable
{
// ...
}
So, let's make a helper method to implement the Dispose
method.
private static void CreateDisposeMethod(
TypeBuilder typeBuilder,
IntPtr libraryHandle)
{
MethodBuilder methodBuilder= typeBuilder.DefineMethod(
nameof(IDisposable.Dispose),
MyMethodAttributes);
typeBuilder.DefineMethodOverride(
methodBuilder,
typeof(IDisposable).GetMethod(nameof(IDisposable.Dispose)));
Type ptrType = Environment.Is64BitProcess ? typeof(long) : typeof(int);
ConstructorInfo intPtrConstructor = typeof(IntPtr).GetConstructor(new Type[] {ptrType});
MethodInfo closeMethod = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ?
typeof(WinLoader).GetMethod(nameof(WinLoader.FreeLibrary)) :
typeof(UnixLoader).GetMethod(nameof(UnixLoader.Close));
ILGenerator generator = methodBuilder.GetILGenerator();
if (Environment.Is64BitProcess)
generator.Emit(OpCodes.Ldc_I8, libraryHandle.ToInt64());
else
generator.Emit(OpCodes.Ldc_I4, libraryHandle.ToInt32());
generator.Emit(OpCodes.Newobj, intPtrConstructor);
generator.Emit(OpCodes.Call, closeMethod);
generator.Emit(OpCodes.Pop); // Toss the returned int.
generator.Emit(OpCodes.Ret);
}
As you can see, this method has to construct the IntPtr
instance because there is no way to embed a constant IntPtr
into the code. As I mentioned before, I recommend just using SharpLab in helping to determine what IL you want to emit. I'm no IL expert; I just write C# code for what I want to accomplish and study the resulting IL.
Just remember to call CreateDisposeMethod
somewhere in your LoadLibrary<T>
method, and you're gold. In part 4, we'll do more upgrades specific to game dev. :)
Top comments (2)
Hello I found wrong ".dll" under Unix:
string library = "path/to/custom/sqlite3.dll";
bool isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
IntPtr libraryHandle = isWindows ?
WinLoader.LoadLibrary(library) :
UnixLoader.Open(library, 1);
Replace with:
bool isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
IntPtr libraryHandle = isWindows ?
WinLoader.LoadLibrary("path/to/custom/sqlite3.dll") :
UnixLoader.Open("path/to/custom/sqlite3.so", 1);
Because you use only *.dll o_O
If you use dynamic then it is file type as so for Linux/FreeBSD and dll for Windows
Thanks!
For the sake of simplicity, I've actually compiled my native libs to have a
.dll
extension even under Linux. Then my loader code can be the same for all platforms. I would never name a Linux library.dll
if it were to become a system/shared library (as in/usr/lib
), but if it's bundled with my own code for private usage, I don't see a problem.