Why specifying the call type when performing P/Invoke is important
I was working on a C# project that required interaction between unmanaged and managed code. It’s nothing complex, just calling various functions in a DLL through a wrapper class, then I hit this exception:
Strange thing is, this “stack unbalanced” exception only pop up when running in debug mode, the application still work just fine without debugging, but leave bugs in applications is not my habit, so I decided to reach out to Google. The unmanaged code used __declspec(dllexport) to specify exported functions, internet forums suggested this exception is caused by the difference in calling convention.
So, I changed the unmanaged code from
extern "C" __declspec(dllexport) int evLPROpen(int width, int height, int widthStep, int depth = 8, int channel = 3);
to
extern "C" __declspec(dllexport) int __cdecl evLPROpen(int width, int height, int widthStep, int depth = 8, int channel = 3);
And the wrapper from
[DllImport("LicensePlateRecognitionLib.dll")] public static extern int evLPROpen(int width, int height, int widthStep, int depth, int channel);
to
[DllImport("LicensePlateRecognitionLib.dll", CallingConvention=CallingConvention.Cdecl)] public static extern int evLPROpen(int width, int height, int widthStep, int depth, int channel);
(using the clear stack by caller convention), and the exception goes away. By default, .NET assumes you use __stdcall, but I didn’t have much success trying this đŸ™‚