I’m currently porting a Linux C++ pricing library and agent to Win32 – the goal is to grid enable it and deploy it on a DataSynapse. Now this is a very large project with many hundreds of headers and libs. Understanding the make dependencies and seeing how it would be possible to decompose the code to be grid friendly is complex to say the least.

Simply trying to build the project on Visual Studio, MinGW or CygWin is not enough to see what is missing. The build only shows you what the next few headers and libs are missing and you need to look deep into the dependancy graph and not worry about what builds and what doesn’t.

 IncludeManager from Profactor is a saviour when you need to do this. It’s targeted more at the ‘what will rebuild if I change this file’ use, but I find it very useful for gaining a clearer picture of complex dependancies. It’s easy to collapse the graph to reduce dependancy noise.

And it’s very cheap!

Sometimes you just have no choice. You have resort to some hack in order to make your code work. I recently had to write a GUI client to provide some specialist admin tasks on DataSynapse GridServer v2.6. For various reasons this had to be a C# .NET GUI and DataSynapse thoughtfully provide a .NET API which made the coding tasks pretty straightforward. Everything worked beautifully except when the GUI closes and the App Domain doesn’t unload. Consequently the .exe and vshost.exe just hang around like ghostly processes.

Some investigation revealed that whenever any of the DataSynapse admin .NET API methods are called there’s a sleep, wait or join that never exits.

[In a sleep, wait, or join]
mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout, bool exitContext) + 0x14 bytes

It turns out this is a known bug that I’m sure will be fixed at some stage. However, that didn’t really help me. Despite some failed attempts to terminate the thread, I resorted to some rug pulling. The code now uses my old Suppuku class to make sure everything terminates.

static void Main()
{
      Application.SetCompatibleTextRenderingDefault(false);
      new ShellApplication().Run();
      Suppuku.Sword.Commit(0);
}

And here’s the brutal implementation.

namespace Suppuku
{
      public class Sword
      {
            [DllImport("Kernel32.dll")]
            private static extern bool TerminateProcess(int handle, int exitCode);
            public static void Commit(int exitCode)
            {
                  TerminateProcess(-1, exitCode);
            }
      }
}

Sometimes needs must and Win32 comes to the rescue.  Any other solutions gratefully excepted.

Over the last couple of weeks, I’ve had to do a few recruiting interviews for Lab49: It’s been illuminating. Delegates are always a good things to chat about and usually, a good candidate will easily rattle off what Invoke and BeginInvoke are and what they do. Sometimes you even get an explanation about how BeginInvoke uses a thread allocated from the ThreadPool. So how does BeginInvoke for a form or controls work? “Oh it works the same doesn’t it? Ooooo, not too sure now you mention it”.

Despite Invoke and BeginInvoke having the same signatures for Controls and delegates, they work very differently.

Delegate Invoke
The method referred to by the delegate instance is called on the target object (if there is one), and the result is returned.

Delegate BeginInvoke
This uses a thread allocated from the ThreadPool to execute the target method. So although the thread creation time is generally fast, there are only 25 threads (by default) per CPU so don’t go hogging them for time consuming stuff.

Control/WinForm BeginInvoke
This doesn’t use a thread to execute the referenced delegate. Instead it uses good old PostMessage to put a registered message on the windows message queue. It’s then picked up by the applications message pump and the target will then be execute on the main GUI thread. Remember that PostMessage returns immediately the message has been put on the queue – that’s how it make the async call.

Control/WinForm Invoke
Those of you still listening, may assume Control Invoke therefore uses SendMessage to make the synchronous call. But no. A little investigation reveals that Invoke and BeginInvoke preserve the calling order, so it seems Invoke is doing the same as BeginInvoke except the call is synchronous. Under the hood, Invoke maybe does something like:

object Invoke(Delegate method)
{
  return EndInvoke(BeginInvoke(method, null));
}

This is confirmed by spying the windows message when Invoke and BeginInvoke are used. The mysterious  WindowsForms12_ThreadCallbackMessage registered message appears in message queue in both cases. I guess the motivation for this was being kind to the message pump. Messages will still be pumped while an Invoke is being called.

A word of caution when using Control Invoke/BeginInvoke. Because they both post messages to the message queue, you might expect everything to behave sweetly. However, if you repeatedly call the control delegates quickly, you’ll suddenly find your GUI will be unresponsive and won’t paint. This is because WM_PAINT and WM_TIMER are the amoeba of the windows message world and will always get pushed to the bottom of the message queue. If you have to deal with high data feed rates, use another mechanism to control the frequency they arrive, otherwise you’ll end up with a sticky GUI. 

Here’s an example of a safe control wrapper that uses control delegates. There’s a plethora of these type of classes, but this is my own twist on it. This approach uses BeginInvoke when called from the non-GUI thread. It results in the GUI being generally more responsive because it’s not having to wait for the the delegate to respond. I also use a technique I’ve called self-referencing delegates. When an invoke is required, the code creates a new delegate to itself and BeginInvokes it. When the same referenced method gets called in the main GUI thread, the base method is called.  

public class SafeTextBox : TextBox
{
  delegate void Set(string text);
  delegate string Get();
  override public string Text
  {
     set
     {
        if (InvokeRequired)
        {
           Set setDelegate = delegate(string text) {Text = text;};
           BeginInvoke(setDelegate, new object[] {value});
        }
        else
           base.Text = value;
     }
     get
     {
        if (InvokeRequired)
        {
           Get getDelegate = delegate() {return Text;};
           return (string)EndInvoke(BeginInvoke(getDelegate, null));
        }
        else
           return base.Text;
     }
  }
}

The worms are turning

February 9, 2007

I wholeheartedly agree with Jeff Ventura I’ve been borrowing my brother-in-laws’ Mac on and off recently, experimenting with Parallels as a viable XP development base for a while. I can have OSX on one monitor and my PC development using VS2005 under XP running on the other, completely seamlessly.

The decision was made a while ago. When I get a new PC it will be a Mac – no question.  

It’s interesting that despite C# and the CLR making our lives easier by providing simpler code constructs and managing memory for us, it’s still very easy to screw things up; especially when it comes to thread safety and race conditions. In the good (bad) old days of C++, delete, _beginthread, etc you had to write more code to develop your multithreaded apps and particularly when developing multithreaded GUIs you had to take care and understand more what was actually happening. If you ended up with a race condition, deadlock or message pump block, it was easier (relatively) to see, hear and smell where the problem was under the hood.

By providing events, delegates and simple threading constructs, C# makes writing multi-threaded GUIs really rather straightforward. But it’s exactly those simple constructs that means developers do more with them and consequently find more devious ways to make things go wrong in seemingly non-deterministic ways.

Take events and delegates. The often quoted, thread unsafe example is:

public class EventPublisher
{
   public event EventHandler theEvent;
   public void InvokeEvent()
   {
      // If delegate has no targets it will be null
      if (theEvent != null)
      {
         // So it seems OK to use, but a thread context
         // switch may happen here and remove all the
         // invocation targets.
         theEvent(this, EventArgs.Empty); // Maybe boom!
      } 
   }
}

Clearly, there is a window of opportunity for it to go wrong. And if there’s an open window …..

One counter measure is to make a copy of the delegate. Delegates are immutable so by copying it to a temporary variable you keep a copy of the original state of the delegate, irrespective of any thread context switches. Making any change to the state of the delegate creates a new one on the heap and updates the reference to which the delegate point.

Remember, EventHandler is a shorthand declaration of both a delegate variable and an event of the same type at the same time, so when I’m talking delegates it’s an EventHandler in the code snippets.

But there’s still an open window for a race condition …

The JIT compiler may decide to optimize out your copying for performance reasons, so you are back where you started. Yet another published solution, again uses copying, but this time the copy happens by passing the delegate (EventHandler) as a method parameter.

public class EventPublisher
{
   public event EventHandler theEvent;
   public void InvokeEvent()
   {
      InvokeEvent(theEvent);
   }
   private void InvokeEvent(EventHandler handler)
   {
      if (handler != null)
      handler(this,EventArgs.Empty);
   }
}

On the face of it this looks to close the race condition window of opportunity, but surprisingly it is still open. The JIT compiler’s optimizations may still beat you and decide to inline your method. A method will be a candidate for inlining when its code is ‘small’; just a few lines of code (less than 32 bytes of IL). Exact details are not easy to find, but David Notario has written a couple of articles on JIT Optimizations that are worth a read; JIT optimiations I and JIT optimizations II

The answer, to be safe, is to prevent inlining for the method using the [MethodImpl(MethodImplOptions.NoInlining)] attribute.

public class EventPublisher
{
   public event EventHandler theEvent;
   public void InvokeEvent()
   {
      InvokeEvent(theEvent);
   }

[MethodImpl(MethodImplOptions.NoInlining)]
   private void InvokeEvent(EventHandler handler)
   {
     if (handler != null)
        handler(this,EventArgs.Empty);
   }
}

Hopefully the window is now tightly closed.

Not so long ago, I wrote an article showing how to implement custom hyperlinks using asynchronous pluggable protocols. Even if you’re not familiar with APPs, you use them everyday on your PC. Every protocol, whether the more familiar http, ftp or less familiar JavaScript, res, news etc. is implemented on Windows PCs as an asynchronous pluggable protocol which plug directly into Windows’ existing protocol framework. One of the more obscure protocols is res: to view resources in a DLL in Internet Explorer.

MSDN fully describes how to implement custom protocols schemes using pluggable handlers. This allows developers to implement new schemes for IE and applications that use URL monikers. To fully implement a protocol requires a fair bit of work and is really focused on use in IE.

My particular requirement at the time was managing users expectations that custom hyperlinks could be used to integrate an application into the desktop using IE and the Office suite. IE would be pretty easy, but Office? However, as the article explains, it’s possible to hijack the regular hyperlink behavior and piggyback your own. Consequently you can use regular html anchor syntax to implement your own link behavior – in this case my very important protocol vip.

<a href="vip:some custom parameters here">My custom hyperlink</a>

Put this in a Word or Excel document, it’s displayed as a regular link and when the user clicks on it, your protocol handler catches it.

More recently, I was contacted by Eric Lawrence, the program manger of IE, informing me I’d made a mistake in calling this asynchronous pluggable protocols. I’d only really used a side effect of it. However, he did point out that what I had discovered was something called Application Protocols; something they use internally in the IE group.

Despite my smugness in discovering this, it was good to be corrected.

Unless you’ve been sitting or living under a very large stone over the past couple of years, you’ll know that Microsoft would have us believe Win32 is legacy. So why have they put so much effort into the Interop services? Trust me; it will be around for some time. I take my hat off to the guys who drew the short straw and did it – I bet it was a mission and the end result pretty much works.

However, to the GUI developer who’s experienced the infuriating little dialog box which says something like:

Managed Debugging Assistant ‘PInvokeStackImbalance’ has detected a problem in ‘C:\dev\ SendToCS\bin\Debug\SendToCS.vshost.exe’.
Additional Information: A call to PInvoke function ‘SendToCS!SendFileTo.MAPI::MAPISendMail’ has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

when tracking down problems in calls from managed to unmanaged code, here’s my pain induced, top ten tips to sorting out problems with interop marshalling code.

  1. Read the Platform Invoke Tutorial
  2. Check for a mismatch between your unmanaged function signature and the declared [DllImport] C# function declaration and double check the DLL name it’s implemented in.
  3. Use ref’s so the CLR will pass your structs as a reference to the unmanaged function so it can modify your object, not a orphan stack copy. You must use ref in the function declaration as well as the call.
  4. Check the parameter types – particulary strings vs StringBuilder and the use of ref for string values returned in parameters. Remember string is immutable, although .NET will marshal both to a LPTSTR.
  5. Check the size of struct parameters match that required by the unmanaged function.
  6. Check the types and size of your struct members.
  7. Remember to use [StructLayout(LayoutKind .Sequential)] attributes on structs or classes you reference in parameter pointers. The CLR controls the physical layout of memory, so if the class needs to be arranged in an expected way use StructLayout to preserve the layout and .NET will not mess with it.
  8. Although it’s platform specific, IntPtr is a useful, thread safe structure to represent a pointer or handle – HWNDs are probably the single most common parameter.
  9. The default marshalling for structs is an LPSTRUCT, so there’s nothing to do. If you make it a class you’ll need to marshal it like this [MarshalAs(UnmanagedType.LPStruct)] MapiMessage pMsg);
  10. Read the Platform Invoke Tutorial one more time.