To clean up an old farm from some setup errors, I recently rebuilt the farm from bare metal, re-attaching all the content DB's after the rebuild to prevent data loss.  Imagine my horror when after 100% successful testing, and flawless installation, the first page I pull up shows this:

Mutex could not be created.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.InvalidOperationException: Mutex could not be created.

After some research I learned that this is an ASP.Net issue that occasionally crops up when an app pool identity (ie the Web App Service Account in SharePoint) is not a local administrator.  When setting the identity, access to the Windows Registry key where the IIS Mutex Key is kept is supposed to be granted (obviously not an issue for local admins).  If it does not have access to read this key, the above error is generated.  To add it, you have to modify the key and grant permissions for you identity account(s) to read that key, then close all Mutex Threads (or just reboot the server).

A big thanx to Jerry Orman who documented this issue and the fix here:

http://blogs.msdn.com/jorman/archive/2006/07/24/system-invalidoperationexception-mutex-could-not-be-created.aspx

As a side note to any trying to implement least-permission security on a locked down server, make sure your local WSS_WPG group has full control of the Windows\Temp folder, or you'll get an odd ASP.Net Temporary files error (that points to a bogus dir).  The WSS_WPG group contains all your Web App Identity accounts, and is created by the SharePoint Installation.