Sun
Apr 6
2008

Canon Elph on Steroids

The open source project CHDK has built firmware enhancements for many of Canon’s compact cameras. Some features:

- RAW support
- Extra fast/slow shutter speeds
- In-camera scripting language
- DOF calculator
- AutoISO based on focal length
- Exposure bracketing
- Live histogram
- Highlight clipping indicators
- Detailed battery level indicator
- Teathered shooting
- Loads from SD card on power-up (no permanent changes to firmware)

Some first impressions… The UI is organized but far from user friendly. Lightroom cannot open the RAW files. At high shutter speeds the flash power was too high and there was no flash compensation. I ended up holding a white envelope in front of the flash. I tried taking some high speed photos with the new firmware on my Canon SD500. The best photo was taken with the stock firmware. The high speed shutter wasn’t needed for this shot – the focus and timing were more important. These photos were taken in daylight with no additional lighting beyond the flash.

Canon SD500 with CHDK (1/4000s)
Water drops with SD500 and CHDK firmware (1/4000)

Canon SD500 (1/60s)
Water drops with Point and Shoot

Nikon D40 (1/500s)
Water drops with SLR

I think I will keep it on my camera for the extra information on the display and the scripting. A few links: installing, download, usage, more usage info

Fri
Mar 28
2008

Processing

A friend told me I needed to write more blog posts. I’m not sure if he was interested in what I have to say or just didn’t want to link to a dead blog. Either way I will put down a few more posts!

Take a look at the work from Robert Hodgin at flight404.com. He is using Processing, a programming language created for experimenting with computer graphics. At the core of these images and videos is a model following simple rules of physics (e.g. two positively charged particles will move away from each other) that is affected by the music (e.g. particle size increases with volume). The final image or video isn’t designed, it is the result of programming these rules and letting the model play itself out.

Magnetic Ink
Magnetic Ink

Magnetic, process video

Solar (high definition version, alternate version with lyrics)

Fri
Aug 3
2007

Global exception handling

Some general notes on exception handling:

  • Setup a global exception handler to catch and handle unexpected exceptions.
  • Only catch exceptions where you can take more intelligent action than the global exception handler.
  • Be careful with exceptions on any thread other than the main thread (spawned threads, asynchronous delegates, timers, etc…)
    • In .NET 1.1, the CLR eats any exceptions, killing the thread but allowing your app to continue.
    • In .NET 2.0, unhandled exceptions take down the entire appdomain (more info).

In ASP.NET
1. Add the following to global.asax:

void Application_Error(object sender, EventArgs e) 
{
    Exception ex = Server.GetLastError();
    if (ex is System.Web.HttpUnhandledException)
    {
       ex = ex.InnerException;
    }
    LogHelper.Log.Error("Unhandled exception", ex);
    Server.Transfer("~/Error.aspx");
}

2. Create an Error.aspx page:
Use Server.GetLastError() to retrieve the exception.
Ensure the exception is logged in global.asax in case the Server.Transfer() fails.
Do not display the exception to the user to prevent hackers from gleaning information.

3. If you are using Forms Authentication, add the following to your web.config. It ensures unauthenticated users can see the Error.aspx page in case an error occurs during logon/logoff. You should do the same for any images and stylesheets as well.

<configuration>
	<location path="Error.aspx">
		<system.web>
			<authorization>
				<allow users="?"/>
			</authorization>
		</system.web>
	</location>
</configuration>

4. If you are using AJAX, you have to add an event handler on every page (are there any better solutions?):

protected void ScriptManager1_AsyncPostBackError(object sender, AsyncPostBackErrorEventArgs e)
{
    LogHelper.Log.Error("Unhandled exception during asyncpostback", e.Exception);
    ScriptManager1.AsyncPostBackErrorMessage = 
        "An unexpected error occured. Please try again or contact the Help Desk.";
}

ASP.NET Web Services
The Global.Application_Error does not fire in a web service.

using System;
using System.Web.Services.Protocols;
using TDR.Common.Utility;
 
namespace MySolution.MyProject.SoapExtensions
{
    public class LoggingSoapExtension : SoapExtension
    {
        public override object GetInitializer(LogicalMethodInfo methodInfo, SoapExtensionAttribute attribute)
        {
            return null;
        }
 
        public override object GetInitializer(Type serviceType)
        {
            return null;
        }
 
        public override void Initialize(object initializer)
        {
        }
 
        public override void ProcessMessage(SoapMessage message)
        {
            if (message.Stage == SoapMessageStage.AfterSerialize)
            {
                if (message.Exception != null)
                {
                    LogManager.Log.Error(message.Exception.Message, message.Exception);
                }
            }
        }
    }
}

2. Add this to your web.config:

<system.web>
    <webServices>
        <soapExtensionTypes>
            <add type="MySolution.MyProject.SoapExtensions.LoggingSoapExtension,MySolution" priority="1" group="1" />
        </soapExtensionTypes>
    </webServices>
</system.web>

Note: SoapExtensions are not called when invoking the a webservice from the built-in test webpage.

WinForms, Console Apps and Services
1. Hook into the domain’s unhandled exception event:

static void Main()
{
    AppDomain.CurrentDomain.UnhandledException +=
        new UnhandledExceptionEventHandler(
            CurrentDomain_UnhandledException);
}
 
/// <summary>
/// Log unhandled exceptions that occur anywhere in the appdomain.
/// </summary>
static void CurrentDomain_UnhandledException(object sender, 
                                     UnhandledExceptionEventArgs e)
{
    try 
    {
 
        LogHelper.Log.Error("Unhandled exception.", 
                   e.ExceptionObject as Exception); 
    }
    catch { }
}
Fri
Aug 3
2007

MSBuild Automated Deploy Scripts for Web Applications

These are suggested steps for creating batch scripts that will deploy a solution to DEV, TEST and PROD. If this article looks like it is more complicated than it should be, I agree. I think batch scripts with an XML read/write utility or NAnt would be a better choice:

  • The syntax for batch files and NAnt are both easier to read/understand
  • NAnt allows custom tasks can be written in C# directly in the build file. MSBuild requires a separate assembly (and associated solution and project files).
  • The big advantage with MSBuild is it it built-in. However this advantage is eliminated as we need to use the MSBuildCommunityTasks DLL to read/write config files.

Here is the MyApp.build file:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <!--
       MY WEB APPLICATION DEPLOY SCRIPT
 
       To deploy this application use the batch files DeployToXXX.bat
 -->
 <Import Project="..\..\Utilities\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
 
 <PropertyGroup>
 
   <TempFolder>..\..\_tempfolder</TempFolder>
   <DevFolder>\\mydevserver\wwwroot\myapp</DevFolder>
   <TestFolder>\\mytestserver\wwwroot\myapp</TestFolder>
 
 </PropertyGroup>
 
 
 
 <Target Name="DeployToDev">
 
   <AspNetCompiler
       VirtualPath="MyApp"
       PhysicalPath="..\ "
       TargetPath="$(TempFolder)"
       Clean="true"
       Force="true"
       Debug="false"
       Updateable="false" />
 
   <XmlRead XmlFileName="$(TempFolder)\web.config"
            XPath="/configuration/appSettings/add[@key='DEV.Log4NetConnectionString']/@value">
     <Output TaskParameter="Value" PropertyName="Log4NetConnectionString"/>
   </XmlRead>
 
   <XmlUpdate XmlFileName="$(TempFolder)\web.config"
              XPath="/configuration/appSettings/add[@key='Environment']/@value"
              Value="DEV" />
 
   <XmlUpdate XmlFileName="$(TempFolder)\web.config"
              XPath="/configuration/log4net/appender[@type='log4net.Appender.ADONetAppender']/connectionString/@value"
              Value="$(Log4NetConnectionString)" />
 
   <XmlUpdate XmlFileName="$(TempFolder)\web.config"
              XPath="/configuration/system.web/compilation/@debug"
              Value="false" />
 
   <CreateItem
       Include="$(DevFolder)\**\*"
     <Output TaskParameter="Include" ItemName="FilesToCleanFromTarget" />
   </CreateItem>
 
   <Delete Files="@(FilesToCleanFromTarget)" />
 
   <CreateItem
     Include="$(TempFolder)\**\*"
     Exclude="$(TempFolder)\Deploy\*">
     <Output TaskParameter="Include" ItemName="FilesToDeploy"/>
   </CreateItem>
 
   <Copy SourceFiles="@(FilesToDeploy)"
         DestinationFiles="@(FilesToDeploy->'$(DevFolder)\%(RecursiveDir)%(Filename)%(Extension)')"
   />
 
   <RemoveDir Directories="$(TempFolder)" />
 
 </Target>
 
 
 
 <Target Name="DeployToTest">
 
   <CreateItem
       Include="$(DevFolder)\**\*"
       Exclude="$(DevFolder)\Deploy\*">
     <Output TaskParameter="Include" ItemName="FilesToDeploy"/>
   </CreateItem>
 
   <Copy SourceFiles="@(FilesToDeploy)"
         DestinationFiles="@(FilesToDeploy->'$(TestFolder)\%(RecursiveDir)%(Filename)%(Extension)')" />
 
   <XmlRead XmlFileName="$(TestFolder)\web.config"
            XPath="/configuration/appSettings/add[@key='TEST.Log4NetConnectionString']/@value">
     <Output TaskParameter="Value" PropertyName="Log4NetConnectionString"/>
   </XmlRead>
 
   <XmlUpdate XmlFileName="$(TestFolder)\web.config"
              XPath="/configuration/appSettings/add[@key='Environment']/@value"
              Value="TEST" />
 
   <XmlUpdate XmlFileName="$(TestFolder)\web.config"
              XPath="/configuration/log4net/appender[@type='log4net.Appender.ADONetAppender']/connectionString/@value"
              Value="$(Log4NetConnectionString)" />
 
   <XmlUpdate XmlFileName="$(TestFolder)\web.config"
              XPath="/configuration/system.web/compilation/@debug"
              Value="false" />
 
 </Target>
 
 
 
 <Target Name="DeployToProd">
 
   <Copy SourceFiles="$(TestFolder)\web.config"
         DestinationFiles="$(TestFolder)\Web.config.test" />
 
   <XmlRead XmlFileName="$(TestFolder)\web.config"
            XPath="/configuration/appSettings/add[@key='PROD.Log4NetConnectionString']/@value">
     <Output TaskParameter="Value" PropertyName="Log4NetConnectionString"/>
   </XmlRead>
 
   <XmlUpdate XmlFileName="$(TestFolder)\web.config"
              XPath="/configuration/appSettings/add[@key='Environment']/@value"
              Value="PROD" />
 
   <XmlUpdate XmlFileName="$(TestFolder)\web.config"
              XPath="/configuration/log4net/appender[@type='log4net.Appender.ADONetAppender']/connectionString/@value"
              Value="$(Log4NetConnectionString)" />
 
   <XmlUpdate XmlFileName="$(TestFolder)\web.config"
              XPath="/configuration/system.web/compilation/@debug"
              Value="false" />
 
   <Prompt Text="Please ask the production administrator to copy MyApp from test to production. Press Enter when complete."/>
 
   <Copy SourceFiles="$(TestFolder)\Web.config.test"
         DestinationFiles="$(TestFolder)\Web.config" />
   <Delete Files="$(TestFolder)\Web.config.test" />
 
 </Target>
 
</Project>

Then to run use the following batch script:

@echo off
echo.
echo --------------------------------------------------
echo.
echo   Deploy the Application to Dev
echo.
echo   Ensure you have retrieved the latest version
echo   from source control.
echo.
echo --------------------------------------------------
echo.
 
call "%VS80COMNTOOLS%\vsvars32.bat" 
 
set MSBuildCommunityTasksPath=..\..\Utilities\MSBuildCommunityTasks
 
msbuild deploy.build /target:DeployToDev
 
pause

This requires the MSBuildCommunityTasks DLLs. Download, install, copy the DLLs to your project directory under Utilities, checkin to VSS and then uninstall. Placing the DLLs in VSS ensures subsequent developers can use the project without hacking their way through a bunch of dependencies.

Fri
Aug 3
2007

Multi-Environment Config

The following class allows you to easily switch between DEV, TEST and PROD appConfig settings. Settings are stored in one location (your web.config or app.config file). You can define settings global to all environments and override settings for each environment as neccessary.

1. In your app, access your settings with:

MessageBox.Show("The current environment is " + Config.Name);
MessageBox.Show("The connection timeout is " + Config.ConnectionTimeout);

2. Setup your web.config or app.config as follows, with an Environment appSetting:

<?xml version="1.0"?>
<configuration>
 <appSettings>
 
   <!-- ENVIRONMENT
        Use the "Environment" key below to select the environment (DEV | TEST | PROD). 
 
        For settings common to all environments, add a normal appSetting key (e.g. "Name").
        To change a setting for a specific environment, add an appSetting key with
        the environment name (e.g. "DEV.Name"). An environment-specific setting
        will override a common setting.
 
        In the example below, the DEV environment will have a connection timeout of 
        2 minutes while the TEST and PROD environments will use 90 minutes.
   -->
 
   <add key="Environment" value="DEV"/>
 
   <add key="DEV.Name"  value="Development"/>
   <add key="TEST.Name" value="Test"/>
   <add key="PROD.Name" value="Production"/>
 
   <add key="ConnectionTimeout"     value="90"/>
   <add key="DEV.ConnectionTimeout" value="2"/>
 
 </appSettings>
</configuration>

3. Add the following Config.cs class to your project.

using System;
using System.Configuration;
using System.Collections.Generic;
using System.Text;
 
/// <summary>
/// A type-safe wapper around the project's config file. This class ensures 
/// all appSetting keys are located in one place and should provide more
/// readable code. This also allow settings to be easily moved to a database
/// or other store.
/// </summary>
public class Config
{
   /// <summary>
   /// Retrieve a value from the config file. The "environment" appsetting
   /// configures the environment (e.g. Fort Hills development, Edmonton test).
   /// Prefix appsetting with this environment value to specifiy an appsetting
   /// that is specific to the environment. 
   /// 
   /// Note: Do NOT make this method public as we want to provide a 
   /// type-safe wrapper and ensure all  constants (the appSetting keys) 
   /// are located in one place (this class).
   /// </summary>
   private static string GetValue(string key)
   {
       string env = ConfigurationManager.AppSettings["environment"];
       string fullKey = env + "." + key;
       string value = ConfigurationManager.AppSettings[fullKey];
 
       if (value == null || value == "")
           value = ConfigurationManager.AppSettings[key];
 
       if (value == null)
           value = "";
 
       return value;
   }
 
   /// <summary>
   /// Name of the environment
   /// </summary>
   public static string Name
   {
       get { return GetValue("Name").Trim(); }
   }
 
   /// <summary>
   /// Note we convert this to a TimeSpan so that users of this class
   /// don't need to know if the timeout is specified in seconds, 
   /// minutes or hours. When adding new settings convert them 
   /// to a data type that will be most useful to users of this class.
   /// </summary>
   public static TimeSpan ConnectionTimeout
   {
        get 
        { 
             return TimeSpan.FromMinutes(
                 Convert.ToDouble(GetValue("ConnectionTimeout"))); 
        }
   }
}
Mon
Jun 25
2007

HttpContext.Cache vs. HttpContext.Items

HttpContext.Cache is simply a pointer the HttpRuntime.Cache also known as the application cache. This can be verified by opening System.Web with Reflector. The MSDN documentation for this member is misleading: “Gets the Cache object for the current HTTP request”. Information stored here is global to the application domain, not a specific request.

To store per-request information use HttpContext.Items.

Wed
May 2
2007

Blogroll

I had better be a good blog citizen and link to some friend’s blogs….

Ross
Danny
Darren

Fri
Apr 20
2007

Impersonate User

/// <summary>
/// Impersonate a user within the current thread.
/// This class will automatically revert the user to the original user 
/// identity when this class is disposed. Use this class with the 
/// C# using or a try/catch block to ensure the user is reverted
/// after an exception.
/// 
/// Suggested usage:
/// using (new ImpersonateHelper("domain", "joe", "password"))
/// {
///     ...code to run as joe
/// }
/// ...code to run as orignial user
/// 
/// Note, this implementation is vulnerable to "exception filter" attacks
/// as described below. Evaluate it against the risk of other attacks
/// such as the credential store used to impersonate.
/// http://blogs.msdn.com/shawnfa/archive/2005/03/22/400749.aspx
/// </summary>
public class ImpersonationHelper : IDisposable
{
    IntPtr m_tokenHandle = new IntPtr(0);
    WindowsImpersonationContext m_impersonatedUser;
 
    #region Win32 API Declarations
 
    const int LOGON32_PROVIDER_DEFAULT = 0;
    const int LOGON32_LOGON_INTERACTIVE = 2;    //This parameter causes LogonUser to create a primary token.
 
    [DllImport("advapi32.dll", SetLastError=true)]
    public static extern bool LogonUser(String lpszUsername, String lpszDomain, String lpszPassword, 
    int dwLogonType, int dwLogonProvider, ref IntPtr phToken);
 
    [DllImport("kernel32.dll", CharSet=CharSet.Auto)]
    public extern static bool CloseHandle(IntPtr handle);
 
    [DllImport("advapi32.dll", CharSet=CharSet.Auto, SetLastError=true)]
    public extern static bool DuplicateToken(IntPtr ExistingTokenHandle, 
    int SECURITY_IMPERSONATION_LEVEL, ref IntPtr DuplicateTokenHandle);
 
    #endregion
 
    /// <summary>
    /// Constructor. Impersonates the requested user. Impersonation lasts until
    /// the instance is disposed.
    /// </summary>
    public ImpersonationHelper(string domain, string user, string password)
    {
        // Call LogonUser to obtain a handle to an access token.
        bool returnValue = LogonUser(user, domain, password,
            LOGON32_LOGON_INTERACTIVE, LOGON32_PROVIDER_DEFAULT,
            ref m_tokenHandle);
        if (false == returnValue)
        {
            int ret = Marshal.GetLastWin32Error();
            throw new System.ComponentModel.Win32Exception(ret);
        }
 
        // Impersonate
        m_impersonatedUser = new WindowsIdentity(m_tokenHandle).Impersonate();
    }
 
    #region IDisposable Pattern
 
    /// <summary>
    /// Revert to original user and cleanup.
    /// </summary>
    protected virtual void Dispose(bool disposing)
    {
        if (disposing)
        {
            // Revert to original user identity
            if (m_impersonatedUser != null)
                m_impersonatedUser.Undo();
        }
 
        // Free the tokens.
        if (m_tokenHandle != IntPtr.Zero)
            CloseHandle(m_tokenHandle);
    }
 
    /// <summary>
    /// Explicit dispose.
    /// </summary>
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }
 
    /// <summary>
    /// Destructor
    /// </summary>
    ~ImpersonationHelper()
    {
        Dispose(false);
    }
 
    #endregion
}
Tue
Apr 17
2007

Log4Net + ASP.Net

In web applications, Log4Net’s ThreadContext properties (typically items like the current user) should be set immediately before each logging call.

Normally these properties are set in Global.Session_Start(). However, log4net internally stores these properties in thread local storage. Using Reflector to disassemble log4net.dll (v1.2.10), start with log4net.ThreadContext which leads to ThreadContextProperties.get_Item() and then to ThreadContextProperties.GetProperties(). The disassembly for this method:

internal PropertiesDictionary GetProperties(bool create)
{
    PropertiesDictionary data;
    data = (PropertiesDictionary) Thread.GetData(s_threadLocalSlot);
    if ((data == null) && create)
    {
        data = new PropertiesDictionary();
        Thread.SetData(s_threadLocalSlot, data);
    }
    return data;
}

The call to Thread.GetData() is the problem. ASP.Net, when under load, may switch threads when processing a single request (see ThreadStatic, CallContext and HttpContext in ASP.Net). You may initialize log4net with the current user on one thread, then attempt to log on a different thread. Log4net.LogicalThreadContext() suffers a similar problem as it uses CallContext to store its data rather than HttpContext.

For reliable logging information, set these ThreadContext properties each time you log an entry. The additional processing time will be negligable in most cases. Further, as a general rule of thumb, use HttpContext to store request-specific information in ASP.Net applications.

10 minutes after finishing this post I come across this post, also from piers7:
log4net Context problems with ASP.Net thread agility. It suggests another workaround.

Wed
Apr 11
2007

General Tools

Dependency Walker
Determines external dependencies (down to the function level) on Windows exe’s, dll’s, etc…
XPath Expression Testbed
FileHelpers
Create strongly typed flat file reader/writers
Unlocker

« Newer PostsOlder Posts »

© 2009 Brian Low. All rights reserved.