A custom task for building WiX under MSBuild (TFS)

My name is Shawn (Hempel) and this is my first submission to the Tranxcoder blog, which is a little ironic since I think it was my idea in the first place. Nonetheless, at least now I am officially “on the board” (and the team can stop razzing me about not posting.)

It seems I’ll be continuing on a running theme of writing about making Team Foundation Server work for us in new and scary ways. If you’re wondering why so much of our blog has centered on that topic, I think it’s two parts: 1) in many ways TFS customization is neither easy or intuitive so as we struggle to make things work, we want to share that with the development community; and 2) since our build process has little to do with the logic in our products, it’s easy to put this stuff out here without worrying about giving away intellectual property. That said, there will probably be more non-build related content as time goes on – we’ll just change variable names to protect the innocent. Alright, on to the good stuff…

As previously mentioned by Richard, we use the WiX toolset to generate Windows Installer binaries (MSI/MSM) for our applications. Personally, I find the recent builds of Votive (WiX’s Visual Studio integration package) to be quite helpful and easy to work with. One of the things it does very well is enable a simple F5 installer build. What it does not do, however, is help you in any way once you decide to graduate from building on the desktop to incorporating an installer build in your TFS processes. Specifically, there will almost certainly be some trouble with the relative and/or absolute paths used in your WiX source files to locate the build outputs to package. Read on to find out how we first solved that problem in a way that made it hard to sleep at night, and subsequently replaced it with a robust, maintainable solution.

Like all good WiX developers, we attempt to abstract away the details about file paths from the actual <File> directives as much as possible. For instance, in Files.wxs we defined the following component:

18 <Component Id=SKUApp Guid=91062BB2-7010-4960-949E-E9452890645F>

19   <File Id=SKUAppExe KeyPath=yes Name=$(var.SKUName).exe Source=$(var.SKUBinPath)\ Vital=yes” />

22   <File Id=SKUAppExeConfig KeyPath=no Name=$(var.SKUName).exe.config Source=$(var.SKUBinPath)\ Vital=yes />

29 </Component>

You can see that rather than providing an actual path (or in this case, even an actual file name) it specifies that a variable $(var.SKUBinPath) will contain the path to the source file and that $(var.SKUName) will contain the name portion of the file path, while the extension will be specific to each file Id. Ultimately, this component will locate and package the $(OutDir)/App.exe and $(OutDir)/App.exe.config files which are generated as output from a vcbuild project.

The variables referenced are defined in a separate file (cleverly) named Variables.wxi (the .wxi extension suggests that it’s an include file rather than normal WiX source). Within the include file are some preprocessor variable definitions as follows:

2 <Include>

3   <?define SKUName = “App” ?>

4

5   <?define SourceLocation = “..”?>

6   <?define CompilationMode=Release?>

7   <?define SKUBinPath=$(var.SourceLocation)\$(var.CompilationMode)?>

8 </Include>

When the WiX project is built the preprocessor will evaluate those definitions and generate $(var.XXX) variables which can be used throughout any source which pulls in this include file (such as the SKUApp component above.) Notice that the value for $(var.SKUBinPath) is built by concatenating $(var.SourceLocation) and $(var.CompilationMode); but look at the value provided for the SourceLocation variable – it’s a relative path meaning “use the current directory’s parent”. In this case, that path will be correct as long as the project is built under Visual Studio as part of its containing solution, but something happens when you try to build it under TFS…

One of the first things I discovered about Team Build is that all output from a solution is placed in a single drop location called $(OutDir). Without any outside influence, that location will be something like “$(BuildDirectoryPath)\$(TeamProject)\Binaries\$(Platform)\”. In other words, the location of the binaries produced by your builds from Team Build will likely not mirror the location of those same binaries when built from Visual Studio on a developer’s machine. So if you want your developers to be able to build and validate the installer simply by hitting F5, and have that same installer project build automatically on the build machine, then the problem is obvious – we have to do something about the SourceLocation path to make it correct for both.

The first approach we used was to take advantage of a WiX preprocessor feature called Conditional Statements, which are vary similar to preprocessor conditionals in C++ and other languages. We used the value of a particular environment variable as sufficient evidence to determine whether the build is running on the build system or on a developer’s machine. And it looked something like this:

1 <?if $(sys.SOURCEFILEDIR) = “C:\Build Directories\App\Project\Sources\Main\Source\Setup\” ?>

2   <?define SourceLocation = “C:\Build Directories\App\Project\Binaries\Mixed Platforms”?>

3 <?else?>

4   <?define SourceLocation = “..”?>

5 <?endif?>

I don’t know about you, but every time I see code like that all I can think of is “somewhere in the world, a kitten just died.” Maybe that’s a bit of an overreaction, but it does evoke a visceral response in that it just screams of future maintenance nightmares. For instance, what if we want to build this project on multiple build machines? What if the path on this build machine changes? What if the SOURCEFILEDIR variable goes away or its meaning changes? Etc, etc..

In my opinion, any approach which relies on environmental evidence to determine which hard-coded path to use is sub-optimal. Not so much because of the evidence, but because of the hard-coded paths. What we really need is a way to allow those paths to be provided by the build process rather than being specified in the checked-in source. While there are myriad options for achieving that goal, the quickest and most flexible seemed to be creating a custom MSBuild task for replacing variable definitions in a WiX file, which is exactly what the remainder of this post describes.

The custom task is called WixVarSubstitution and is intended to be a general solution for replacing the value of any variable defined in any WiX source file. For a concrete example consider the SourceLocation variable from above. To change the value of that variable within the build script on your build machine you could create a target like this:

1 <PropertyGroup>

2   <_SourceFilePath>$(SolutionRoot)\Main\Source\Setup\Variables.wxi</_SourceFilePath>

3   <_VariableDefinitions>

4     <VariableDefinition Name=SourceLocation NewValue=$(BinariesRoot)\$(Platform) />

5   </_VariableDefinitions>

6 </PropertyGroup>

7

8 <Target Name=AdjustInstallerSourceBinaryLocation>

9   <WixVarSubstitution

10     SourceFile=$(_SourceFilePath)

11     VariableDefinitions=$(_VariableDefinitions) />

12 </Target>

When called, that target will open the file Variables.wxi, locate and replace the value in <? define SourceLocation = “..” ?> with a new value determined from $(BinariesRoot)\$(Platform) and save the change. Then when the build for the Setup project runs, it will have the correct location of the output binaries to package up in the installer.

A key assumption made by this custom task is that variables are specified using the <?define name=”value”?> syntax known as an XML processing instruction. If, for instance, WiX changes the PITarget name from “define” to something else, the custom task will have to be modified. Further, if WiX enables a method of defining variables outside of preprocessing instructions then that will need to be taken into account as well. However, for our purposes and given what I know about WiX today, I believe this task covers the typical case – certainly it has handled our primary need. So on to the task itself.

The details of creating a custom MSBuild task are documented in numerous places all over the ‘Net, including blogs and MSDN. One thing that wasn’t immediately clear to me, though, was what to do if you want to pass a structure or, worse, an array of structures to a custom task. Initially I thought I could just use an ITaskItem array, but quickly realized ITaskItem assumes a correlation to a file on the file system. While it’s possible to tweak that model and use it for other things, that approach did not seem clean. Eventually I discovered that you can easily generate XML properties directly within an MSBuild script. To see the creation of a sample XML property, look at the <_VariableDefinitions> declaration above. While that shows an XML document with only a single node, it can easily be expanded to multiple:

3 <_VarDefs>

4   <Root>

5     <VariableDefinition Name=Var1 NewValue=Val1 />

6     <VariableDefinition Name=Var2 NewValue=Val2 />

7     <VariableDefinition Name=Var3 NewValue=Val3 />

8   </Root>

9 </_VarDefs>

The name of the property itself (in this case _VarDefs) is immaterial. Also immaterial is the name of the Root element you create when you want to specify more than one VariableDefinition (necessary because XML documents may only have one root). In fact all that does matter is that you name the material elements “VariableDefinition” and provide values for the attributes “Name” (the name of the variable to locate) and “NewValue” (the value to provide in place of the existing value.) There is no practical limit to the number of variable definitions provided.

That XML document is passed as a string to the custom task in a property.

55 public string VariableDefinitions { get; set; }

Upon calling Execute() the XML is parsed and turned into a dictionary of Key/Value pairs where the variable names are the keys.

79 // Parse the provided XML string

80 var root = XElement.Parse( VariableDefinitions, LoadOptions.SetBaseUri );

81 var ns = root.GetDefaultNamespace();

82

83 // Build up a dictionary of variables and replacement values

84 Array.ForEach( root.Elements( ns + “VariableDefinition” ).ToArray(), n =>

85 {

86   _substitutions.Add( n.Attribute( “Name” ).Value, n.Attribute( “NewValue” ).Value );

87 } );

In order to retrieve elements by name, you must specify the document namespace if one is used – and MSBuild will always include a namespace in its XML properties (specifically, “http://schemas.microsoft.com/developer/msbuild/2003“). However, the task doesn’t need to know anything about the namespace, so it is sufficient to request it from the parsed root XElement and use that when querying for the variable definition elements.

Note, don’t let the lambda expression fool you into thinking there’s anything complicated going on there – it’s simply a for each over an array of XElement instances. Each instance is used to add a new Key/Value pair to the _substitutions dictionary defined as:

28 private var _substitutions = new Dictionary<string, string>( StringComparer.OrdinalIgnoreCase );

Additionally, I have defined a regular expression as a string constant:

22 // This pattern should match the way WiX variables are defined, e.g. ‘<?define SourceLocation = “..” ?>’

23 // except it only matches on the Data portion of the PI: ‘SourceLocation = “..”‘

24 private const string PIDataRegExPattern = “^{0}\\s*=\\s*\”(?<value>.*)\””;

Following the creation of a dictionary is a fun bit of LINQ to XML combined with regular expressions and a little composite formatting. If you’re not familiar with all of these, the comments should be sufficient to give you an idea of what’s happening (note that it loads the file specified in the string property SourceFile, which was seen in the usage sample earlier):

92 // Aggregate _substitutions keys and produce a string like “(key1)|(key2)|(key3)…”

93 var varNames = “(“ + ( ( _substitutions.Keys.Aggregate( ( leftKey, rightKey ) => “(“ + leftKey + “)|(“ + rightKey ) + “)” ).TrimStart( ‘(‘ ) );

94

95 // Because of the way PIDataRegExPattern is laid out, just stuff the aggregated string into the pattern to generate a Regex

96 var varNamesRegEx = new Regex( string.Format( CultureInfo.InvariantCulture, PIDataRegExPattern, varNames ), RegexOptions.IgnoreCase );

97

98 // This query loads the SourceFile as an XDocument and retrieves all XProcessingInstruction nodes

99 // whose Target is ‘define’ and whose name matches any of the _substitutions keys

100 var replaceNodesQuery = from e in XDocument.Load( SourceFile, LoadOptions.PreserveWhitespace ).DescendantNodes()

101   let n = e as XProcessingInstruction

102   where ( e.NodeType == XmlNodeType.ProcessingInstruction )

103         && n != null // Not sure if I really need this

104         && n.Target == “define”

105         && varNamesRegEx.IsMatch( n.Data )

106   select n;

107

108 // Force deferred execution to complete so we can modify the XDocument

109 var replaceNodes = replaceNodesQuery.ToArray();

110

126 // Loop through each substitution with a nested loop against replaceable nodes and do a regex replacement

127 // This is safe because the pattern match will only succeed when substition.Key matches the variable name

128 // Note: It just assigns the Data property back to itself when there’s no match

129 foreach ( var substitution in _substitutions )

130 {

131   Log.LogMessage( MessageImportance.Low, “> Attempting to replace variable ‘{0}’ with value ‘{1}’ in SourceFile ‘{2}’”, substitution.Key, substitution.Value, SourceFile );

132

133   foreach ( var replaceNode in replaceNodes )

134   {

135     var pattern = string.Format( CultureInfo.InvariantCulture, PIDataRegExPattern, substitution.Key );

136     var replacement = string.Format( CultureInfo.InvariantCulture, “{0} = \”{1}\””, substitution.Key, substitution.Value );

137

138     replaceNode.Data = Regex.Replace( replaceNode.Data, pattern, replacement, RegexOptions.IgnoreCase );

139   }

140 }

141

142 // Write the changes back out to the input file

143 replaceNodes[ 0 ].Document.Save( SourceFile, SaveOptions.DisableFormatting );

Essentially we’re processing an XML file (the WiX source), performing a lookup against a dictionary of keys, replacing matched values, and rewriting the file all in less than 20 lines. In the full source code there is some additional error checking and logging in place, but the meat is all here.

In a future post I (or someone else) will discuss how we go about getting our custom tasks into the build system and make use of them in our scripts. However, most of that is well documented around the ‘Net. In the meantime, I’m providing the source code for this post in the form of a WixVarSubstitution.cs file, which you are welcome to use in your own build environments and/or modify to suit your needs. The only thing required to build it beyond a standard C# DLL project is references to System.XML.Linq, Microsoft.Build.Framework and Microsoft.Build.Utilities.v3.5.

About these ads

4 Responses to A custom task for building WiX under MSBuild (TFS)

  1. Steven Bone says:

    Thanks for the excellent article – saved me from having to write a similar task. Your trick of passing a set of XML Properties will no doubt come in handy at some point in the future. Perhaps you should submit this to the Wix folks!

  2. Sam Shiles says:

    I had some trouble getting this to work as described. The problem seemed to stem from the fact that the “VariableDefinitions” being passed in from the msbuild file did not contain a root element. So instead of getting:

    ..

    We only got

    ..

    I modified the custom task with the following which solved the problem:

    private const string ROOT_ELEMENT = “”;
    private const string ROOT_ELEMENT_FORMAT = “{0}”;
    private string _variableDefinitions;
    public string VariableDefinitions {
    get { return _variableDefinitions; }
    set
    {
    value = value.Trim();
    if (!value.StartsWith(ROOT_ELEMENT))
    value = string.Format(ROOT_ELEMENT_FORMAT, value);

    _variableDefinitions = value;
    }
    }

    There is probably a way of solving this with MSBUILD but I’m more comfortable with c# so that was my solution. Perhaps this is caused by differences between the versions we have so for the sake of comparisson:

    TFS 2008 SP1

  3. Sam Shiles says:

    Sorry the tags got cut out:

    <_VariableDefinitions>
    <VariableDefinition/>
    ..
    <_VariableDefinition/>
    <_/_VariableDefinitions>

    We only got
    <_VariableDefinition/>
    ..
    <_VariableDefinition/>_

  4. hempelcx says:

    Hi, Sam.

    Sorry you had trouble. Note that I did specify later in the post that you will need to wrap a root element around the variable definitions. MSBuild will always strip off the outer element that contains the XML document when it passes it to your custom task.

    <_VariableDefinitions>
      <Root>
        <VariableDefinition Name=“Var1“ NewValue=“Val1“ />
        <VariableDefinition Name=“Var2“ NewValue=“Val2“ />
        <VariableDefinition Name=“Var3“ NewValue=“Val3“ />
      </Root>
    </_VariableDefinitions>

    That outer <Root> element is required for it to be a valid XML document once the <_VariableDefinitions> element is stripped. I don’t recall what MSBuild will do if it’s missing – but sending an invalid document containing just VariableDefinition elements could be the result. Also note that the name of the root element is irrelevant – the XML parsing code doesn’t care.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: