A custom task for building WiX under MSBuild (TFS)

July 3, 2008

My name is Shawn (Hempel) and this is my first submission to the Tranxcoder blog, which is a little ironic since I think it was my idea in the first place. Nonetheless, at least now I am officially “on the board” (and the team can stop razzing me about not posting.)

It seems I’ll be continuing on a running theme of writing about making Team Foundation Server work for us in new and scary ways. If you’re wondering why so much of our blog has centered on that topic, I think it’s two parts: 1) in many ways TFS customization is neither easy or intuitive so as we struggle to make things work, we want to share that with the development community; and 2) since our build process has little to do with the logic in our products, it’s easy to put this stuff out here without worrying about giving away intellectual property. That said, there will probably be more non-build related content as time goes on – we’ll just change variable names to protect the innocent. Alright, on to the good stuff…

As previously mentioned by Richard, we use the WiX toolset to generate Windows Installer binaries (MSI/MSM) for our applications. Personally, I find the recent builds of Votive (WiX’s Visual Studio integration package) to be quite helpful and easy to work with. One of the things it does very well is enable a simple F5 installer build. What it does not do, however, is help you in any way once you decide to graduate from building on the desktop to incorporating an installer build in your TFS processes. Specifically, there will almost certainly be some trouble with the relative and/or absolute paths used in your WiX source files to locate the build outputs to package. Read on to find out how we first solved that problem in a way that made it hard to sleep at night, and subsequently replaced it with a robust, maintainable solution.

Read the rest of this entry »


WiX, MSBuild, and Build Verification, Part 1

June 27, 2008

Probably the least thought about, and planned for process in the development cycle is the installer. Since few projects get completed early, time for a well planned installation takes a back seat to the main development effort. However, we’ve all heard over and over that the customer’s first impression of your product will be your installer. At Tranxition, we were given the time to research the various installation products currently available, and decided to with WiX for a number of reasons.

Why WiX?

WiX has a couple great advantages. It integrates seamlessly with Visual Studio. This means it can be included in our nightly build process simply by including our installation projects into our Visual Studio solution. It produces an .msi file, allowing our installer to work on all Windows versions we need to support. Also, we get the installer repair and uninstall options for free. Speaking of getting things for free… WiX is free. Yes, a free open source Microsoft originated project.

You say nothing is ever free? That is somewhat true, there are some disadvantages. Documentation is scarce, you will depend heavily on blogs to learn the product, and figure out your issues. As an open source product, it has bugs (that so far I’ve been able to work around), but some times I’ve had to go back to an older installation, or needed to manually edit the project files. However, we have multiple configuration of our final product, seven different released products. The differences are mainly to the installer GUI, differing help files, and a couple customer specific DLLs. WiX works beautifully with these different configurations, and we don’t need to create a separate project per installer.

Over the next few months, I will detail our WiX installation scripts and configuration, the integration of our WiX projects into our MSBuild process, along with our build verification process. This first article, I will describe our installers, our build configurations, how we use the project configuration to create each of our product releases, and why this might help you decided to also use WiX.

Read the rest of this entry »


Upgrading to Team Foundation Server 2008

June 23, 2008

This weekend I finally pulled the trigger on upgrading our TFS environment from TFS 2005 to 2008. By and large this was a pretty painless upgrade with the notable exception of TFS Build which has turned out to be a royal pain.

Don’t get me wrong, feature-wise TFS Build 2008 is far superior to the 2005 version, but it was pretty discouraging to find that the build types created for a set of Visual C++ 2005 solutions just completely stopped working. I eventually gave up on the existing build types and started over with a fresh build type and by numerous searches, I’ve been able to get most things working.

Building VS 2005 Solutions

For various reasons, we have one project that we can’t upgrade to Visual Studio 2008 quite yet, so I had to keep the solution files as VS 2005 solution files, but still be able to build them with TFS Build 2008. After poking around a bit, these blog posts told me how to configure that:

In my TFSBuild.proj, I specified the solutions to build as:

<SolutionToBuild Include="<solution path>">
    <Targets></Targets>
    <Properties>ProjectToolsVersion=2.0;VCBuildToolPath=$(ProgramFiles)\Microsoft Visual Studio 8\vc\vcpackages</Properties>
</SolutionToBuild>

The property ProjectToolsVersion tells TFS Build to target .NET 2.0 and the VCBuildToolPath tells it to use the specified path to find VCBuild.exe which is used to build Visual C++ projects.

Generating Custom Build Numbers

We already had functionality to automatically update the product and file versions of our binaries as part of the build so that every build generated binaries with a unique version number. TFS Build 2008, however, offers the possibility to customize the build number. I thought it would be useful to let the build number match the file version generated by the build, making it easier to identify a particular build just by looking at the version number.

Again, there were several blog posts that talked about how to generate custom build numbers:

I wound up using a slightly different approach than the above. I’m still using a file containing the version number of the previous build (e.g. 1.2.3.4), but I wanted my file to be version controlled which posed a problem since the build number must be generated at the very beginning of the build, before all the source files to use for the build have been retrieved.

Since all I had available to me were the files that are located in my build type directory, I simply moved my version file to the build type directory which ensures it’s available to me. Next, I used the Version task from the MSBuild Community Tasks project to update this file:

<Target Name="BuildNumberOverrideTarget">

<!– Update the product version number –>

<Version VersionFile="$(TxProductVersionFile)" BuildType="None" RevisionType="Increment">
  <Output TaskParameter="Major" PropertyName="TxMajorVersion" />
  <Output TaskParameter="Minor" PropertyName="TxMinorVersion" />
  <Output TaskParameter="Build" PropertyName="TxPatchVersion" />
  <Output TaskParameter="Revision" PropertyName="TxBuildVersion" />
</Version>

<CreateProperty Value="$(TxMajorVersion).$(TxMinorVersion).$(TxPatchVersion).$(TxBuildVersion)">
 
<Output TaskParameter="Value" PropertyName="TxCompleteVersion"/>
</CreateProperty>

<CreateProperty Value="$(TxMajorVersion),$(TxMinorVersion),$(TxPatchVersion),$(TxBuildVersion)">
 
<Output TaskParameter="Value" PropertyName="TxCompleteVersionRc"/>
</CreateProperty>

<!– Define the build version number –>
<PropertyGroup
>
 
<BuildNumber>$(TeamProject)_$(BuildDefinitionName)_$(TxCompleteVersion)</BuildNumber
>
</PropertyGroup>

</Target>

This gives me my custom build number, but I also need to check the changed version number file back in. The easiest way to do this was to override the AfterGet target:

<Target Name="AfterGet">
 
 <Message Text="Updating the version-controlled product version file"/>
 
 <CallTarget Targets="UpdateVersionedProductVersionFile"/>
 
<Message Text="Updating the product version information in preparation for the build." />
  <CallTarget Targets="SetProductVersionForBuild" />
</Target>

Where the definition of UpdateVersionedProductVersionFile is:

<PropertyGroup>
  <UpdateVersionedProductVersionFileDependsOn>
  </UpdateVersionedProductVersionFileDependsOn>
</PropertyGroup>
<Target Name="UpdateVersionedProductVersionFile" DependsOnTargets="$(UpdateVersionedProductVersionFileDependsOn)">
  <!– Check out the product version file we now have available to us after the source files have been retrieved. –>
  <Exec WorkingDirectory="$(SolutionRoot)" Command="$(TxTf) checkout &quot;$(TxVersionedProductVersionFile)&quot;"/>

  <!– Copy the previously updated product version file to the version-controlled product version file –>
  <Copy SourceFiles="$(TxProductVersionFile)" DestinationFiles="$(TxVersionedProductVersionFile)"/>

  <!– And finally check the file back in so it’s available for the next build. –>
  <Exec WorkingDirectory="$(SolutionRoot)" Command="$(TxTf) checkin /override:&quot;Automated build check-in, no associated work items&quot; /comment:&quot;Checking in the updated product version file for build $(BuildNumber)&quot; &quot;$(TxVersionedProductVersionFile)&quot;"/>
</Target>

TxProductVersionFile defines the fully qualified path of the version number file in the BuildType directory created by TFS Build when the build is initiated and TxVersionedProductVersionFile the fully qualified path of the same file in the Sources directory (which of course means that I have configured my build type so that the build type itself is included in the workspace, this is easy to do in the new build type editor).

More Stuff

The above is what I’ve come up with so far. I still have to figure out why my code signing is failing (this is my fault, I changed the user account used to run TFS build, so I have to move the certificate to the appropriate certificate store) and my unit tests don’t appear to run properly yet.

I’ll post an update once I get that sorted out.

Update

I did manage to get the code signing sorted out. It turns out I was trying to import the code-signing certificate in a .spc format which isn’t good enough for code-signing purposes. The certificate needed to be in .pfx format to be imported and used properly for code-signing (the certificate would still show up in the Certificates MMC console, but SignFile task would fail with the error message that no certificate was found that matched all criteria).

Another item I found is that there now are several new targets you can override for the build, see Customizable Team Foundation Build Targets for more information. For our needs, the AfterCompileConfiguration target comes in handy since that lets us conveniently create a single build that builds the code, signs the binaries, builds the installer and then finally sings the installer.

That said, I still haven’t figured out why my unit tests won’t run…

Another Update

After having upgraded our solution to VS2008, the unit tests are now running just fine on our build server. I didn’t spend a whole lot of time researching the issue while the solution was still using VS2005 as moving it to VS2008 was one of our short term goals anyway.

Here are the definitions of the TxProductVersionFile and TxVersionedProductFile properties.

<!–

Define the names of the files containing the product version information.
TxProductVersionFile is the file stored in the build type directory. This file is
copied to the build directory before all the source files are copied to the temporary
workspace. This file has the version number incremented and the resulting version
number is then used to generate the build number.TxVersionedProductVersionFile is only available to us after we’ve retrieved the
source files required for the build. This file is checked out, updated so that
it contains the same information as TxProductVersionFile and then checked back in
so that we’re assured of having the updated version number available to us for
the next time the build type is invoked.
–>
<
TxProductVersionFile>$(SolutionRoot)\..\BuildType\ProductVersion.txt</TxProductVersionFile>
<
TxVersionedProductVersionFile>$(SolutionRoot)\TeamBuildTypes\Main\ProductVersion.txt</TxVersionedProductVersionFile>

More Build Definitions

In response to Stephen’s questions:

1. $(TxTf) is defined as:

    <!-- The fully qualified path of tf.exe, used to check in/out items during the build -->
    <TxTf>&quot;$(TeamBuildRefPath)\..\tf.exe&quot;</TxTf>

Basically it defines the fully qualified path of tf.exe which I use to run TFS-related commands that don’t have a convenient existing MSBuild target (keep in mind that I did fairly minimal work on this build script as part of the update, there may have been new targets added that I just haven’t looked for yet in TFS 2008).

2. That was a typo I introduced when copying the code from the .targets file to WordPress, the extra quotes have to be XML encoded, I’ve updated that code above.

3. The definitions of SetProductVersionForBuild target (and dependent targets) are:

  <!--
    SetProductVersionForBuild increments the version number in the global version number
    file and updates the product version information in all .rc files accordingly.
  -->
  <PropertyGroup>
    <SetProductVersionForBuildDependsOn>
      IncrementRcProductVersions;
      IncrementRcFileVersions;
    </SetProductVersionForBuildDependsOn>
  </PropertyGroup>
  <Target Name="SetProductVersionForBuild" DependsOnTargets="$(SetProductVersionForBuildDependsOn)">
    <Message Text="Updated the version information in preparation for the build."/>
  </Target>

  <!--
    IncrementRcProductVersions updates the ProductVersion information in all
    .rc files in the project.
  -->
  <PropertyGroup>
    <IncrementRcProductVersionsDependsOn>
      CheckOutRcFiles;
    </IncrementRcProductVersionsDependsOn>
  </PropertyGroup>
  <Target Name="IncrementRcProductVersions" DependsOnTargets="$(IncrementRcProductVersionsDependsOn)">
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="PRODUCTVERSION\s+\d+\s*,\s*\d+\s*,\s*\d+\s*,\s*\d+" ReplacementText="PRODUCTVERSION $(TxMajorVersion),$(TxMinorVersion),$(TxPatchVersion),$(TxBuildVersion)"/>
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="VALUE\s+&quot;ProductVersion&quot;\s*,\s*&quot;\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*(\)*&quot;" ReplacementText="VALUE &quot;ProductVersion&quot;, &quot;$(TxMajorVersion).$(TxMinorVersion).$(TxPatchVersion).$(TxBuildVersion)&quot;"/>
  </Target>

  <!--
    IncrementRcFileVersions updates the ProductVersion information in all
    .rc files in the project.
  -->
  <PropertyGroup>
    <IncrementRcFileVersionsDependsOn>
      CheckOutRcFiles;
    </IncrementRcFileVersionsDependsOn>
  </PropertyGroup>
  <Target Name="IncrementRcFileVersions" DependsOnTargets="$(IncrementRcFileVersionsDependsOn)">
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="FILEVERSION\s+\d+\s*,\s*\d+\s*,\s*\d+\s*,\s*\d+" ReplacementText="FILEVERSION $(TxMajorVersion),$(TxMinorVersion),$(TxPatchVersion),$(TxBuildVersion)"/>
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="VALUE\s+&quot;FileVersion&quot;\s*,\s*&quot;\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*(\)*&quot;" ReplacementText="VALUE &quot;FileVersion&quot;, &quot;$(TxMajorVersion).$(TxMinorVersion).$(TxPatchVersion).$(TxBuildVersion)&quot;"/>
  </Target>

  <!--
    CheckOutRcFiles checks out all .rc files in the Source directory in preparation
    for them to be updated with new version information.
  -->
  <PropertyGroup>
    <CheckOutRcFilesDependsOn>
      FindAllRcFiles;
    </CheckOutRcFilesDependsOn>
  </PropertyGroup>
  <Target Name="CheckOutRcFiles" DependsOnTargets="$(CheckOutRcFilesDependsOn)">
    <!-- Note that this command is run in our Source directory so we don't check out files from the Extern directory -->
    <Exec WorkingDirectory="$(TxSourceDir)" Command="$(TxTf) checkout /recursive $(TxRcSpec)"/>
  </Target>

  <!--
    CheckOutRcFiles checks in all .rc files in the Source directory that were
    updated with the new version information.
  -->
  <PropertyGroup>
    <CheckInRcFilesDependsOn>
    </CheckInRcFilesDependsOn>
  </PropertyGroup>
  <Target Name="CheckInRcFiles" DependsOnTargets="$(CheckInRcFilesDependsOn)">
    <!-- Note that this command is run in our Source directory so we don't check out files from the Extern directory -->
    <Exec WorkingDirectory="$(TxSourceDir)" Command="$(TxTf) checkin /override:&quot;Automated build check-in, no associated work items&quot; /comment:&quot;Checking in the updated resource files for build $(BuildNumber)&quot; /recursive $(TxRcSpec)"/>
  </Target>

  <!--
    The FindAllRcFiles target finds all .rc files under $(SolutionRoot) and
    stores that information in the $(TxRcFiles) property.
  -->
  <PropertyGroup>
    <FindAllRcFilesDependsOn>
    </FindAllRcFilesDependsOn>
  </PropertyGroup>
  <Target Name="FindAllRcFiles" DependsOnTargets="$(FindAllRcFilesDependsOn)">
    <CreateItem Include="$(TxSourceDir)\**\$(TxRcSpec)">
      <Output ItemName="TxRcFiles" TaskParameter="Include"/>
    </CreateItem>
  </Target>


Generating Custom TFS Check-in E-mails – Part 3

June 9, 2008

When I started this project, I already had some limited experience with programming against the TFS object model since I had written a utility to migrate bugs from our old bug tracking system to TFS. That said, there were still some new areas to explore since my previous experience was all with the WorkItemStore portion of TFS and nothing with the VersionControlServer portion (I’ve linked to the TFS 2005 version of the documentation since we haven’t yet upgraded to TFS 2008).

To make life a little bit easier for myself I started out by cleaning up the code from my previous utility and refactoring it in a way that made it a bit more broadly useable.

Read the rest of this entry »


Generating Custom TFS Check-in E-mails – Part 2

May 28, 2008

As discussed in Part 1, I wanted to provide the functionality of CVSspam for TFS in order to give everyone on the team a better understanding of the code being checked in. The first step in this was, obviously, to find a way to compare text files so that I could present the differences between two versions of a file.

I was much too lazy to actually implement a difference algorithm myself and instead went to search for an existing library to do this. After a bit of searching I came across the article An O(ND) Difference Algorithm for C# which after writing some test code seemed like it would do the trick nicely.

Read the rest of this entry »


Generating Custom TFS Check-in E-mails – Part 1

May 13, 2008

We do the vast majority of our development in Visual Studio 2005 and 2008, so when we were picking a new source code management tool last year, getting something that integrates nicely with Visual Studio was obviously a priority.

We had all worked with a relatively wide variety of tools in the past (Visual SourceSafe, ClearCase, CVS, Perforce, etc.) with most experiences being good (Visual SourceSafe being an obvious exception ;-) . We also wanted a nicely integrated solution with bug tracking, build management and so on. Given that we’re a pretty small team we wanted something that would be easy to maintain.

After having looked at a few alternatives, we decided to go with Microsoft’s Team Foundation Server mostly because it seemed to provide most of the features we wanted with a minimum of fuss.

We’re currently running TFS 2005, but will be upgrading to TFS 2008 in the not too distant future (meaning in the next 2 months or so).

In a previous job I worked on a project that used CVS for source code control (along with Ant and Maven to round things out), but the tool that really made a difference (especially considering that the developers on the project were scattered across the country) was CVSspam which sends out e-mails with highlighted code differences for every check-in (see an example here).

I found these e-mails to be an excellent way of getting an understanding of what was going on in the code base, not to mention that they worked as a (very) informal code-review. The standard TFS check-in notification e-mails leave quite a bit to be desired and although you can customize them by modifying the XSLT files, I couldn’t see any obvious way of replicating CVSspam using just XSLT so I decided to write my own tool (cleverly called TFS Spam) to accomplish this.

Over the next couple of posts I will detail how this tool was created and sometime later this summer we’re going to put the source code up on CodePlex just in case anyone else finds it useful. In the meantime, here’s a screen shot of one of our test check-in notifications:

Check-in Email