Upgrading to Team Foundation Server 2008

This weekend I finally pulled the trigger on upgrading our TFS environment from TFS 2005 to 2008. By and large this was a pretty painless upgrade with the notable exception of TFS Build which has turned out to be a royal pain.

Don’t get me wrong, feature-wise TFS Build 2008 is far superior to the 2005 version, but it was pretty discouraging to find that the build types created for a set of Visual C++ 2005 solutions just completely stopped working. I eventually gave up on the existing build types and started over with a fresh build type and by numerous searches, I’ve been able to get most things working.

Building VS 2005 Solutions

For various reasons, we have one project that we can’t upgrade to Visual Studio 2008 quite yet, so I had to keep the solution files as VS 2005 solution files, but still be able to build them with TFS Build 2008. After poking around a bit, these blog posts told me how to configure that:

• Overriding the ToolsVersion for Your Projects in Team Build
• Building VC++ 2005 Projects With Team Build 2008

In my TFSBuild.proj, I specified the solutions to build as:

<SolutionToBuild Include="<solution path>">
    <Targets></Targets>
    <Properties>ProjectToolsVersion=2.0;VCBuildToolPath=$(ProgramFiles)\Microsoft Visual Studio 8\vc\vcpackages</Properties>
</SolutionToBuild>

The property ProjectToolsVersion tells TFS Build to target .NET 2.0 and the VCBuildToolPath tells it to use the specified path to find VCBuild.exe which is used to build Visual C++ projects.

Generating Custom Build Numbers

We already had functionality to automatically update the product and file versions of our binaries as part of the build so that every build generated binaries with a unique version number. TFS Build 2008, however, offers the possibility to customize the build number. I thought it would be useful to let the build number match the file version generated by the build, making it easier to identify a particular build just by looking at the version number.

Again, there were several blog posts that talked about how to generate custom build numbers:

• Aligning Build Numbers with Assembly Versions in TFS2008
• Orcas SP1 TFS Build Changes, Part 2

I wound up using a slightly different approach than the above. I’m still using a file containing the version number of the previous build (e.g. 1.2.3.4), but I wanted my file to be version controlled which posed a problem since the build number must be generated at the very beginning of the build, before all the source files to use for the build have been retrieved.

Since all I had available to me were the files that are located in my build type directory, I simply moved my version file to the build type directory which ensures it’s available to me. Next, I used the Version task from the MSBuild Community Tasks project to update this file:

<Target Name="BuildNumberOverrideTarget">

<!– Update the product version number –>

<Version VersionFile="$(TxProductVersionFile)" BuildType="None" RevisionType="Increment">
  <Output TaskParameter="Major" PropertyName="TxMajorVersion" />
  <Output TaskParameter="Minor" PropertyName="TxMinorVersion" />
  <Output TaskParameter="Build" PropertyName="TxPatchVersion" />
  <Output TaskParameter="Revision" PropertyName="TxBuildVersion" />
</Version>

<CreateProperty Value="$(TxMajorVersion).$(TxMinorVersion).$(TxPatchVersion).$(TxBuildVersion)">
  <Output TaskParameter="Value" PropertyName="TxCompleteVersion"/>
</CreateProperty>

<CreateProperty Value="$(TxMajorVersion),$(TxMinorVersion),$(TxPatchVersion),$(TxBuildVersion)">
  <Output TaskParameter="Value" PropertyName="TxCompleteVersionRc"/>
</CreateProperty>

<!– Define the build version number –>
<PropertyGroup>
  <BuildNumber>$(TeamProject)_$(BuildDefinitionName)_$(TxCompleteVersion)</BuildNumber>
</PropertyGroup>

</Target>

This gives me my custom build number, but I also need to check the changed version number file back in. The easiest way to do this was to override the AfterGet target:

<Target Name="AfterGet">
   <Message Text="Updating the version-controlled product version file"/>
   <CallTarget Targets="UpdateVersionedProductVersionFile"/>
  <Message Text="Updating the product version information in preparation for the build." />
  <CallTarget Targets="SetProductVersionForBuild" />
</Target>

Where the definition of UpdateVersionedProductVersionFile is:

<PropertyGroup>
  <UpdateVersionedProductVersionFileDependsOn>
  </UpdateVersionedProductVersionFileDependsOn>
</PropertyGroup>
<Target Name="UpdateVersionedProductVersionFile" DependsOnTargets="$(UpdateVersionedProductVersionFileDependsOn)">
  <!– Check out the product version file we now have available to us after the source files have been retrieved. –>
  <Exec WorkingDirectory="$(SolutionRoot)" Command="$(TxTf) checkout &quot;$(TxVersionedProductVersionFile)&quot;"/>

  <!– Copy the previously updated product version file to the version-controlled product version file –>
  <Copy SourceFiles="$(TxProductVersionFile)" DestinationFiles="$(TxVersionedProductVersionFile)"/>

  <!– And finally check the file back in so it’s available for the next build. –>
  <Exec WorkingDirectory="$(SolutionRoot)" Command="$(TxTf) checkin /override:&quot;Automated build check-in, no associated work items&quot; /comment:&quot;Checking in the updated product version file for build $(BuildNumber)&quot; &quot;$(TxVersionedProductVersionFile)&quot;"/>
</Target>

TxProductVersionFile defines the fully qualified path of the version number file in the BuildType directory created by TFS Build when the build is initiated and TxVersionedProductVersionFile the fully qualified path of the same file in the Sources directory (which of course means that I have configured my build type so that the build type itself is included in the workspace, this is easy to do in the new build type editor).

More Stuff

The above is what I’ve come up with so far. I still have to figure out why my code signing is failing (this is my fault, I changed the user account used to run TFS build, so I have to move the certificate to the appropriate certificate store) and my unit tests don’t appear to run properly yet.

I’ll post an update once I get that sorted out.

Update

I did manage to get the code signing sorted out. It turns out I was trying to import the code-signing certificate in a .spc format which isn’t good enough for code-signing purposes. The certificate needed to be in .pfx format to be imported and used properly for code-signing (the certificate would still show up in the Certificates MMC console, but SignFile task would fail with the error message that no certificate was found that matched all criteria).

Another item I found is that there now are several new targets you can override for the build, see Customizable Team Foundation Build Targets for more information. For our needs, the AfterCompileConfiguration target comes in handy since that lets us conveniently create a single build that builds the code, signs the binaries, builds the installer and then finally sings the installer.

That said, I still haven’t figured out why my unit tests won’t run…

Another Update

After having upgraded our solution to VS2008, the unit tests are now running just fine on our build server. I didn’t spend a whole lot of time researching the issue while the solution was still using VS2005 as moving it to VS2008 was one of our short term goals anyway.

Here are the definitions of the TxProductVersionFile and TxVersionedProductFile properties.

<!–

Define the names of the files containing the product version information.
TxProductVersionFile is the file stored in the build type directory. This file is
copied to the build directory before all the source files are copied to the temporary
workspace. This file has the version number incremented and the resulting version
number is then used to generate the build number.TxVersionedProductVersionFile is only available to us after we’ve retrieved the
source files required for the build. This file is checked out, updated so that
it contains the same information as TxProductVersionFile and then checked back in
so that we’re assured of having the updated version number available to us for
the next time the build type is invoked.–>
<TxProductVersionFile>$(SolutionRoot)\..\BuildType\ProductVersion.txt</TxProductVersionFile>
<TxVersionedProductVersionFile>$(SolutionRoot)\TeamBuildTypes\Main\ProductVersion.txt</TxVersionedProductVersionFile>

More Build Definitions

In response to Stephen’s questions:

1. $(TxTf) is defined as:

    <!-- The fully qualified path of tf.exe, used to check in/out items during the build -->
    <TxTf>&quot;$(TeamBuildRefPath)\..\tf.exe&quot;</TxTf>

Basically it defines the fully qualified path of tf.exe which I use to run TFS-related commands that don’t have a convenient existing MSBuild target (keep in mind that I did fairly minimal work on this build script as part of the update, there may have been new targets added that I just haven’t looked for yet in TFS 2008).

2. That was a typo I introduced when copying the code from the .targets file to WordPress, the extra quotes have to be XML encoded, I’ve updated that code above.

3. The definitions of SetProductVersionForBuild target (and dependent targets) are:

  <!--
    SetProductVersionForBuild increments the version number in the global version number
    file and updates the product version information in all .rc files accordingly.
  -->
  <PropertyGroup>
    <SetProductVersionForBuildDependsOn>
      IncrementRcProductVersions;
      IncrementRcFileVersions;
    </SetProductVersionForBuildDependsOn>
  </PropertyGroup>
  <Target Name="SetProductVersionForBuild" DependsOnTargets="$(SetProductVersionForBuildDependsOn)">
    <Message Text="Updated the version information in preparation for the build."/>
  </Target>

  <!--
    IncrementRcProductVersions updates the ProductVersion information in all
    .rc files in the project.
  -->
  <PropertyGroup>
    <IncrementRcProductVersionsDependsOn>
      CheckOutRcFiles;
    </IncrementRcProductVersionsDependsOn>
  </PropertyGroup>
  <Target Name="IncrementRcProductVersions" DependsOnTargets="$(IncrementRcProductVersionsDependsOn)">
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="PRODUCTVERSION\s+\d+\s*,\s*\d+\s*,\s*\d+\s*,\s*\d+" ReplacementText="PRODUCTVERSION $(TxMajorVersion),$(TxMinorVersion),$(TxPatchVersion),$(TxBuildVersion)"/>
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="VALUE\s+&quot;ProductVersion&quot;\s*,\s*&quot;\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*(\)*&quot;" ReplacementText="VALUE &quot;ProductVersion&quot;, &quot;$(TxMajorVersion).$(TxMinorVersion).$(TxPatchVersion).$(TxBuildVersion)&quot;"/>
  </Target>

  <!--
    IncrementRcFileVersions updates the ProductVersion information in all
    .rc files in the project.
  -->
  <PropertyGroup>
    <IncrementRcFileVersionsDependsOn>
      CheckOutRcFiles;
    </IncrementRcFileVersionsDependsOn>
  </PropertyGroup>
  <Target Name="IncrementRcFileVersions" DependsOnTargets="$(IncrementRcFileVersionsDependsOn)">
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="FILEVERSION\s+\d+\s*,\s*\d+\s*,\s*\d+\s*,\s*\d+" ReplacementText="FILEVERSION $(TxMajorVersion),$(TxMinorVersion),$(TxPatchVersion),$(TxBuildVersion)"/>
    <FileUpdate Files="@(TxRcFiles)" Encoding="ASCII" Regex="VALUE\s+&quot;FileVersion&quot;\s*,\s*&quot;\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*[\.,]\s*\d+\s*(\)*&quot;" ReplacementText="VALUE &quot;FileVersion&quot;, &quot;$(TxMajorVersion).$(TxMinorVersion).$(TxPatchVersion).$(TxBuildVersion)&quot;"/>
  </Target>

  <!--
    CheckOutRcFiles checks out all .rc files in the Source directory in preparation
    for them to be updated with new version information.
  -->
  <PropertyGroup>
    <CheckOutRcFilesDependsOn>
      FindAllRcFiles;
    </CheckOutRcFilesDependsOn>
  </PropertyGroup>
  <Target Name="CheckOutRcFiles" DependsOnTargets="$(CheckOutRcFilesDependsOn)">
    <!-- Note that this command is run in our Source directory so we don't check out files from the Extern directory -->
    <Exec WorkingDirectory="$(TxSourceDir)" Command="$(TxTf) checkout /recursive $(TxRcSpec)"/>
  </Target>

  <!--
    CheckOutRcFiles checks in all .rc files in the Source directory that were
    updated with the new version information.
  -->
  <PropertyGroup>
    <CheckInRcFilesDependsOn>
    </CheckInRcFilesDependsOn>
  </PropertyGroup>
  <Target Name="CheckInRcFiles" DependsOnTargets="$(CheckInRcFilesDependsOn)">
    <!-- Note that this command is run in our Source directory so we don't check out files from the Extern directory -->
    <Exec WorkingDirectory="$(TxSourceDir)" Command="$(TxTf) checkin /override:&quot;Automated build check-in, no associated work items&quot; /comment:&quot;Checking in the updated resource files for build $(BuildNumber)&quot; /recursive $(TxRcSpec)"/>
  </Target>

  <!--
    The FindAllRcFiles target finds all .rc files under $(SolutionRoot) and
    stores that information in the $(TxRcFiles) property.
  -->
  <PropertyGroup>
    <FindAllRcFilesDependsOn>
    </FindAllRcFilesDependsOn>
  </PropertyGroup>
  <Target Name="FindAllRcFiles" DependsOnTargets="$(FindAllRcFilesDependsOn)">
    <CreateItem Include="$(TxSourceDir)\**\$(TxRcSpec)">
      <Output ItemName="TxRcFiles" TaskParameter="Include"/>
    </CreateItem>
  </Target>

Generating Custom TFS Check-in E-mails – Part 3.5

I realized that I haven’t talked anything about how the whole TFS Spam system hangs together, something that may be useful as I go on with this series. In my best managerial style I, of course, broke out PowerPoint to create a pretty picture that shows the major components of the system (click the image to see a larger version):

I apologize for the use of PowerPoint and promise that I’ll try not to use it too often.

Everything starts with a check-in to TFS (as I’ve mentioned, we’re still on TFS 2005 but will upgrade to TFS 2008 in the not too distant future). Any time something is checked-in, the TFS server raises an event to any interested parties. You can read more about the event system in TFS here.

In our case, we have a web service that sits and listens for these events and when one is received, it parses the notification to get the id of the changeset from the notification data. The notification comes in the form of a relatively simple XML document:

<?xml version=”1.0″ encoding=”utf-16″?>
<CheckinEvent xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”>
  <AllChangesIncluded>true</AllChangesIncluded>
  <Subscriber>user name</Subscriber>
  <CheckinNotes>
    <CheckinNote xsi:type=”NameValuePair”
                 name=”Code Reviewer” val=”" />
    <CheckinNote xsi:type=”NameValuePair”
                 name=”Performance Reviewer” val=”" />
    <CheckinNote xsi:type=”NameValuePair”
                 name=”Security Reviewer” val=”" />
  </CheckinNotes>
  <PolicyFailures>
    <PolicyFailure xsi:type=”NameValuePair”
                   name=”[Invalid Policy]”
                   val=”Internal error in Changeset Comments Policy” />
  </PolicyFailures>
  <CheckinInformation />
  <Artifacts>
    <Artifact xsi:type=”ClientArtifact”
              ArtifactType=”Changeset” ServerItem=”">
      <Url>changeset link to standard TFS web UI</Url>
    </Artifact>
    <Artifact xsi:type=”ClientArtifact”
              ArtifactType=”VersionedItem”
              Item=”item name”
              Folder=”item folder path”
              TeamProject=”project name”
              ItemRevision=”revision number”
              ChangeType=”edit”
              ServerItem=”item server path”>
      <Url>item URL</Url>
    </Artifact>
    <Artifact xsi:type=”ClientArtifact”
              ArtifactType=”VersionedItem”
              Item=”item name”
              Folder=”item folder”
              TeamProject=”project name”
              ItemRevision=”revision number”
              ChangeType=”edit”
              ServerItem=”item server path”>
      <Url>item URL</Url>
    </Artifact>
  </Artifacts>
  <Title>
    event title (the same one you’d see in the standard e-mail
    subject header).
  </Title>
  <ContentTitle>
    Changeset 43690: Changed the action and namespace
    information for the Notify method. Also changed…
  </ContentTitle>
  <Owner>user name</Owner>
  <Committer>user name</Committer>
  <Number>43690</Number>
  <CreationDate>2/12/2008 11:06:08 AM</CreationDate>
  <Comment>check-in comment</Comment>
  <TimeZone>Pacific Standard Time</TimeZone>
  <TimeZoneOffset>-08:00:00</TimeZoneOffset>
  <TeamProject>project name</TeamProject>
  <PolicyOverrideComment>Policy issue.</PolicyOverrideComment>
</CheckinEvent>

You’ll recognize much of the information in this notification document from the standard check-in e-mail that TFS can produce.

Once we have the id of the changeset, we pass that to a class that processes the changeset. At a high level, this class generates a XML document that contains the information about the changeset, including all the code file differences and then processes this document with a XSLT transform which generates a HTML document that gets e-mailed to the configured subscribers.

All-in-all, there’s nothing particularly fancy about this, but it’s still useful to know how things hang together.

Generating Custom TFS Check-in E-mails – Part 3

When I started this project, I already had some limited experience with programming against the TFS object model since I had written a utility to migrate bugs from our old bug tracking system to TFS. That said, there were still some new areas to explore since my previous experience was all with the WorkItemStore portion of TFS and nothing with the VersionControlServer portion (I’ve linked to the TFS 2005 version of the documentation since we haven’t yet upgraded to TFS 2008).

To make life a little bit easier for myself I started out by cleaning up the code from my previous utility and refactoring it in a way that made it a bit more broadly useable.

Generating Custom TFS Check-in E-mails – Part 2

As discussed in Part 1, I wanted to provide the functionality of CVSspam for TFS in order to give everyone on the team a better understanding of the code being checked in. The first step in this was, obviously, to find a way to compare text files so that I could present the differences between two versions of a file.

I was much too lazy to actually implement a difference algorithm myself and instead went to search for an existing library to do this. After a bit of searching I came across the article An O(ND) Difference Algorithm for C# which after writing some test code seemed like it would do the trick nicely.

Generating Custom TFS Check-in E-mails – Part 1

We do the vast majority of our development in Visual Studio 2005 and 2008, so when we were picking a new source code management tool last year, getting something that integrates nicely with Visual Studio was obviously a priority.

We had all worked with a relatively wide variety of tools in the past (Visual SourceSafe, ClearCase, CVS, Perforce, etc.) with most experiences being good (Visual SourceSafe being an obvious exception . We also wanted a nicely integrated solution with bug tracking, build management and so on. Given that we’re a pretty small team we wanted something that would be easy to maintain.

After having looked at a few alternatives, we decided to go with Microsoft’s Team Foundation Server mostly because it seemed to provide most of the features we wanted with a minimum of fuss.

We’re currently running TFS 2005, but will be upgrading to TFS 2008 in the not too distant future (meaning in the next 2 months or so).

In a previous job I worked on a project that used CVS for source code control (along with Ant and Maven to round things out), but the tool that really made a difference (especially considering that the developers on the project were scattered across the country) was CVSspam which sends out e-mails with highlighted code differences for every check-in (see an example here).

I found these e-mails to be an excellent way of getting an understanding of what was going on in the code base, not to mention that they worked as a (very) informal code-review. The standard TFS check-in notification e-mails leave quite a bit to be desired and although you can customize them by modifying the XSLT files, I couldn’t see any obvious way of replicating CVSspam using just XSLT so I decided to write my own tool (cleverly called TFS Spam) to accomplish this.

Over the next couple of posts I will detail how this tool was created and sometime later this summer we’re going to put the source code up on CodePlex just in case anyone else finds it useful. In the meantime, here’s a screen shot of one of our test check-in notifications:

Introducing the TranxCoders

Welcome to the TranxCoders blog. This blog is written by the development team at Tranxition, a small software company in Beaverton, Oregon, USA.

This is mostly a developer-oriented blog where we talk about things we’re currently working on, tools we use (or create) and other things that might be of general interest for developers. The goal is to keep up a fairly regular posting schedule, so feel free to yell at us if we start to slack off.