Converting .vcproj files to MSBuild projects – Brain Dump
About eight months ago, I spent time working on a solution for a gap in MSBuild
VC++ support. When you currently build a VC++ solution with MSBuild, MSBuild is
really executing vcbuild.exe to do the actual work. There are no cl and link tasks or
any other C++ MSBuild tasks, and C++ projects still use the old VS project format
instead of relying on MSBuild. Thus, I prototyped an application that could convert
VC++ projects to MSBuild projects and created cl and link tasks.
Although this effort has been abandoned, my goal is to provide a brain dump that
describes the issues I discovered while prototyping and how you might go about
building this tool. My hope is that some interested people in the community will
continue this project to fill this gap in MSBuild compatibility.
This document is a cross between a specification and implementation; it provides
insight into how I intended the solution to work and some implementation details.
The analysis is broken down into three pieces: Comparing the .vcproj format to
MSBuild, designs for the cl and link tasks, and discussion of the application that will
produce the new project file.
The workflow I envisioned for this tool was: A user creates a standard VC++ solution
with VS 2005 that he wants to build with MSBuild. The user runs an application
which converts the VC++ projects in the solution to MSBuild formatted .proj files.
These project files import a C++ .targets file that relies on cl and link MSBuild tasks.
Examining the .vcproj and MSBuild project layouts
For the first part of the discussion, let’s compare the layout of .vcproj and MSBuild
files. This will be useful when we consider how to rewrite the .vcproj file.
Reviewing a VC++ project file, you’ll notice that it uses XML and the file is broken
down within one huge element. Specifically, all of the data in a .vcproj file is
contained within a VisualStudioProject element that starts the file. Within this
element, there are nodes for Platforms, Configurations, References, Files, and
Globals. To convert this into an MSBuild compatible file, we must decide how to
map each node to an MSBuild equivalent.
Let’s start by listing the attributes of the VisualStudioProject element and their
counterpart MSBuild properties:
VisualStudioProject Attributes
ProjectType
Version
Name
ProjectGUID
Keyword
MSBuild Properties
N/A
ProductVersion
N/A
ProjectGuid
N/A
While the ProductVersion and ProjectGuid properties are explicitly defined in
MSBuild project files, the remaining VS attributes don’t appear by default. Instead,
the ProjectType and Name attributes are defined through reserved MSBuild
properties which can be accessed in the project file. The project type is implicitly
defined by the suffix of the MSBuild project file (i.e. C# project files use the .csproj
suffix) and is retrieved by accessing the MSBuildProjectExtension property. The
project name is simply the name of the project file and is obtained through the
MSBuildProjectName property. Note that I never found a mapping for Keyword; we’ll
just ignore it for now.
The Platforms node in a .vcproj lists the platforms that the VC++ project targets on
a build. The listed platforms factor into the next block of XML, which is the list of
Configurations. Each Configuration node has a name attribute, which specifies a
Configuration name (such as Debug or Release) followed by a Platform (such as
Win32). Below is a sample Debug|Win32 configuration from a VC++ project file1:
<Configuration
Name="Debug|Win32"
OutputDirectory="Debug"
IntermediateDirectory="Debug"
ConfigurationType="1"
UseOfMFC="2"
CharacterSet="2">
<Tool
Name="VCCLCompilerTool"
Optimization="0"
PreprocessorDefinitions="WIN32;_WINDOWS;_DEBUG"
MinimalRebuild="TRUE"
BasicRuntimeChecks="3"
RuntimeLibrary="3"
TreatWChar_tAsBuiltInType="TRUE"
UsePrecompiledHeader="3"
WarningLevel="3"
Detect64BitPortabilityProblems="TRUE"
DebugInformationFormat="4"/>
<Tool
Name="VCLinkerTool"
LinkIncremental="2"
GenerateDebugInformation="TRUE"
SubSystem="2"
TargetMachine="1"/>
<Tool
Name="VCPostBuildEventTool"/>
<Tool
Name="VCResourceCompilerTool"
PreprocessorDefinitions="_DEBUG"
Culture="1033"
AdditionalIncludeDirectories="$(IntDir)"/>
<Tool
Name="VCWebServiceProxyGeneratorTool"/>
</Configuration>
The MSBuild approach is to arrange configuration properties in property groups;
these properties are set when the Configuration and Platform match a
PropertyGroup’s condition. Here’s an example from a C# project:
1
To shorten the example, some of the Tool elements that would normally be listed in a configuration
element have been removed.
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' ==
'Debug|AnyCPU' ">
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<Optimize>false</Optimize>
<OutputPath>bin\Debug\</OutputPath>
<DefineConstants>DEBUG;TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
In this example, if $(Configuration) is set to Debug and $(Platform) is set to
AnyCPU, the elements in the group are set with the specified values. Notice that,
unlike the C++ sample, this group contains only a small list of properties. The list of
possible tools is missing. Additionally, only some of the properties in the
PropertyGroup map to VC++ Configuration attributes. For example, the OutputPath
property does map to the OutputDirectory attribute of the Configuration element.
However, the Optimize property doesn’t have an equivalent Configuration attribute.
Instead, it maps to the Optimization attribute in the VCCLCompilerTool element.
Reviewing the list of tools in the Configuration element, it’s clear that some map to
real executables: VCCLCompilerTool maps to cl.exe and VCLinkerTool represents
link.exe. Additionally, there are .vcproj tool elements for al.exe, lc.exe, midl.exe,
bscmake.exe, and others. In the MSBuild world, each of these tools would be
invoked by a separate MSBuild task and these collective tasks would be referenced in
a .targets file. For the prototype, I simplified the situation by focusing solely on
cl.exe and link.exe as the two tool elements I processed. For a complete solution,
you would want to expand your scope to include tasks that map to the various C++
related tools. For “tools” that don’t have any mappings to real world applications,
such as VCPostBuildEventTool, you’ll need to determine their MSBuild equivalents.
For example, post build events in MSBuild are specified between <PostBuildEvent>
elements that are defined within a PropertyGroup.
After the Configuration section, there’s the References node. This is where all the
assembly references for a C++ project are listed. Although I didn’t handle this in the
prototype, it’s clear that a complete solution would need to include these in the
MSBuild project file. To match MSBuild’s format, we’ll want to maintain individual
elements for each Reference but list them together in an ItemGroup. We’ll need to
change the tag from AssemblyReference to Reference and change some attribute
names; for example, MSBuild specifies the assembly name with an Include
attribute, not an AssemblyName attribute.
The Files node has sub-nodes for source files, header files, resource files, and files
not included in a build (like .txt files). These groupings can be mapped to ItemGroup
elements that contain MSBuild Compile tags where the Include attribute would
specify the relative path of the file. Preserving the FileConfiguration data might
be accomplished using child elements of a Compile element or specifying the
properties in the project wide property groups.
<Files>
<Filter
Name="Source Files"
Filter="cpp;c;cxx;def;odl;idl;hpj;bat;asm;asmx"
<File
RelativePath=".\stdafx.cpp"
>
<FileConfiguration
Name="Debug|Win32"
>
<Tool
Name="VCCLCompilerTool"
UsePrecompiledHeader="1"/>
</FileConfiguration>
So, we’ve now taken a quick look at the nodes in .vcproj files and we’ve considered
what their equivalent elements could be in an MSBuild project. For the initial
prototype, I kept my sample .vcproj files very simple so I could focus on compiling
and linking.
Designing cl and link MSBuild tasks
If you review the csc and vbc tasks, you’ll see that they’re invoked with a single XML
element that contains a list of attributes. These attributes mostly translate to flags
used by the tool. I designed my cl and link tasks similarly:
<cl
Sources="@(FilesToCompile)"
AdditionalIncludeDirectories="$(CLAdditionalIncludeDirectories)"
AdditionalOptions="$(CLAdditionalOptions)"
AdditionalUsingDirectories="$(CLAdditionalUsingDirectories)"
...
WarnAsError="$(CLWarnAsError)"
WarningLevel="$(CLWarningLevel)"
WholeProgramOptimization="$(CLWholeProgramOptimization)"
IntermediateDirectory="$(IntDir)">
<Output TaskParameter="ObjectFiles" ItemName="Objects" />
</cl>
<link
Sources="@(Objects)"
AdditionalDependencies="$(LinkAdditionalDependencies)"
AdditionalOptions="$(LinkAdditionalOptions)"
AssemblyDebug="$(LinkAssemblyDebug)"
...
Subsystem="$(LinkSubsystem)"
TargetMachine="$(LinkTargetMachine)"
Version="$(LinkVersion)" />
Here, each attribute corresponds to a cl.exe or link.exe flag. A list of C++ compiler
flags is at http://msdn2.microsoft.com/en-us/library/19z1t1wy(VS.80).aspx and a
list of link.exe flags is available at http://msdn2.microsoft.com/enus/library/y0zzbyt4(VS.80).aspx.
Some things to note are that the cl and link properties are all prepended with CL and
Link respectively. This was done to resolve situations where properties would have
the same name for multiple tasks. For example, both tasks have an
AdditionalOptions attribute and, subsequently, could have an AdditionalOptions
property. Prepending CL and Link distinguishes which AdditionalOptions property
applies to which task.
The Sources attributes for the cl and link tasks are assigned item groups that
contain the list of source files and the list of produced .obj files, respectively. The list
of object files is obtained by using the Output element in the cl task. This element
ties the cl task’s ObjectFiles output property to an MSBuild item named Objects.
The ObjectFiles parameter provides the list of objects produced by cl.exe. By
assigning the list to Objects, the link task can reference Objects as an input.
Since a tool’s flags will be specified through task properties, we need to determine
the granularity of those properties. Consider the many optimization flags: /O1, /O2,
/Ob, /Od, /Og, /Oi, /Os, /Ot, /Ox, and /Oy. We want to represent these 10 flags with
some cl properties. A gut reaction may be to have individual properties for each of
them but that’s not the most efficient or natural. For example, the /Os (favors small
code) and /Ot (favors fast code) compiler flags both affect optimization. However,
you wouldn’t set both independently since you can either favor small code or fast
code, but not both. One option is to use a property like FavorSizeOverSpeed that
adds /Os to the compiler line if set to true or adds /Ot otherwise. Later, I’ll discuss a
more systematic way to determine which properties these tasks should have.
To write a task for cl or link, we’ll need to create classes that represent each tool.
MSBuild provides a very convenient interface for this, known as a ToolTask. The
purpose of a ToolTask is to provide functionality for a task that wraps a command
line tool. As demonstrated above, the properties of a tool task can be used to set
flags for the wrapped tools. MSBuild also provides helper classes for building a
command line to invoke a tool and logging output from the tool. More information on
MSBuild ToolTasks is available here.
For the sample cl and link tasks I wrote, I found most of the work involved storing
data regarding the flags to set since both cl and link have tons of flags. One
approach is to use a table where you relate the task’s property name to a
corresponding flag. If the property is set, you parse its value and add the
appropriate flag to the table.
When you’re ready to build the command line, you can simply run through the table
values. However, you’ll want to maintain the order of flags on the command line.
This makes it easier to verify that an upgraded version of the task hasn’t altered the
way the underlying tool is executed since the flag order would be consistent.
After building the command line, you’ll want to pass the string of flags to the
responseFileCommands argument of the base ExecuteTool method. This allows you
to avoid issues with the command line being too long.
Converting the .vcproj file
We need a way to actually read the .vcproj file and produce a new project file that
MSBuild can understand. The new project file can then import our .targets files which
use our VC++ tasks. For the prototype, I created a console application that could
perform the conversion.
Reading a project file is pretty easy to do using the interfaces in the VCProject and
VCProjectEngine namespaces; these interfaces expose the contents of .vcproj files.
To load a project:
VCProjectEngine vcprojEngine = new VCProjectEngineObject();
VCProject vcproj =
(VCProject)vcprojEngine.LoadProject(projectFileName);
Here, we’re creating a VCProjectEngine and then loading a .vcproj file. In the
prototype, projectFileName was supplied by the user on the command line.
With the project loaded, we can use the properties of the VCProject object to obtain
the various elements in the .vcproj file. For example, to obtain a list of files under
the Source Files filter, you can do:
List<VCFile> files = new List<VCFile>();
IVCCollection fileCollection = (IVCCollection)vcproj.Files;
foreach (VCFile vcFile in fileCollection)
{
if (vcFile.Parent is VCFilter)
{
if (((VCFilter)(vcFile.Parent)).Name.Equals(“Source
Files”))
{
files.Add(vcFile);
}
}
}
Here, we’ve used the project’s Files field to obtain an IVCCollection. An
IVCCollection resides in the VCProjectEngine namespace; it’s different from other
collections since it contains a reference to the VCProjectEngine. A VCFile object
describes the operations that can take place on a file in the .vcproj. Here, we iterate
over the collection of VCFile objects and for those that have a parent that’s a
VCFilter, we check if the filter’s name is Source Files. In that case, we add the
vcFile object to our collection of files.
With a VCProject object, you can obtain collections for all the data in a VC++
project: Configurations, Files, Filters, Platforms, and References.
Additionally, a VCProject object has properties that specify the project’s name, its
GUID, the project directory’s name, the root namespace, and more. These are some
of the properties we’re interested in for our new project’s property group.
To pick up the tools, we need to start by processing each configuration since the
same tool can have different attributes in each configuration. Similar to above, you
can iterate over a list of configurations using:
foreach (VCConfiguraton vcConfig in
((IVCCollection)vcproj.Configurations))
VCConfiguration objects have fields that specify the configuration name
(vcConfig.Name) and access the associated VCPlatform object
(vcConfig.Platform). Using the platform object, you can obtain the platform name
(((VCPlatform)vcConfig.Platform).Name). To get the lists of tools associated with
a configuration, obtain the collection from the Tools field (vcConfig.Tools). Since
there’s a defined list of tools, it’s easiest to just specify the tool you want from the
collection. For cl.exe:
vcCompilerTool =
(VCCLCompilerTool)(vcTools.Item(“VCCLCompilerTool”));
Similarly for link.exe:
vcLinkerTool = (VCLinkerTool)(vcTools.Item(“VCLinkerTool”));
In both examples, vcTools is an IVCCollection from vcConfig.Tools.
With the VCCLCompilerTool and VCLinkerTool objects, it’s now possible to access
the related attributes in the .vcproj file. This enables us to take the data from the
.vcproj file and write it out to a new project file.
Once our converter application has stored all the .vcproj data, it’s time to produce an
MSBuild project file that also contains this data. This is where we’ll tie the attributes
of the cl and link tasks to the fields in the corresponding tool objects. Before that,
however, let’s begin by producing the initial properties for the project file.
Since I didn’t find any classes like VCProjectEngine for MSBuild, I chose to generate
the project file with an XmlTextWriter object. To make the XML easier to read, I set
the writer’s formatting field to Formatting.Indented. Note that I’m writing the
project properties and the actual tasks in the project files. Typically, the tasks would
be in their own .targets file(s) but my approach was more convenient for testing
since I could programmatically reflect changes to task attributes instead of needing
to manually edit the .targets files. As I describe the elements I wrote to the project
file, I’ll provide sample code so you can see which fields I used to obtain the data.
The first step is to write out the required MSBuild header, which is present in any
MSBuild project file:
xmlw.WriteStartElement("Project");
xmlw.WriteAttributeString("xmlns",
"http://schemas.microsoft.com/developer/msbuild/2003");
The next step is to add UsingTask elements to specify the tasks being used and the
assembly file they’re in.
xmlw.WriteStartElement("UsingTask");
xmlw.WriteAttributeString("TaskName", "cl");
xmlw.WriteAttributeString("AssemblyFile",
"VCProjTasks.dll");
xmlw.WriteEndElement();
xmlw.WriteStartElement("UsingTask");
xmlw.WriteAttributeString("TaskName", "link");
xmlw.WriteAttributeString("AssemblyFile",
"VCProjTasks.dll");
xmlw.WriteEndElement();
Following this is a PropertyGroup for the properties common to the entire project.
This is where we’ll list the Product GUID, the project name, the root namespace, and
the schema version:
xmlw.WriteStartElement("PropertyGroup");
xmlw.WriteElementString("ProjectGuid", vcproj.ProjectGUID);
xmlw.WriteElementString("ProjectName", vcproj.Name);
xmlw.WriteElementString("RootNamespace", vcproj.RootNamespace);
xmlw.WriteElementString("SchemaVersion",
Environment.Version.ToString(2));
xmlw.WriteEndElement();
Next, add an MSBuild ItemGroup that specifies the files to be compiled:
xmlw.WriteStartElement("ItemGroup");
foreach (VCFile vcf in files)
{
xmlw.WriteStartElement("FilesToCompile");
xmlw.WriteAttributeString("Include", vcf.Name);
xmlw.WriteEndElement();
}
xmlw.WriteEndElement();
At this point, we’re ready to write the build configuration properties. To start, we
need to print the PropertyGroup element, but this time with an attribute that tests a
condition for the configuration’s name and platform. Next, we’ll write configuration
specific data that’s not tied to a tool. Using the Configurations field from our
VCProject object, we’ll examine each VCConfiguration. The VCConfiguration
object has fields for the IntermediateDirectory, the OutputDirectory, and the
output path using PrimaryOutput.
We can use the configuration’s Tools collection to obtain references to
VCCLCompilerTool and VCLinkerTool. With these references, we’ll begin writing cl
and link specific properties to the file. Additionally, we can use the fields of these
objects to drive the attributes we need for the cl and link tasks. Specifically, we’ll
have an attribute for each property we write in the project file.
For writing the properties, I created helper methods that invoked
WriteElementString. For the cl task, the method looked like:
private void WriteCLProperty(string property, string value)
{
// Don't print properties with no values.
if (!String.IsNullOrEmpty(value))
{
xmlw.WriteElementString("CL" + property, value);
}
}
For each cl property I wanted to track, I’d invoke this method specifying the property
name and the field of the VCCLCompilerTool that contained its value. For example:
WriteCLProperty("AdditionalIncludeDirectories",
vcCompilerTool.AdditionalIncludeDirectories);
AdditionalIncludeDirectories is a simple example since I’m literally printing the
value of a string. Similarly, for boolean fields, I invoked ToString() after the field to
obtain a true or false string.
There are some more complicated situations which require additional work. One
example is with the ExceptionHandling field, which provides a value from an
enumeration that contains the possible values. To handle this scenario, I cast the
value of ExceptionHandling to an Int32. With this integer, I indexed a string array,
excepHandlingParamMap, of easily readable2 string representations of the different
options. The cl task can then convert the string into the appropriate flag. For
example:
static readonly string[] excepHandlingParamMap = { "Disabled",
"CPP", "ASync" };
WriteCLProperty("ExceptionHandling",
GetValueString((Int32)vcclTool.ExceptionHandling,
excepHandlingParamMap));
GetValueString returns the appropriate value from the array given the array and
the index. I used a helper method to ensure the index is never out of bounds for the
array.
The Linker flags followed a similar pattern and once all the properties had been
written, I had a property group which looked like:
<PropertyGroup Condition="'$(ConfigurationName)|$(PlatformName)' ==
'Debug|Win32'">
<ProjectDir>c:\Documents and Settings\clichten\My Documents\Visual
Studio 2005\CPXSolutions\Investigations\VCP2MSB\bin\Debug\
</ProjectDir>
<IntDir>$(ConfigurationName)</IntDir>
<OutDir>$(ProjectDir)$(ConfigurationName)</OutDir>
<TargetDir>c:\documents and settings\clichten\my documents\visual
studio 2005\cpxsolutions\investigations\vcp2msb\bin\debug\
</TargetDir>
<TargetFileName>TestVCProj.exe</TargetFileName>
<TargetName>TestVCProj</TargetName>
<CLAnalyze>False</CLAnalyze>
<CLBufferSecurityCheck>True</CLBufferSecurityCheck>
<CLCallingConvention>CDecl</CLCallingConvention>
<CLCompileAsManaged>true</CLCompileAsManaged>
...
<LinkSetChecksum>False</LinkSetChecksum>
<LinkShowProgress>Disabled</LinkShowProgress>
<LinkSubsystem>NotSet</LinkSubsystem>
2
Readability is important since MSBuild project files should be human readable.
<LinkTargetMachine>X86</LinkTargetMachine>
</PropertyGroup>
These property groups should be created for all relevant build configurations.
Finally, we need to write out a target element which will invoke our tasks. For the
prototype, I created a target named BuildVCProj that invoked the cl and link tasks:
xmlw.WriteStartElement("Target");
xmlw.WriteAttributeString("Name", "BuildVCProj");
{
WriteCLTask();
WriteLinkTask();
}
xmlw.WriteEndElement();
WriteCLTask and WriteLinkTask print the actual cl and link tasks, along with
assigning attributes to the matching property. The task output matches the cl and
link tasks that I presented above.
With all these pieces, you can convert a .vcproj file to a MSBuild project file and build
that project using your cl and link tasks. The next step would be to add additional
tasks for other VC++ tools to cover additional build scenarios.