December 2016
Volume 31 Number 13
[Roslyn]
Generate JavaScript with Roslyn and T4 Templates
By Nick Harrison | December 2016
The other day my daughter told me a joke about a conversation between a smartphone and a feature phone. It goes something like this: What did the smartphone say to the feature phone? “I’m from the future, can you understand me?” Sometimes it feels like that when learning something new and cutting edge. Roslyn is from the future and can be difficult to understand at first.
In this article, I’ll discuss Roslyn in a way that might not get as much focus as it deserves. I’ll focus on using Roslyn as a source of metadata for generating JavaScript with T4. This will use the Workspace API, some of the Syntax API, the Symbol API and a runtime template from the T4 engine. The actual JavaScript generated is secondary to understanding the processes used to gather the metadata.
Because Roslyn also provides some nice options for generating code, you might think that the two technologies would clash and not work well together. Technologies often clash when their sandboxes overlap, but these two technologies can play together rather nicely.
Wait, What’s T4?
If T4 is new to you, a 2015 e-book from the Syncfusion Succinctly series, “T4 Succinctly,” provides all the background you need (bit.ly/2cOtWuN).
For now, the main thing to know is that T4 is Microsoft’s template-based text transformation toolkit. You feed metadata to the template and the text becomes the code you want. Actually, you aren’t limited to code. You can generate any type of text, but source code is the most common output. You can generate HTML, SQL, text documentation, Visual Basic .NET, C# or any text-based output.
Look at Figure 1. It shows a simple Console Application program. In Visual Studio, I added a new runtime text template named AngularResourceService.tt. The template code automatically generates some C# code that’ll implement the template at run time, which you can see in the console window.
.png "Using T4 for Design-Time Code Generation")
Figure 1 Using T4 for Design-Time Code Generation
In this article, I’ll show you how to use Roslyn to gather metadata from a Web API project to feed to T4 to generate a JavaScript class and then use Roslyn to add that JavaScript back to the solution.
Conceptually, the process flow will look like Figure 2.
.png "T4 Process Flow")
Figure 2 T4 Process Flow
Roslyn Feeds T4
Generating code is a metadata-hungry process. You need metadata to describe the code you want generated. Reflection, Code Model and the Data Dictionary are common sources of readily available metadata. Roslyn can provide all the metadata you would’ve received from Reflection or the Code Model but without some of the problems these other approaches incur.
In this article, I’ll use Roslyn to find classes derived from ApiController. I’ll use the T4 template to create a JavaScript class for each Controller and expose a method for each Action and a property for each property in the ViewModel associated with the Controller. The result will look like the code in Figure 3.
Figure 3 Result of Running the Code
var app = angular.module("challenge", [ "ngResource"]);
app.factory(ActivitiesResource , function ($resource) {
return $resource(
'https://localhost:53595//Activities',{Activities : '@Activities'},{
Id : "",
ActivityCode : "",
ProjectId : "",
StartDate : "",
EndDate : "",
, get: {
method: "GET"
}
, put: {
method: "PUT"
}
, post: {
method: "POST"
}
, delete: {
method: "DELETE"
}
});
});
Gathering the Metadata
I start gathering metadata by creating a new console application project in Visual Studio 2015. In this project, I’ll have a class devoted to gathering metadata with Roslyn, as well as a T4 template. This will be a runtime template that will generate some JavaScript code based on the gathered metadata.
Once the project is created, the following commands from the Package Manager Console are issued:
Install-Package Microsoft.CodeAnalysis.CSharp.Workspaces
This ensures the latest Roslyn code for the CSharp compiler and related services are being used.
I place the code for the various methods in a new class called RoslynDataProvider. I’ll refer to this class throughout the article and it’ll be a handy reference whenever I want to gather metadata with Roslyn.
I use the MSBuildWorksspace to get a workspace that will provide all the context needed for the compilation. Once I have the solution, I can easily walk through the projects looking for the WebApi project:
private Project GetWebApiProject()
{
var work = MSBuildWorkspace.Create();
var solution = work.OpenSolutionAsync(PathToSolution).Result;
var project = solution.Projects.FirstOrDefault(p =>
p.Name.ToUpper().EndsWith("WEBAPI"));
if (project == null)
throw new ApplicationException(
"WebApi project not found in solution " + PathToSolution);
return project;
}
If you follow a different naming convention, you can easily incorporate it into the GetWebApiProject to find the project in which you’re interested.
Now that I know which project I want to work with, I need to get the compilation for that project, as well as a reference to the type that I’ll use to identify the controllers of interest. I need the compilation because I’ll use the SemanticModel to determine whether a class derives from System.Web.Http.ApiController. From the project, I can get the documents included in the project. Each document is a separate file, which could include more than one class declaration, although it’s a good best practice to only include a single class in any file and have the name of the file match the name of the class; but not everyone follows this standard all the time.
Finding the Controllers
Figure 4 shows how to find all of the class declarations in every document and determine if the class is derived from ApiController.
Figure 4 Finding the Controllers in a Project
public IEnumerable<ClassDeclarationSyntax> FindControllers(Project project)
{
compilation = project.GetCompilationAsync().Result;
var targetType = compilation.GetTypeByMetadataName(
"System.Web.Http.ApiController");
foreach (var document in project.Documents)
{
var tree = document.GetSyntaxTreeAsync().Result;
var semanticModel = compilation.GetSemanticModel(tree);
foreach (var type in tree.GetRoot().DescendantNodes().
OfType<ClassDeclarationSyntax>()
.Where(type => GetBaseClasses(semanticModel, type).Contains(targetType)))
{
yield return type;
}
}
}
Because the compilation has access to all of the references needed to compile the project, it will have no problem resolving the target type. When I get the compilation object, I’ve started compiling the project, but am interrupted part way through once I have the details to get the needed metadata.
Figure 5 shows the GetBaseClasses method that does the heavy lifting for determining if the current class derives from the target class. This does a bit more processing than is strictly needed. To determine whether a class is derived from ApiController, I don’t really care about the interfaces implemented along the way, but by including these details, this becomes a handy utility method that can be used in a wide variety of places.
Figure 5 Finding Base Classes and Interfaces
public static IEnumerable<INamedTypeSymbol> GetBaseClasses
(SemanticModel model, BaseTypeDeclarationSyntax type)
{
var classSymbol = model.GetDeclaredSymbol(type);
var returnValue = new List<INamedTypeSymbol>();
while (classSymbol.BaseType != null)
{
returnValue.Add(classSymbol.BaseType);
if (classSymbol.Interfaces != null)
returnValue.AddRange(classSymbol.Interfaces);
classSymbol = classSymbol.BaseType;
}
return returnValue;
}
This type of analysis gets complicated with Reflection because a reflective approach will rely on recursion and potentially needing to have to load any number of assemblies along the way to get access to all of the intervening types. This type of analysis isn’t even possible with the Code Model, but is relatively straightforward with Roslyn using the SemanticModel. The SemanticModel is a treasure trove of metadata; it represents everything the compiler knows about the code after going through the trouble of binding the syntax trees to symbols. In addition to tracking down base types, it can be used to answer tough questions like overload/override resolution or finding all references to a method or property or any Symbol.
Finding the Associated Model
At this point, I have access to all the Controllers in the project. In the JavaScript class, it’s also nice to expose the properties found in the Models returned by the Actions in the Controller. To understand how this works, take a look at the following code, which shows the output from running scaffolding for a WebApi:
public class Activity
{
public int Id { get; set; }
public int ActivityCode { get; set; }
public int ProjectId { get; set; }
public DateTime StartDate { get; set; }
public DateTime EndDate { get; set; }
}
In this case, the scaffolding was run against the Models, as shown in Figure 6.
Figure 6 Generated API Controller
public class ActivitiesController : ApiController
{
private ApplicationDbContext db = new ApplicationDbContext();
// GET: api/Activities
public IQueryable<Activity> GetActivities()
{
return db.Activities;
}
// GET: api/Activities/5
[ResponseType(typeof(Activity))]
public IHttpActionResult GetActivity(int id)
{
Activity activity = db.Activities.Find(id);
if (activity == null)
{
return NotFound();
}
return Ok(activity);
}
// POST: api/Activities
[ResponseType(typeof(Activity))]
public IHttpActionResult PostActivity(Activity activity)
{
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
db.Activities.Add(activity);
db.SaveChanges();
return CreatedAtRoute("DefaultApi", new { id = activity.Id }, activity);
}
// DELETE: api/Activities/5
[ResponseType(typeof(Activity))]
public IHttpActionResult DeleteActivity(int id)
{
Activity activity = db.Activities.Find(id);
if (activity == null)
{
return NotFound();
}
db.Activities.Remove(activity);
db.SaveChanges();
return Ok(activity);
}
The ResponseType attribute added to the actions will link the ViewModel to the Controller. Using this attribute, you can get the name of the model associated with the action. As long as the Controller was created using scaffolding, then every action will be associated with the same model, but Controllers created by hand or edited after being generated might not be so consistent. Figure 7 shows how to compare against all actions to get a complete list of the models associated with a Controller in case there’s more than one.
Figure 7 Finding Models Associated with a Controller
public IEnumerable<TypeInfo> FindAssociatedModel
(SemanticModel semanticModel, TypeDeclarationSyntax controller)
{
var returnValue = new List<TypeInfo>();
var attributes = controller.DescendantNodes().OfType<AttributeSyntax>()
.Where(a => a.Name.ToString() == "ResponseType");
var parameters = attributes.Select(a =>
a.ArgumentList.Arguments.FirstOrDefault());
var types = parameters.Select(p=>p.Expression).OfType<TypeOfExpressionSyntax>();
foreach (var t in types)
{
var symbol = semanticModel.GetTypeInfo(t.Type);
if (symbol.Type.SpecialType == SpecialType.System_Void) continue;
returnValue.Add( symbol);
}
return returnValue.Distinct();
}
There’s interesting logic going on in this method; some of it’s rather subtle. Remember what the ResponseType attribute looks like:
[ResponseType(typeof(Activity))]
I want to access the properties in the type referenced in the type of expression, which is the first parameter to the attribute—in this case, Activity. The attributes variable is a list of the ResponseType attributes found in the controller. The parameters variable is a list of the parameters to these attributes. Each of these parameters will be a TypeOfExpressionSyntax, and I can get the associated type through the type property of the TypeOfExpressionSyntax objects. Once again, the SemanticModel is used to pull in the Symbol for that type, which will give all the details you could want.
Distinct at the end of the method will ensure that each model returned is unique. In most circumstances, you’d expect to have duplicates because multiple actions in the Controller will be associated with the same model. It’s also a good idea to check against the ResponseType being void. You won’t find any interesting properties there.
Examining the Associated Model
The following code shows how to find the properties from all of the models found in the Controller:
public IEnumerable<ISymbol> GetProperties(IEnumerable<TypeInfo> models)
{
return models.Select(typeInfo => typeInfo.Type.GetMembers()
.Where(m => m.Kind == SymbolKind.Property))
.SelectMany(properties => properties).Distinct();
}
Finding the Actions
In addition to showing the properties from the associated Models, I want to include references to the methods that are in the Controller. The methods in a Controller are Actions. I’m only interested in the Public methods, and because these are WebApi Actions, they should all be translated to the appropriate HTTP Verb.
There are a couple of different conventions followed for handling this mapping. The one followed by the scaffolding is for the method name to start with the verb name. So the put method would be PutActivity, the post method would be PostActivity, the delete method would be DeleteActivity, and there will generally be two get methods: GetActivity and GetActivities. You can tell the difference between the get methods by examining the return types for these methods. If the return type directly or indirectly implements the IEnumerable interface, the get method is the get all; otherwise, it’s the get single item method.
The other approach is that you explicitly add attributes to specify the verb, then the method could have any name. Figure 8 shows the code for GetActions that identifies the public methods and then maps them to verbs using both methods.
Figure 8 Finding the Actions on a Controller
public IEnumerable<string> GetActions(ClassDeclarationSyntax controller)
{
var semanticModel = compilation.GetSemanticModel(controller.SyntaxTree);
var actions = controller.Members.OfType<MethodDeclarationSyntax>();
var returnValue = new List<string>();
foreach (var action in actions.Where
(a => a.Modifiers.Any(m => m.Kind() == SyntaxKind.PublicKeyword)))
{
var mapName = MapByMethodName(semanticModel, action);
if (mapName != null)
returnValue.Add(mapName);
else
{
mapName = MapByAttribute(semanticModel, action);
if (mapName != null)
returnValue.Add(mapName);
}
}
return returnValue.Distinct();
}
The GetActions method first tries to map based on the name of the method. If that doesn’t work, it’ll then try to map by attributes. If the method cannot be mapped, then it won’t be included in the list of actions. If you have a different convention that you want to check against, you can easily incorporate it into the GetActions method. Figure 9 shows the implementations for the MapByMethodName and MapByAttribute methods.
Figure 9 MapByName and MapByAttribute
private static string MapByAttribute(SemanticModel semanticModel,
MethodDeclarationSyntax action)
{
var attributes = action.DescendantNodes().OfType<AttributeSyntax>().ToList();
if ( attributes.Any(a=>a.Name.ToString() == "HttpGet"))
return IdentifyIEnumerable(semanticModel, action) ? "query" : "get";
var targetAttribute = attributes.FirstOrDefault(a =>
a.Name.ToString().StartsWith("Http"));
return targetAttribute?.Name.ToString().Replace("Http", "").ToLower();
}
private static string MapByMethodName(SemanticModel semanticModel,
MethodDeclarationSyntax action)
{
if (action.Identifier.Text.Contains("Get"))
return IdentifyIEnumerable(semanticModel, action) ? "query" : "get";
var regex = new Regex("\b(?'verb'post|put|delete)", RegexOptions.IgnoreCase);
if (regex.IsMatch(action.Identifier.Text))
return regex.Matches(action.Identifier.Text)[0]
.Groups["verb"].Value.ToLower();
return null;
}
Both methods start by explicitly searching for the Get Action and determining which type of “get” to which the method refers.
If the action isn’t one of the “gets,” MapByAttribute checks to see if the action has an attribute that starts with Http. If one’s found, then the verb can be determined by simply taking the attribute name and removing Http from the attribute name. There’s no need to check explicitly against each attribute to determine which verb to use.
MapByMethodName is structured similarly. After first checking for a Get action, this method uses a regular expression to see if any of the other verbs match. If a match is found, you can get the verb name from the named capture group.
Both of the mapping methods need to differentiate between the Get Single and Get All Actions and they both use the IdentifyEnumerable method shown in the following code:
private static bool IdentifyIEnumerable(SemanticModel semanticModel,
MethodDeclarationSyntax actiol2n)
{
var symbol = semanticModel.GetSymbolInfo(action.ReturnType);
var typeSymbol = symbol.Symbol as ITypeSymbol;
if (typeSymbol == null) return false;
return typeSymbol.AllInterfaces.Any(i => i.Name == "IEnumerable");
}
Again, the SemanticModel plays a pivotal role. I can differentiate between the get methods by examining the return type of the method. The SemanticModel will return the symbol bound to the return type. With this symbol, I can tell whether the return type implements the IEnumerable interface. If the method returns a List<T> or an Enumerable<T> or any type of collection, it’ll implement the IEnumerable interface.
The T4 Template
Now that all of the metadata is gathered, it’s time to visit the T4 template that will tie all these pieces together. I start by adding a Runtime Text Template to the project.
For a Runtime Text Template, the output of running the template will be a class that will implement the template that’s defined and not the code I want to produce. For the most part, anything you can do in a Text Template you can do with a Runtime Text Template. The difference is how you run the template to generate code. With a Text Template, Visual Studio will handle running the template and creating the hosting environment in which the template will run. With a Runtime Text Template, you’re responsible for setting up the hosting environment and running the template. This puts more work on you, but it also gives you a great deal more control over how you run the template and what you do with the output. It also removes any dependencies on Visual Studio.
I start by editing the AngularResource.tt and adding the code in Figure 10 to the template.
Figure 10 Initial Template
<#@ template debug="false" hostspecific="false" language="C#" | #>
var app = angular.module("challenge", [ "ngResource"]);
app.factory(<#=className #>Resource . function ($resource) {
return $resource('<#=Url#>/<#=className#>',{<=className#> : '@<#=className#>'},{
<#=property.Name#> : "",
query : {
method: "GET"
, isArray : true
}
' <#=action#>: {
method: "<#= action.ToUpper()#>
}
});
});
Depending on how familiar you are with JavaScript, this might be new to you—if so, don’t worry.
The first line is the template directive and it tells T4 that I’ll be writing template code in C#; the other two attributes are ignored for Runtime templates, but for the sake of clarity, I’m explicitly stating that I have no expectation from the hosting environment and don’t expect the intermediate files to be preserved for debugging.
The T4 template is a bit like an ASP page. The <# and #> tags delimit between code to drive the template and text to be transformed by the template. The <#= #> tags delimit a variable substitution that is to be evaluated and inserted into the generated code.
Looking at this template, you can see that the metadata is expected to provide a className, URL, a list of properties and a list of actions. Because this is a Runtime template there are a couple of things that I can do to simplify matters, but first take a look at the code that’s created when this template is run, which is done by saving the .TT file or by right-clicking on the file in Solution Explorer and selecting Run Custom Tool.
The output from running the template is a new class, which matches the template. More important, if I scroll down, I’ll find that the template also generated the base class. This is important because if I move the base class to a new file and explicitly state the base class in the template directive, it’ll no longer be generated and I’m free to change this base class as needed.
Next, I’ll change the template directive to this:
<#@ template debug="false" hostspecific="false" language="C#"
inherits="AngularResourceServiceBase" #>
Then I’ll move the AngularResourceServiveBase to its own file. When I run the template again, I’ll see that the generated class still derives from the same base class, but it was no longer generated. Now I’m free to make any changes needed to the base class.
Next, I’ll add a few new methods and a couple of properties to the base class to make it easier to provide the metadata to the template.
To accommodate the new methods and properties, I’ll also need a few new using statements:
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp.Syntax;
I’ll add properties for the URL and for the RoslynDataProvider that I created at the beginning of the article:
public string Url { get; set; }
public RoslynDataProvider MetadataProvider { get; set; }
With these pieces in place, I’ll also need a couple of methods that will interact with the MetadataProvider, as shown in Figure 11.
Figure 11 Helper Methods Added to AngularResourceServiceBase
public IList<ClassDeclarationSyntax> GetControllers()
{
var project = MetadataProvider.GetWebApiProject();
return MetadataProvider.FindControllers(project).ToList();
}
protected IEnumerable<string> GetActions(ClassDeclarationSyntax controller)
{
return MetadataProvider.GetActions(controller);
}
protected IEnumerable<TypeInfo> GetModels(ClassDeclarationSyntax controller)
{
return MetadataProvider.GetModels(controller);
}
protected IEnumerable<ISymbol> GetProperties(IEnumerable<TypeInfo> models)
{
return MetadataProvider.GetProperties(models);
}
Now that I have these methods added to the base class, I’m ready to extend the template to use them. Look at how the template changes in Figure 12.
Figure 12 Final Version of the Template
<#@ template debug="false" hostspecific="false" language="C#" inherits="AngularResourceServiceBase" #>
var app = angular.module("challenge", [ "ngResource"]);
<#
var controllers = GetControllers();
foreach(var controller in controllers)
{
var className = controller.Identifier.Text.Replace("Controller", "");
#> app.facctory(<#=className #>Resource , function ($resource) {
return $resource('<#=Url#>/<#=className#>',{<#=className#> : '@<#=className#>'},{
<#
var models= GetModels(controller);
var properties = GetProperties(models);
foreach (var property in properties)
{
#>
<#=property.Name#> : "",
<#
}
var actions = GetActions(controller);
foreach (var action in actions)
{
#>
<#
if (action == "query")
{
#> query : {
method: "GET"
Running the Template
Because this is a Runtime template, I’m responsible for setting up the environment for running the template. Figure 13 shows the code needed to run the template.
Figure 13 Running a Runtime Text Template
private static void Main()
{
var work = MSBuildWorkspace.Create();
var solution = work.OpenSolutionAsync(Path to the Solution File).Result;
var metadata = new RoslynDataProvider() {Workspace = work};
var template = new AngularResourceService
{
MetadataProvider = metadata,
Url = @"https://localhost:53595/"
};
var results = template.TransformText();
var project = metadata.GetWebApiProject();
var folders = new List<string>() { "Scripts" };
var document = project.AddDocument("factories.js", results, folders)
.WithSourceCodeKind(SourceCodeKind.Script)
;
if (!work.TryApplyChanges(document.Project.Solution))
Console.WriteLine("Failed to add the generated code to the project");
Console.WriteLine(results);
Console.ReadLine();
}
The class created when I save the template or run the custom tool can be instantiated directly and I can set or access any public properties or call any public methods from the base class. This is how the values for the properties are set. Calling the TransformText method will run the template and return the generated code as a string. The results variable will hold the generated code. The rest of the code deals with adding a new document to the project with the code that was generated.
There’s a problem with this code, however. The call to AddDocuments successfully creates a document and places it in the scripts folder. When I call the TryApplyChanges, it returns successful. The problem comes when I look in the solution: There’s a factories file in the scripts folder. The problem is that instead of factories.js, it’s factories.cs. The AddDocument method isn’t configured to accept an extension. Regardless of the extension, the document will be added based on the type of project to which it’s added. This is by design.
Therefore, after the program runs and generates the JavaScript classes, the file will be in the scripts folder. All I have to do is change the extension from .cs to .js.
Wrapping Up
Most of the work done here centered on getting metadata with Roslyn. Regardless of how you plan to use this metadata, these same practices will come in useful. As for the T4 code, it’ll continue to be relevant in a variety of places. If you want to generate code for any language not supported by Roslyn, T4 is a great choice and easy to incorporate into your processes. This is good because you can use Roslyn to generate code for only C# and Visual Basic .NET while T4 lets you generate any type of text, which could be SQL, JavaScript, HTML, CSS, or even plain-old text.
It’s nice to generate code like these JavaScript classes because they’re tedious and error-prone. They also easily conform to a pattern. As much as is feasible, you want to follow that pattern as consistently as possible. Most important, the way you want that generated code to look is likely to change over time, especially for something new, as best practices are formed. If all you have to do is update a T4 template to change to the new “best way to do it,” you’re more likely to adhere to the emerging best practices; but if you have to modify large amounts of hand-generated monotonous, tedious code, you’ll probably have multiple implementation as each iteration of the best practice was in vogue.
Nick Harrison is a software consultant living in Columbia, S.C., with his loving wife Tracy and daughter. He’s been developing full stack using .NET to create business solutions since 2002. Contact him on Twitter: @Neh123us, where he also announces his blog posts, published articles and speaking engagements.
Thanks to the following Microsoft technical expert for reviewing this article: James McCaffrey
Dr. James McCaffrey works for Microsoft Research in Redmond, Wash. He has worked on several Microsoft products including Internet Explorer and Bing. Dr. McCaffrey can be reached at jammc@microsoft.com.