The demo featured a chip called the Spark Core (https://www.spark.io/). This remarkable chip contains a micro controller like the one found in the Arduino, coupled with some very important extras: One is a WiFi chip directly built into the chip, and the other is infrastructure in the cloud that allows some very interesting scenarios. In addition, the chip has several digital input / output pins (that can be used to turn things on or off, or to sense things in the world surrounding it), and analog pins, that can measure a voltage.

In the demo I showed a thermometer implementation where a DHT11 (http://www.adafruit.com/products/386) chip is used to measure temperature and relative humidity. This chip is a little tricky to interface to the spark, but there are several other sensors that are quite a bit easier to deal with (typically simply providing an analog voltage that can be measured with the spark's analog data pins: One example is the LM35 chip  (http://www.ti.com/product/lm35)). I wired up the D4 data pin to the DHT11, and put in some code (in C++) that gets the data. The code is maintained (edited, compiled and stored) entirely in the cloud: No complicated tool chain is needed on your own box. When the code is compiled successfully, it may be flashed onto the chip over the air (i.e. through the wireless connection). From that point on, it will run this code from until it is switched off. Since the code is downloaded into ROM, it will start right up after being turned on again.

The driver part of the code I used is enclosed. In the true spirit of the maker movement, I downloaded the library that takes care of the details of talking to the DHT11 from the web from here: https://github.com/adafruit/DHT-sensor-library. I show the main part of the code for reference below:

To aid debugging (which is otherwise difficult) I added a serial connection to my laptop and installed a USB to serial converted. I use putty (http://www.putty.org/) to maintain the connection to the chip, and to show the debug output.

When it is running the chip communicates with a service that the chip manufacturer runs (and you can now run it locally if you wish). The communication is initiated by the chip. Whenever a consumer needs to access the data it does so by asking the service in the cloud – No consumer has any idea on which IP address the chip is located. This means that there are no problems with NATs, or using home networks where the IP address may change unpredictably etc. The experience is very smooth, and safe, since all communication is secured. This is the value of the infrastructure provided by more and more providers.

Let’s get back to the code: When the code is running, the values measured are available in some variables: temperature, humidity and heatIndex. Since these are set up as variables in the code below:

Spark.variable("temperature", &temperature, DOUBLE);
Spark.variable("humidity", &humidity, DOUBLE);
Spark.variable("heatindex", &heatIndex, DOUBLE);

They are available to be read at any time from interested code from anywhere on the internet by making simple calls where the URL contains the name of the required variable. The resulting JSON payload will contain the value. It is that simple. There are other possibilities as well: You can ask to have a function executed, and you can ask for the values of all variables at once, but the results are invariantly returned as a JSON document:

Let me show some code that does this in X++:

{
    str sparkCoreName = "";
    str accessToken = "";
    str variableName = "temperature";
    TextBuffer tb = new TextBuffer();
    str values, temp;
    int digits;
    int ptr;
    real rr;
    System.Net.WebClient webclient;
    str uri;
    
    boolean IsDigit(str s1)
    {
     return s1 >= "0" && s1 <= '9';
    }
    
    webclient = new System.Net.WebClient();
    uri = strfmt(“https://api.spark.io/v1/devices/%1/%2?access_token=%3”,
        sparkCoreName, variableName, accessToken);
    
    // Get the value from the server given the URL.
    values = webclient.DownloadString(uri);
    
    // Find the result value from the JSON
    ptr = strScan(values, '"result": ', 1, strLen(values));
    ptr += strLen('"result": ');
    
    // now read while we are looking at digits.
    digits = 0;
    while(IsDigit(subStr(values, ptr + digits, 1)))
    { 
        digits += 1;
    }
    
    rr = str2num(subStr(values, ptr, digits)); 
    info( values);
}

This code is not brilliant by any means, but it works. In production code I would use a JSON library to parse the JSON document, and access the payload data (the result field), but for this demo I did not bother. Note that I have omitted the values that identify your spark chip from the code above.

In the demo I wanted a little more pizzazz so I integrated it with a gauge control on a form. I will leave this as an exercise to the reader…

The Microsoft Dynamics AX 2012 system allows an individual table field, in any one record, to contain multiple values. Typically this is done by creating an extended data type (EDT) that is an array type, and then using the EDT as the type for a field in a table. The most well-known case is the Dimension field of the EDT named Dimension. The Dimension type is shown in the AOT image, along with its array elements.

This is a flexible and dynamic feature. As soon as the user adds a new element to the Dimension type’s list of array elements, everything immediately lights up with the new dimension everywhere the type is used.

1.  X++ Support

The X++ language supports these array fields without problems. Let us imagine that the table called PVsTable has a field called DimensionField of type Dimension. The following are legal X++ code snippets: 

   The first select statement selects all the array elements of the DimensionField, while the second fetches only the array element for index 2. In all code, values of type Dimension are indexable by an integer value.

2.  Business Connector: Challenge and Trick

There is no API that enables you to get the value of an individual array element from the DimensionField field through the .NET Business Connector of AX. Instead, you use a little trick to get individual elements. Let’s set the stage with a little sample C# code:

As you can see above, the get_Field method on the record does not actually get the array of values, but merely the first value in the array; which is at index 1. It can certainly be questioned whether this is a reasonable choice or not, but that is the behavior in place since the very first versions of the business connector.

3.  Access any Individual Element

If you want to access individual elements, you have to append the index you want by using a syntax borrowed from array indexing:

4.  Proxies

A proxy is a C# class which represents an AX object such as a class or table.  The AX plug-in for Visual Studio generates the proxy code when you drag a class or table from the Application Explorer onto your project in the Solution Explorer. Your project must first be added in MorphX under AOT > Visual Studio Projects. The proxies are updated whenever a change is detected in the AX object's schema.

A proxy enables your C# program to interact seamlessly with its corresponding AX object. Internally the proxy uses the .NET Business Connector for AX. You can see the code for a proxy by placing the cursor on an AX object and pressing F12 (or by selecting Go To Definition). You would see proxy code similar to the following:

The proxy uses the business connector to talk to AX, by using the API that we investigated above. As you can see we are always relying on returning and setting the simple value, namely the first value. There are no provisions for getting the complete array, or even for explicitly identifying any individual array element of the DimensionField field. This is obviously unfortunate, since it hurts the C# interoperability story. However, all is not lost.

5.  Add Your Own Partial Class File to Extend the Proxy

The proxy class is declared as partial. Therefore you can add extensions by using your own partial class in the same project. In your added partial class, you add your own properties that access the DimensionField elements. There are at least two ways to accomplish this:

• Add a property that returns the array
• Add properties for the individual array elements

Below we show examples for both of these approaches.

5.1  Fetch an Array of All Elements

In the following C# code example, a string array of all the elements in the DimensionField field is fetched in one call:

5.2  Fetch an Individual Element 

The other option is to get the individual elements by using explicit index values:

 You could even add a little value by using the names of the array elements to reference them in the DimensionField field.

 //End.

In an area known as Operational Research (O.R.), there are many problems and solutions that are applicable to enterprise resource planning (ERP). O.R. began at the end of World War II. Initially it was used to solve logistical problems, and it has expanded to address a broad spectrum of problems. One area of progress for O.R. concerns problems that involve so many possible combinations as potential answers that they cannot be solved by brute force calculation.

This essay helps you to:

• Recognize the types of problems that O.R. can solve; and
• Understand the C# code that you can write to leverage O.R.

The basic installation of Microsoft Dynamics AX 2012 includes a .NET managed library that is extremely well suited for solving the problems in the O.R. domain. Use of the library requires no extra expenses, no extra licenses to accept, and no extra installation steps. The library is called the Microsoft Solver Foundation (MSF). It is not too difficult to negotiate the learning curve, and the results it can provide will impress you.

1  The simple transportation problem

We start the discussion with a very simple problem called the transportation problem. There are a set of producers of beer, and there are a set of consumers of beer. The producers have a production capacity in units of crates per day. The consumers demand a certain amount of beer. There is a cost to transporting the beer from the producers to the consumers. The objective is to arrange the shipping such that the cost is minimized, while still satisfying the demands of consumers and not exceeding the capacity of the producers.

Let’s take a minute to write these rules a little more formally:

 Where:

• cost_i,j — is the (non-negative) cost of transporting one item from i to j; and
• x_i,j — is the number of items to move from i to j; and
• demand_j — is the demand at consuming site j; and
• capacity_i — is the capacity of producing site i.

The result that we are looking for is the matrix x_i,j , which is the number of crates to move from producer i to consumer j.

1.1  OML code model to describe the problem

The MSF library allows you to express the problems you want to solve in a declarative language called Optimization Modeling Language (OML). I stress the point that scripts in this language do not tell the computer how to solve the problem. Instead, OML merely describes the problem to solve. Let’s look at the preceding transportation problem as expressed in OML:

Model[
    Parameters[Sets,Producers,Consumers],
    Parameters[Reals[0,Infinity],
        Capacity[Producers],
        Demand[Consumers],
        Cost[Producers,Consumers]],
    Decisions[Reals[0,Infinity],x[Producers,Consumers],TotalCost],
    Constraints[
        TotalCost == Sum[{i,Producers},{j,Consumers},Cost[i,j]*x[i,j]],
        Foreach[{i,Producers}, Sum[{j,Consumers},x[i,j]]

        Foreach[{j,Consumers}, Sum[{i,Producers},x[i,j]]>=Demand[j]]
    ],
    Goals[Minimize[TotalCost]]
]

If you compare the OML to the original problem formulation in mathematical notation, you will find many similarities. The results we are looking for are the decisions. The input parameters for the model are the costs, demands, and capacities. The constraints describe what the solution must satisfy. The goal states that we wish to minimize the total cost.

1.2  Paste the OML code as a string in C#

A C# method to load this model would look something like:

public class TransportationModel   // C#
{
    ///
    /// Create model by adding decisions, constraints and goals. This version works by
    /// passing the OML model (as a string) to the LoadModel method that will then
    /// create the model. The data is bound to the model thereafter.
    ///
    ///The solver context from which the model is created.
    /// The model, ready for solving after data binding.
    public Model CreateModelFromOML(SolverContext context)
    {
        string modelSource = @"
        Model[
            Parameters[Sets,Plants,Markets],
            Parameters[Reals,Capacity[Plants],Demand[Markets],Cost[Plants,Markets]],
            Decisions[Reals[0,Infinity],x[Plants,Markets],TotalCost],
            Constraints[
                TotalCost == Sum[{i,Plants},{j,Markets},Cost[i,j]*x[i,j]],
                Foreach[{i,Plants}, Sum[Foreach[{j,Markets},x[i,j]]],
                Foreach[{j,Markets}, Sum[{i,Plants},x[i,j]]>=Demand[j]] // Shorthand form.
            ],
            Goals[Minimize[TotalTransportationCost -> TotalCost]]
        ]";

        context.LoadModel(FileFormat.OML, new StringReader(modelSource));
        return context.CurrentModel;
    }
    ...
}

You will need to add a reference to the Microsoft.Solver.Foundation.dll, typically found in the …\Client\bin directory.

Before we continue discussing the solver and its context parameter, let’s mention data binding. You see no data binding in the CreateModelFromOML method. In other words, there is no declaration of the costs, demands, and capacities. We see that the OML model defines only the model of the problem, and that the OML does not define the data. We need another way to provide data.

1.3  Data binding for the OML model

First, here is how we would call the preceding CreateModelFromOML method:

namespace TransportationDemoSupport
{
    using Microsoft.SolverFoundation.Services;
    using Microsoft.SolverFoundation.Common;

    class Program
    {
        static void Main(string[] args)
        {
            var tp = new TransportationModel();
            SolverContext context = new SolverContext();

            Model model = tp.CreateModelFromOML(context);
            if (model != null)
            {
            tp.DataBind(model);

            var solution = context.Solve();
            var report = solution.GetReport(ReportVerbosity.All);
            }
        }
    }
}

Next we must write the DataBind method. We must provide values for the parameters, so let’s start by retrieving the values from the model we defined in OML:

public Model DataBind(Model model)
    {
    var demandParameter = model.Parameters.Where(
        p => p.Name.StartsWith("Demand")).First();
    var capacityParameter = model.Parameters.Where(
        p => p.Name.StartsWith("Capacity")).First();
    var costParameter = model.Parameters.Where(
        p => p.Name.StartsWith("Cost")).First();
 
    ... // Code inside this method continues below.
 }
 

Obviously “Demand”, “Capacity”, and “Cost” are the names used in the model. But so far there is no data associated with these names.

Next in this DataBind method we must call the SetBinding method on the parameters to set their values. There are many overloads for this method, so there are many choices on how to provide the values. We start the discussion with a simple example using anonymous types:

public Model DataBind(Model model)
{
    ... // Code inside this method continues from above.

    capacityParameter.SetBinding(new[] {
        new { supply = 200.0, name = "Chicago" },
        new { supply = 30.0, name = "New York" } }, "supply", "name");

    demandParameter.SetBinding(new[] {
        new { demand = 50.0, name = "Philadelphia" } }, "demand", "name");

    costParameter.SetBinding(new[] {
        new {distance=200.0, producer="Chicago", consumer="Philadelphia"},
        new {distance=300.0, producer="New York", consumer="Philadelphia"},
        }, "distance", "producer", "consumer");

    return model;
}

In the calls to the SetBinding method for the capacity and demand parameters, we are using anonymous types to express sequences of objects which each contain two named values (‘supply, name’, and ‘demand, name’). The framework will use reflection calls to these bi-value objects to get the values. 

The costParameter is a little more complicated. It is a sequence of tri-value objects. Each object captures the locations for a producer and consumer, plus the distance between the two locations. The sample data are a little contrived for simplicity.

1.4  Report the solution

In the preceding Main method, the call to get a report is:

    var report =  solution.GetReport(ReportVerbosity.All);

The call produces a report in plain text which displays solution. A copy of the report is pasted in next (after I removed a few pieces of uninteresting data for clarity).

===Solver Foundation Service Report===
Version: Microsoft Solver Foundation 3.0.2.10919 Enterprise Edition
Model Name: DefaultModel
Capabilities Applied: LP
Solve Time (ms): 169
Total Time (ms): 353
Solve Completion Status: Optimal
Solver Selected: Microsoft.SolverFoundation.Solvers.SimplexSolver
…
===Solution Details===
Goals:
TotalTransportationCost: 10000
Decisions:
x(Chicago, philadelphia): 50
x(New York, philadelphia): 0
TotalCost: 10000

The reports shows that it took less than two tenths of a second to calculate the optimal solution for this problem by using the simplex solver. The optimal solution, incurring a cost of 10000, is the transportation of 50 crates of beer from Chicago to Philadelphia, and zero crates from New York.

2  LINQ for binding to AX data

The preceding approach to data binding does not provide the power we need to use OML and the Microsoft Solver Foundation in our AX work. We do not want to write lots of tedious code to obtain and assign data from AX by using either the .NET Business Connector or AX services. A much better approach is to use LINQ for AX in our C# code. (For background information about LINQ for AX, see http://msdn.microsoft.com/library/jj677293.aspx).

By using LINQ for AX we can reference table of data in AX with great ease and succinct code. Let’s imagine there are tables that store the values we need. We need a table called Cities that defines the cities that produce and consume beer. The Cities table has the following fields:

CITYNAME, COUNTRY, LATITUDE, LONGITUDE, STATE

To keep the example unrealistically simple, let’s say that the distances between the cities will serve as the only costing factor. The crates of beer will be shipped by trucks, each with a capacity. We would add criteria to make sure we filled the trucks to capacity. This spares our example the complexity of unit cost differences in scenarios where we might ship only one crate on a given truck, instead of filling the truck to capacity.

2.1  Rewrite the DataBind method

Now that we have the cities data ready, we need a way to calculate the distances between the cities, based on their latitude and longitude. For this calculation we use the Haversine formula, as explained in http://mathforum.org/library/drmath/view/51879.html. This algorithm is encoded into the static Distance method in the DistanceBetweenLocations class.

We will now present the new DataBind method. The few var declarations at the start of the method are identical to those in the earlier code example for the DataBind method. But now after the var declarations we use LINQ queries to access the data from AX:

public Model DataBind(Model model)
{
    var demandParameter = model.Parameters.Where(
        p => p.Name.StartsWith("Demand")).First();
    var capacityParameter = model.Parameters.Where(
        p => p.Name.StartsWith("Capacity")).First();
    var costParameter = model.Parameters.Where(
        p => p.Name.StartsWith("Cost")).First();

    QueryProvider provider = new AXQueryProvider(null);
    QueryCollection consumers = new QueryCollection(provider);
    QueryCollection producers = new QueryCollection(provider);
    QueryCollection cities = new QueryCollection(provider);

    // Bind data to the parameters using LINQ to AX.

    var tempCities = (from city in cities
                        select new
                        {
                            RecId = city.RecId,
                            CityName = city.CityName,
                            Latitude = city.Latitude,
                            Longitude = city.Longitude
                        }).ToArray();

    var capacityQuery = from city in tempCities
                        from p in producers
                        where p.cityRecId == city.RecId
                        select new {
                           supply = Convert.ToDouble(p.Capacity),
                           name = city.CityName };

    capacityParameter.SetBinding(capacityQuery, "supply", "name");

    var demandQuery = from city in tempCities
                        from c in consumers
                        where c.cityRecId == city.RecId
                        select new {
                            demand = Convert.ToDouble(c.Demand),
                            name = city.CityName };

    demandParameter.SetBinding(demandQuery, "demand", "name");

    var producerCityLocations = from city in tempCities
                                from producer in producers
                                where city.RecId == producer.cityRecId
                                select new {
                                    Name = city.CityName,
                                    Latitude = city.Latitude,
                                    Longitude = city.Longitude };

    var consumerCityLocations = from city in tempCities
                                from consumer in consumers
                                where city.RecId == consumer.cityRecId
                                select new {
                                     Name = city.CityName,
                                     Latitude = city.Latitude,
                                     Longitude = city.Longitude };

    // Cost.
    var costQuery =
        from producer in producerCityLocations
        from consumer in consumerCityLocations
        select new
        {
            distance = DistanceBetweenLocations.Distance(
               consumer.Latitude,
               consumer.Longitude,
               producer.Latitude,
               producer.Longitude),
            producing = producer.Name,
            consuming = consumer.Name
        };

    costParameter.SetBinding(costQuery, "distance", "producing", "consuming");

    return model;
}

In the example above, we start by reading the whole Cities table into memory, into the tempCities array. Reading the whole table improves the speed of queries in the subsequent code. It also helps us in debugging. But if your table contains a lot of data, you might instead need to perform multiple smaller reads from the table, incurring a slight overhead. The point is that the LINQ solution does not rely on the optimization of reading the whole Cities table.

2.2  WPF control to host a Bing map

Now we have what we need to use the model from AX. To make things a little more interesting and easier to demo, I have used this code from a form which shows a Bing map with the cities, and which also shows the solution.

This is all made very easy by using the managed control host to host a WPF control (the Bing maps control). This feature has always been a little under-appreciated and under hyped. It really is extremely useful whenever you want to present data in an interesting way in an AX form. You can download the Bing maps control from http://www.microsoft.com/en-us/download/details.aspx?id=27165.

3 Network model

The preceding example model is a simple case. You can easily expand it to the network model, where limits are placed on the amount of goods shipped, and equations maintain that nothing is ever lost in the grid. You can find interesting examples here:

http://www.me.utexas.edu/~jensen/ORMM/models/unit/network/subunits/distribution/index.html

These problems are solved just as the simple example we have just shown.

3.1  Assignment problem

One of the interesting special cases of the network problem is the assignment problem. The definition of this problem reads as follows:

There are a number of agents and a number of tasks. Each agent must be assigned exactly one task, and each task must be assigned exactly one agent. Any agent can be assigned to perform any task. The costs incurred may vary depending on particular attributes of the agent and task in each paring. The challenge is to choose the agent-task assignments that minimize the total cost.

The standard example used to illustrate this problem is that of a set of workers and a set of jobs performed by the workers. The cost of personi doing jobj is input to the algorithm. Some people are better at a particular job, so their cost is smaller than for someone who is not yet skilled at that particular job. The solution is a boolean matrix that maps each job onto a machine such that the minimal cost is incurred.

Another embodiment of this problem is the taxi company problem. There is a set of customers and a set of taxis to serve them. The cost of a trip is a function of the distance that an available taxi is from the customer. The objective is to minimize that cost.

3.2  Knapsack problem

Another problem that often occurs is the knapsack problem. Imagine you have a knapsack of a certain maximum weight capacity. The challenge is to pack the knapsack with items having as much total monetary value as possible without exceeding the weight capacity of the knapsack. 

 The equations are:

The first equation says that we should maximize the monetary value of the content in the knapsack (vi is the value of the ith item, and xi is the count of that item to bring). The second equation says that the cost of each item (or number of items) cannot exceed the constraint W, where W is the capacity of the knapsack. In our example, the cost is the weight that you have to carry, and W is the maximum weight that you are willing to accept.

There are three flavors of knapsack problems: 

We illustrate the difference with an example shown in Wikipedia (http://en.wikipedia.org/wiki/Knapsack_problem). Here the limiting factor here is the weight capacity of the knapsack (W=15kg). There are five types of items: the green, gray, blue, yellow, and orange types. There are many items available of each type. The weight and monetary value is written on each item.

The solutions are as follows:

• 0-1: yellow, blue, gray, orange.
  (value 10$ + 2$ + 2$ + 1$ = 15$; weight = 4 + 2 + 1 + 1 = 8kg)
• Unbounded: 3 yellow and 3 gray boxes.
  (value 3*10$ + 3*2$ = 16$; weight = 3*4 + 3 * 1 = 15kg)

Again, it is useful to look at what this looks like in OML:

Model[
    Parameters[Sets, items],
    Parameters[Integers[0,Infinity],
        Cost[items], Value[items], W
    ],

    Decisions[Integers[0,1], x[items]], // Either it is included, or it isn't.
    Decisions[Integers, TotalCost, TotalValue]

    Constraints[
        TotalCost == Sum[{i,items}, x[i]*Cost[i]],
        TotalValue == Sum[{i,items},x[i]*Value[i]],
        TotalCost <= W
    ],

    Goals[Maximize[TotalValue]]
]

You can convert the above OML model into the Bounded or Unbounded knapsack problem by changing the definition of the x decision variable. As with all of the examples in this area, the problem statement is an abstraction of the problems that you would apply this to in real life.

You would use the same procedure as described above to do the data binding.

3.3  Traveling salesman

The traveling salesman problem is a classic that rears its head in many interesting places. The description is simple enough:

• A salesman has customers who are placed in different cities.
• He must visit each of his customers exactly once.
• Then he must return to where he started.

The challenge is to devise the cheapest itinerary, given the cost of transportation between the cities.

Research into this area has been intensive because the cost of calculating the result for many cities is high. It is an interesting problem because it is not really possible to solve it using brute-force approaches. The number of possible itineraries is a function of the factorial of the cities to be visited, which very quickly gets out-of-hand. The bits accompanying this blog entry provide a solution expressed in OML, and a test-bed with a form to show the results on a map. I have adapted the solution from some excellent work by Nathan Brixus in http://nathanbrixius.wordpress.com/2009/04/27/solving-traveling-salesman-problems-using-solver-foundation/.

Conclusion

There are many more interesting examples of O.R. problems that can be solved by using the Microsoft Solver Foundation library. You can find some very interesting samples here: http://msdn.microsoft.com/en-us/library/ff524501(v=vs.93).aspx. The samples map to interesting business scenarios, all of which would be above and beyond most developers to solve using X++ code. The source code for the samples shown here can be found here: http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-86-72/5381.PrivateProject_5F00_TechnicalConference2012-_2800_1_2900_-_2D00_-Copy.xpo

I hope this has been a useful blog entry, one that shows you how to get started in this interesting field. There are many things to learn, but a lot of benefits to reap.

.

The ERDs are part of our “AxErd” website, which can be reached at the following Http URL:

     AxErd website:    http://go.microsoft.com/fwlink/p/?linkid=296623

Design Principle for ERDs

Dynamics AX has several thousand tables. It is impossible to construct one ERD that contains every table. And it is impossible to construct enough specialized ERDs that foresee every precise need of every partner developer. So the many ERDs we provide were designed to capture the core tables of each module, and to include some major tables from other related modules.

Minor compromises were made so that each ERD can fit onto one side of a standard sheet of paper when printed.

Support Files for ERD Extension

The AxErd website also contains hundreds of inter-linked support .htm files designed to assist the partner developer who needs to extend one of the ERDs that AxErd provides. In AX, the partner developer can easily look up all the parent tables of a given table, by navigating:

      AOT > Data Dictionary > Tables > MyTable > Relations > MyRelation

However, it is much harder for the developer to look up all the child tables of that same given table. Cross-reference does technically provide this information, but laborious clicks and scrolls are required to perform the look ups. In contrast, the AxErd website performs this look up with one click.

For instructions on how you can semi-automate the task of extending or creating an ER diagram with Dynamics AX 2012, click the 'Help' link on the AxErd Home page.

Tables and ERDs are Categorized by Module

For each table, the AxErd webpages indicate which module the table is categorized into. And in the reverse direction, for any one module there is a webpage that lists all the tables which are categorized into the module.

Example ERD

Here is one ERD from the AxErd website. The image has been reduced in size for display here.

The next image shows an AxErd webpage that lists all the child tables of the InventTable table, meaning tables that have a foreign key column that references the values in the primary key of the InventTable table.

The portion that is circled in red shows that the AgreementLine table is a child of the InventTable table. The arrow always points from child to parent.

In the image, every word that is blue is a clickable link to related information.

Gene Milener , 2013-May-23

1. Understanding the Difference

There has been some confusion about the behavior of X++ arrays in Microsoft Dynamics AX 2012, because the behavior can differ from AX 2009. Let’s clear up the confusion.

In X++, you can assign an array of primitive integers to another array. This is shown in line 3:

In AX 2009 and earlier, when one whole array was assigned to another, the assignment mechanism for primitive types was always by value. However...

Starting in AX 2012, for X++ arrays the assignment mechanism of primitive types is sometimes by value, and other times by reference. The assignment mechanism is determined by the X++ execution mode, which is either the interpreter or as CIL (new in AX 2012).

The following table relates both X++ execution modes to both assignment mechanisms:

2. Terminology

By Value: In the by value case, the assignment creates a copy or second occurrence of the assigned primitive value. The assignee variable is given the address of the copy. Now the assigner’s copy is unaffected when the assignee changes its value.

By Reference: In the by reference case, the assignment does not create any copy or second occurrence of the assigned primitive value. Instead, the assignee variable is given the same internal memory location of the value that the assigner has. Now both the assigner and assignee variables are reading from and writing to the same memory location. One of the two variables might be used to over-write the location with a new value, thereby affecting both variables (perhaps unintentionally).

Primitive versus Object: In X++ the primitive data types include int, str, and boolean. An instance of a class is an object.

X++ arrays can store only primitives. They cannot store objects like those constructed from classes.

3. X++ Code Snippet for Explanation

In the following X++ code snippet, consider the array assignment at line 3. Merely reading the X++ code does not enable us to ascertain whether the array assignment mechanism is by value or by reference. We have to know the execution mode. Line 4 has different effects between the two execute modes:

4. The Fix:  Uniform Behavior is Achievable

The variation in array assignment mechanisms has caught some X++ developers by surprise, quite understandably. To avoid array bugs in your X++ code, you should design your code to behave the same in both execution modes.

In both modes, your code should consistently use only one of the assignment mechanisms between the two arrays: uniformly use either (a) by value assignments between individual elements of the two arrays, or (b) by reference assignments between the two arrays.

Either of these two assignment mechanisms (by value or reference) can be achieved in either of these modes (X++ interpreted or as CIL), with a pair of either X++ arrays or AX Array class objects.

The following table provides code snippets to illustrate the design options.

5. Design Decision of the AX Product Team

During development of AX 2012, the product team strove to minimize any functional differences between interpreted X++ versus X++ as CIL. However, X++ arrays were one compromise we felt was necessary.

It would have been expensive to generate CIL that implements pass-by-value semantics for arrays. Also, the interoperability between AX and code written in other .NET languages would have become contorted, which would have been an unwise trade-off in the long run.

6. Demo: ArrayTestClass

This optional section provides X++ code for a runnable demonstration. The demo does the following:

1. Runs each test twice: once in the interpreter, and once as CIL.
2. Compares the behaviors of X++ arrays versus the AX Array class.
3. Compares the effects of assigning a whole array versus assigning each element one at a time.

Steps:
To run the demo, you can perform the following steps:

1. In AX, create a class named ArrayTestClass.
2. Add the following method names, and paste in the code from below. 
   
   • Main – Runs Test1 in interpreter, and again as CIL. Same for Test2.
   • Test1 – Assigns a whole array; uses X++ arrays.
   • Test2 – Assigns a whole array object; uses the AX Array class.
   • Test3 – Assigns one element at a time; uses X++ arrays.
   • Test4 – Assigns one element at a time; uses the AX Array class.
3. Menu: Build > Generate Incremental CIL.
4. In the X++ code editor, open the Main method, and then run it (by pressing F5, or clicking the green arrow to “Go”).

Outputs:
The demo outputs for each Test method are in /**comment sections**/ at the end of each Test method’s code.

Main

 Test1

Test2

 Test3

Test4

7. Links to AX 2012 Help on MSDN

This blog post refers primarily to product version Microsoft Dynamics AX
2012 (AX 2012, AX2012).

"X++ Arrays Behave Differently in .NET CIL, in Dynamics AX 2012"

Gene Milener & Peter Villadsen, September 2012

//eof

This blog entry investigates issues concerning the deterministic garbage collection strategy that is employed in Microsoft Dynamics AX 2012.  The AX garbage collector (GC) is compared to the Microsoft .NET Framework GC, in both strategy and performance. 

Background of the AX GC

The basic strategy of the AX 2012 GC is the same as it was even before AX 4.  AX uses a deterministic GC to manage the lifetime of X++ objects and table buffers. The GC is deterministic in the sense that objects are guaranteed to be disposed from the C++ memory space at the earliest possible moment after they are no longer needed.

The GC employs a reference counting algorithm that is enforced by the X++ interpreter. Whenever a reference is made to an object, the reference count is increased. Whenever a reference is removed the reference count is decreased. A reference can be removed when either (a) another value is assigned to a variable that had been holding the reference or (b) the variable that holds the reference drops out of scope.

Diagrammed Examples

Below is a diagrammed example of the reference count functionality in the AX GC.

The first line of X++ job code constructs an instance of MyClass and assigns the object reference to the stack frame variable mc. In heap memory on the right, the blue circle represents the MyClass object. The first blue circle has one incoming arrow which represents the reference to the object. The first blue circle is labeled ‘1’ to show the count of references to the object so far.

Next, the assignment mc1=mc creates a second reference to the blue circle object.  Now the blue circle labeled ‘2’ shows the new total of two references to the object. (Note the arrow labeled ‘mc2’ should say ‘mc1’.)

Next, the assignment mc1 =null removes one of the references, making the state again like the blue circle labeled ‘1’.

Finally the job ends and the stack frame holding the variable mc goes out of scope.  The blue circle labeled ‘0’ is marked with a symbol to indicate the AX GC removes the object from the heap at this moment.

Figure 1 Object deletion through reference counting.

Unfortunately there are more complex scenarios that reveal the weakness of the reference count strategy. Scenarios that involve reference loops require more operations by the GC. Consider the following case of two objects that refer to each other:

Figure 2 Mutually dependent objects, requiring loopcount

In this case the objects should be removed because there are no external references to either object. To handle reference loops, the GC also maintains a loop count. The loop count indicates the number of loops in which the object currently takes part. When the difference of the reference count and the loop count is 0, the memory occupied by the object is removed.

The algorithm works across the client/server boundary as illustrated below:

Figure 3 Object graph spanning the client server boundaries.

The algorithm guarantees that storage is reclaimed as soon as it is possible to do so, and the application programmer has no hooks into this. It is not possible to implement a method that is called when the object is reclaimed (akin to implementing the IDisposable interface in .NET). It is a common misconception that in AX the finalize method can be used for this purpose, yet this method has no special meaning to the AX system.

In this distributed scenario the loop/ref information is maintained though the exchange of suitable messages over the network, causing considerable network traffic and performance degradation.

The AX GC uses a similar algorithm for table buffers. However, this algorithm is simpler for the GC because table buffers cannot hold references to each other, and there is no loop count to maintain.

Microsoft .NET Framework GC

The garbage collection strategy used in .NET is very different from what we just described. It relies on a mark-and-sweep algorithm where the set of objects are examined periodically to check if references are held to them. When no references are found to a given object, the object is removed. This strategy emphasizes that no guarantees are made about whether or when an unreferenced object might be removed. A complex set of criteria are considered before the .NET GC is invoked, and the GC itself has discretions. This is why the .NET garbage collection strategy is said to be non-deterministic.

As an interesting aside, the .NET garbage collection comes in two different flavors. One GC is used on the server side, and the other is used for client side applications. The server GC is optimized for throughput on multiprocessor machines, and the client workstation flavor is optimized for machines with one processor. If you are interested in the differences between the two flavors, the following webpage is a good place to get educated: http://msdn.microsoft.com/en-us/library/s973858.aspx.

The Garbage Collection Problems in AX

The basic problem with the AX GC algorithm described above is that sometimes it scales poorly. The GC performs poorly in the face of an increasing number of object having loops in the object graph. The reason is that any assignment to an object may require extensive analysis to update the loop counts for all the objects in the structure. Plus this can cause significant network traffic when the objects are distributed.

Customers and internal Microsoft developers have seen a few issues that have been caused by the weakness of the AX GC strategy. The scenarios always involve structures that are self-referential containing many objects. Typically a programmer writes code that generates structures in which the objects are a monotonously increasing function of the number of records in a database table. This may work fine for test purposes, but might fail in production cases.

Workarounds

There are workarounds that can be applied to reduce the problem. One workaround is to manage the reference loops through lookups in suitable structures. For example, instead of a standard direct reference, the code could store a GUID, and have an auxiliary structure to map the GUIDs onto references.

Performance Graph of Experimental Results

To illustrate the problems with the AX GC, we have provided the contents of an X++ .XPO file at the end of this blog. You can import the .XPO into your test AX system. This code sample was deliberately designed to demonstrate the worst case behavior, and it does not represent code that is likely to be used in real-life applications.

The X++ sample builds a circular linked list of elements. After the list is built it is dismantled by taking each element out of the loop one at a time. Each time a link is removed from the list, the AX GC must rescan the whole remaining list. In this case all objects are created and destroyed on the same tier, namely the client tier.

The following graph reports the wall clock time that the AX GC consumed as we varied the number of elements our circular linked list. The test was run on a standard laptop computer. The run consumed one core 100%.

Figure 4 Performance of Deterministic GC relative to .NET GC

The x-axis is the number of elements in the circular list. The y-axis is the time spent in milliseconds on the algorithm in Appendix A.

The steeply curved green line is the performance of the deterministic AX GC. The performance degraded exponentially as the number of link nodes increased linearly.

The completely horizontal blue line is the performance achieved when the X++ code is translated into MSIL using the AX 2012 feature. For technical reasons it was not possible to sample the data with resolution of more than 1 second. The execution never managed to register a reading of more than 1 second, even when the number of nodes exceeded 350,000. Note that the x-axis extends only to 30,500.

AX GC Problems Stack Up

The AX deterministic GC has one other flaw. Its traversal of the object graph is recursive. This means that it (in very rare situations) can run out of C++ stack space, which invariably results in a crash on the tier doing the garbage collection (either the client or the server). We have not seen this in production code, but it is a potential problem, even for graphs that do not contain loops.

The weakness of the AX GC is serious enough that the exponential performance degradation shown by our experiment can occur even in less severe X++ samples.  The example used in this test was chosen to show the problems in the AX 2012 GC. The example consists of object allocation and de-allocation for loops, plus some glue to make it run. Our X++ sample can be seen as pathological in that it will produce an extreme performance benefit from being converted into MSIL. Use of the MSIL feature can be a workaround in a few cases. However, most realistic X++ code in production AX applications includes frequent reads and writes of data with the database, and those parts of your X++ code do not run any faster in MSIL mode.

Appendix A: The X++ Source Code Used in the Example

 The following X++ source code contains two classes:

• ·         ListElement - the type used for the each node in the linked list.
• ·         GCPerformanceTest - has the static Main method to run the test.
  • o   Main calls ::SetupGraph, the ::TearDownGraph.

Please find the X++ source code attached.

GCPerformance.xpo

If you employ developer talent that knows managed code and Visual Studio well, it makes sense to leverage that asset. Despite its benefits, this managed (C#, VB.NET etc.) alternative does not change the fact that the main problem people have when they ramp up on business logic development projects in AX is understanding the huge code base and its many complex interactions.

In general we do not try to compete with .NET functionality: Rather we leverage .NET and implement complimentary features for X++.

The following table describes some scenarios where the C# alternative tends to be a good choice.

The following table describes some scenarios where traditional interpreted X++ tends to be a good choice.

To demonstrate the faster performance offered by C# compiled to .NET CIL, X++ code for the “towers of Hanoi” is provided below. This algorithm features lots of recursive calls. The number of calls is an exponential function of the number of “discs” that is passed to the method. If there are n discs, the number of calls required is 2ⁿ -1, the so-called Mersenne numbers.

In AX 2012 we have the new luxury of being able to execute X++ as CIL code directly, which makes it easier to prove the performance benefit of CIL. However, the conclusions are the same regardless of whether the CIL is derived from X++ or C#.

Below is the contents of an .XPO file that you can import into Microsoft Dynamics AX 2012 (not into AX 2009). The .XPO contains the following two methods:

RunInCIL

RunInXppInterpreter

Create a simple AX job (under AOT > Jobs) where you call the two methods with the same parameter value. Feel free to experiment by passing in values from 17 to 22. After each run you should compare the time spent evaluating in p-code and in CIL. You will see that the speed advantage of CIL over p-code can grow to any order of magnitude as you increase the number of discs to move.

Note that you may see an overhead from the JIT’ing and from loading the assembly containing the code into the current app domain the first time the code is called. Just call it again to get more meaningful numbers.

 1.  Super() cannot Call an Abstract Method

Suppose an abstract method is implemented by a derived class.  If the method in the derived class tries to call super() to reach the abstract method in the base class, the X++ compiler will report an error.  For example:

 Reason for the Change

Abstract methods have no code body. Therefore any call to an abstract method is meaningless.

2.  All Default Parameters are required to follow the Last Non-default

The X++ compiler rejects any method parameter signature which lists a default parameter before the last regular non-default parameter.

In the following X++ method signatures, parameter ‘i’ is a default parameter that causes both methods to fail to compile.

Reason for the Change

It does not make sense to have non-default parameters follow default parameters. In such cases all calls to the method must provide an explicit value for the default parameter, and the parameter could never take its default. Therefore it understandable that .NET CIL does not support such cases. Starting in AX 2012 the X++ compiler no longer supports such cases either, to be compatible with CIL.

3.  Methods that Override Should Match the Parent Signature

When a method in a derived class has the same name as a method in its base class, the X++ compiler requires the exact same parameter signature for the two methods. Also, both methods must have the exact same return type. Otherwise the derived class will fail to compile.

For example, in AX 2012 under AOT > Classes, the X++ abstract class SysAttribute is extended by the class SrsReportNameAttribute. Therefore an instance of SrsReportNameAttribute can be passed to a method that takes a parameter of type SysAttribute. But for overriding a method, these two types would not be considered an exact match, and the X++ compiler would not consider these two types to be the exact same kind of parameter or return type.

In the following X++ code example, the MyDerivedClasswould fail to compile: 

 Further, no default parameter can exactly match a non-default parameter, or vice verse. The number of parameters must also match. X++ supports method overriding, but not method overloading.

These rules also apply when a class implements an interface.

Reason for the Change

Without this new restriction enforced by the X++ compiler, these incompatibility errors would be detected only during run time. The above X++ code example would compile in AX 2009, but in AX 2012 the X++ compiler will reject this example.

4.  Interface Implementation Must be Complete

An X++ class that implements an interface must provide code for all the methods of the interface. This includes any methods the interface inherits from another interface.

In the following X++ code example, BadClass fails to compile in AX 2012:

Reason for the Change

Same as for item 3 above.

5.  The Same Method Name cannot be Inherited from both Base Class and Interface

Suppose a derived class both extends a base class and implements an interface.  And suppose that the base class and the interface specify the same method name. The X++ compiler no longer tolerates this situation.

In the following X++ code example, the SadClass class fails to compile in AX 2012:

Reason for the Change

Same as for item 3 above.

In X++ you can optionally declare a string to have a specified maximum length:

str 5 maxLen5String;

Semantically, if you try to store more than 5 characters to this string, it will be truncated.

In March 2010 we blogged here about the X++ max-length string type, and asked how we should handle it in Microsoft Dynamics AX 2012. This became an issue when we began to compile X++ into .NET Framework CIL (Common Intermediate Language). X++ has the concept of a max-length string, but CIL does not. Thus an inconsistency in behavior was to be introduced. X++ max-length strings are truncated as necessary when the X++ runs in interpreted mode, but they are not truncated when the X++ runs as CIL.

Respondents to our earlier blog were generally indifferent to whether we truncated strings in interpreted X++ or not. But we were also concerned about backward compatibility. After much discussion we decided that interpreted X++ should continue string truncation in AX 2012, just as it has done in earlier versions of AX. In other words, we were willing to live with the inconsistency with CIL.

However, as part of this discussion we wanted to be able to easily analyze this to better understand what this inconsistency would mean in real-world code. To that end, we added an option to log where string truncation was actually happening in the interpreter (and, by extension, where it would not be happening in CIL).

We found this option to be so beneficial that we decided to productize it so that everyone could benefit.

New Option to Log String Truncations

The name of the new switch that activates the logging of string truncations is logstrtrunc. The switch is honored by both the AX client and the AX server. Logstrtrunc is off by default since there is some performance cost associated with it.

As with most AX option switches, logstrtrunc can be specified on either the command line or in the Windows Registry. Whenever the command line and the Registry conflict, the command line value takes precedence.

Command Line Syntax

• Ax32.exe  -logstrtrunc
• Ax32serv.exe  -logstrtrunc

Note:  /logstrtrunc  is invalid syntax.  The option name must be preceded by a dash (-).

Registry Keys

Alternatively, an optional Windows Registry key for the logstrtrunc option can added, either manually using regedit.exe, or by double-clicking on a .REG file containing the following data:

The first key contains the path node of \Dynamics\, indicating that it applies to the Ax32.exe client. The second key contains the path node of \Dynamics Server\, indicating that it applies to the Ax32serv.exe server. Either or both may be specified. The registry key option is especially useful for the Server (AOS), which is usually run as a service and not from the command line directly.

After these keys are in the Registry, you can turn this logstrtrunc option off and on by toggling the dword value between 0 and 1 respectively. Remember to disable the key after you are done testing to avoid any performance impact.

The client and server read these Registry keys only when they start running, so you will need to restart the session before your change will take effect.

Effect of logstrtrunc Option

When the logstrtrunc option is on, every time an X++ max-length string gets truncated, the system writes an entry to the Application section of the Windows Event Log. You can view these entries by using the Windows Event Viewer. In each entry the Source and Event ID values are as follows:

·        Source:

o   Client:   "Microsoft Dynamics AX"

o   Server:   "Dynamics Server 01"

·        Event ID:  191

These values are very useful for filtering and sorting when you want to locate the relevant log entries in the event log. Of course, if your X++ code contains no max-length strings or if none of the assignments to your max-length strings exceed the length boundaries, nothing will be written to the log. We will now focus on what to do if log entries do show up.

Code Example and Sample Log Entries

Here is a simple X++ job which assigns excessively long values to a max-length string. Assume you run this job in interpreted mode with logstrtrunc on:

Code line 6 attempts the assignment of an excessively long string which will be truncated. The body of the event log entry message displays the whole value that the X++ code attempted to assign so that you can see what was truncated:

X++ str 5 buffer truncated during assignment:

Hello world

Stack trace

(C)\Jobs\Job1 - line 6

Code line 7 attempts to make the concatenation of two strings be the new value of a maxStringBuffer. In this case the message displays both strings that were involved in the concatenation so that you can figure out what got truncated:

X++ str 5 buffer truncated during concatenation:

Hello

 there world

Stack trace

(C)\Jobs\Job1 - line 7

Notice that a stack trace is included, which is very helpful in more complicated examples.

Analysis Considerations

Assume your system has several log entries from logstrtrunc being on. These entries could result for a number of different reasons.

The following table categorizes some possible causes of those log entries and in each case a possible approach that can eliminate the automatic truncation of strings (and eliminate future log entries). In many cases, this will allow you to fix real “bugs” and also eliminate any discrepancies when run interpreted versus run as CIL:

Try Various Scenarios

With logstrtrunc on in your test environment, run several different scenarios to give all possible string truncation code the greatest likelihood of truncating and thus being logged. Scenarios that involve localization can be very important to test. The resulting log information should enable you to understand and eliminate any unwanted automatic truncations that can occur when X++ runs in interpreted mode.

In summary, the logstrtrunc option makes it easy for X++ developers to track down any potential string truncation discrepancies associated with compiling X++ to .NET CIL.