When we go to buy grocery, the point of sales personnel scans barcodes from each item. This updates backend inventory and all kinds of reports are generated. This labelling of even technically incapable items like fruits/soaps is nothing but a simple example of IoT in retail. Microsoft has worked with London Underground transport system on this concept where not only rails but also things like lifts, escalators, cctvs, temperature/humidity monitoring units etc. are all connected and providing intelligence in real-time.

IoT and StreamInsight: With large number of entities connected, there are IoT scenarios where we need to analyse data in near real-time. We cannot wait to store data coming from small devices to be first stored in database and then analysed later. Microsoft’s StreamInsight (that we released back in SQL 2008 R2) which is also available as an Azure Service holds important place in such IoT scenarios. Using StreamInsight we can perform BI on data on real-time before it even gets stored in database which is a must for many scenarios.

IoT and BigData: With IoT, it is natural to have a very large data storage with wide data formats (not just relational) that traditional SQL Server were never designed for. With such large storage of data it is also expected to analyse this data at a speed that SQL Server cannot deliver (SQL Server is typically good for Terabytes only). Many organizations cannot afford solutions like SQL Parallel Data warehouse (PDW). So, the only choice remaining with large data associated with IoT is using Big Data techniques like Microsoft’s HDInsight (Hadoop implementation on Azure) or HDP for windows (Hadoop implementation on premise).

IoT and Embedded Systems: With the need for every device to be connected (by IoT philosophy), it is natural to build intelligent small devices. Technologies like .NET Micro framework/ .NET Compact frameworks or Windows embedded 8.1. Microsoft’s Intelligent Systems Service is the central place to stop by for building such devices.

IoT and Service Bus: Microsoft is also creating the base for developing custom solutions that are very much relevant in IoT. Networking Technologies (like VPN) may not be sufficient enough to realize IoT in scenarios where small devices cannot be part of VPN (lets say) but with technologies like Service Bus, we can expect connected device vision.

IoT and Industry Verticals: Microsoft’s point of view about IoT is not just connecting everything (Internet of everything), rather connecting all those things that matter to us most. This creates many scenarios for IoT like IoT in retail, IoT in healthcare, IoT in manufacturing etc.

IoT and Microsoft: With Microsoft’s April 2014 announcement of Azure Intelligent Systems Service, we’ve provided a practical solution towards IoT demonstrating how IoT solutions could be built using HDInsight, PowerBI, embedded devices, Analytics Platform System (combining SQL (PDW) and Big Data (HDInsight)).

-Rahul Gangwar

Pre-requisites:

I assume the reader understands the concept of Claims Authentication and related terms like STS, IP...

About:

This post explains my understanding about Live ID authentication that you might want to integrate in your applications like Win Forms, SharePoint, CRM or a custom ASP.NET application.Live ID exposes different ways of authenticating against it. The primary ones being:

·         Using WS-Federation:

If you want to use WS-Federation, then you need to (1) Register your application with Live ID STS as trusted RP (use MSM https://msm.live.com/wizard/default.aspx ). This mechanism is also known by name Relying Party Suite. (2) Configure your application to start using Live ID: Live ID STS exposes its Federation Metadata which makes it easy for you to use FedUtil.exe in your application to establish this trust from RP to Live ID STS. the Federation metadata is found here: Metadata XML URL Seems easy. BUT the very first step of configuring your application as RP using MSM is not exposed to external customers. It is available only for Microsoft Partners. So as per my conclusion, even though Live ID supports WS-Federation, you won’t be able to use it “directly”. I will explain in next option why I have used the term “directly”. I am not sure why do we have this article published when we cannot actually use it publicly: https://technet.microsoft.com/en-us/library/ff973117.aspx

·         Using Azure ACS:

Microsoft Azure Access Control Service (ACS) is an cloud STS. It offers Live ID authentication as out of box identity provider. In order to use this path for Live ID authentication, you need to (1) Configure ACS to start trusting your application as trusted RP. (2) Add Live ID as Identity Provider in ACS (3) Configure your application to start trusting ACS as trusted STS. Both these steps are pretty easy to be followed with minimal coding. Here is one post that explains in step by step manner: https://msdn.microsoft.com/en-us/wazplatformtrainingcourse_introtoacslabsv2_topic2.aspx
If this is so easy, then why doesn’t one always use it?

§  Cost: Azure ACS doesn’t come for free. You need to pay for it.

§  It adds an extra layer between your application and Live ID (a less agreed reason).

·         Using APIs offered by Microsoft:

So, isn’t there any free, publicly available way of directly working with Live ID authentication? Well, yes there is and you can do it via APIs that Microsoft offered. These set of APIs (previously known with different names) are clubbed under the common name called as “Messenger Connect”. Messenger connect can not only be used to authenticate users with Live ID but also access their information stored on Live ID Servers like pictures, contacts etc. Here are details on how to achieve it with an ASP.NET application: https://msdn.microsoft.com/en-us/library/ff748287.aspx

You can also develop a custom claims provider that uses Messenger Connect and configure it with AD FS 2.0 so that you don’t have to code it every time you plan to use Live ID authentication.

Rahul Gangwar

• There is no fixed account for this so you would not find any account under in built accounts while trying to set ACL permissions.
• Corresponding to each AppPool that you have a new account is created when the AppPool starts. The naming convention is: “IIS APPPOOL\your_app_pool_name”. So if you have an application is running under “Classic .NET AppPool” then the local user account created is IIS APPPOOL\Classic .NET AppPool.
• Just grant permissions to this account and you are set to go.
• The benefit of this approach is that without you taking the trouble of creating any local user account under a system, you can configure to have your application run under different identity. Different applications can run simultaneously without any possibility to access each other’s data. Automatic provisioning also becomes easy.
• Since this account actually doesn’t exist on the system there are less chances that you would have given any other rights to this account and as a result any hacking threat doesn’t compromises your system.
• But in order to best use this feature you need to make sure that for such compartmented security requirement, you need to run your application under a dedicated custom Application Pool.

Rahul Gangwar

I just came to understand that while using the Step by step guides for ADFS 2.0 alongwith combination of the Virtual machines that are present at connect Microsoft website, you might receive "Service Unavailable" errors while trying to navigate to ADFS asmx services or FederationMetadata.xml file.

The virtual Machines present at Microsoft connect location (below) have the Token-signing certificate and token-decrypting certificates on ContosoDC server expired on April 22 2011. Because of this ADFS is not able to build the certificate chain for these operations and as a result metadata endpoints are not exposed. An end user trying to work with these labs simply doesn’t see federationmetadata.xml file.

Temporary Resolution:

While it would take time to get the VMs updated, you can work ahead by renewing the certificates manually. In order to do so folow below steps:

1. Log in as Administrator on ContosoDC

2. Open Powershell in administrative mode and enter the command: "Add-PSSnapin Microsoft.Adfs.Powershell"

3. Enter following commands one by one. After hitting enter, you need to wait till the command prompt returns back.

Update-ADFSCertificate -CertificateType: Token-Signing -Urgent:$true

Update-ADFSCertificate -CertificateType: Token-Decrypting-Urgent:$true

VM Location: https://connect.microsoft.com/site642/Downloads/DownloadDetails.aspx?DownloadID=29506

 Rahul Gangwar

I just came to understand that while using the Step by step guides for ADFS 2.0 alongwith combination of the Virtual machines that are present at connect Microsoft website, you might receive "Service Unavailable" errors while trying to navigate to ADFS asmx services or FederationMetadata.xml file.

The virtual Machines present at Microsoft connect location (below) have the Token-signing certificate and token-decrypting certificates on ContosoDC server expired on April 22 2011. Because of this ADFS is not able to build the certificate chain for these operations and as a result metadata endpoints are not exposed. An end user trying to work with these labs simply doesn’t see federationmetadata.xml file.

Temporary Resolution:

While it would take time to get the VMs updated, you can work ahead by renewing the certificates manually. In order to do so folow below steps:

1. Log in as Administrator on ContosoDC

2. Open Powershell in administrative mode and enter the command: "Add-PSSnapin Microsoft.Adfs.Powershell"

3. Enter following commands one by one. After hitting enter, you need to wait till the command prompt returns back.

Update-ADFSCertificate -CertificateType: Token-Signing -Urgent:$true

Update-ADFSCertificate -CertificateType: Token-Decrypting-Urgent:$true

VM Location: https://connect.microsoft.com/site642/Downloads/DownloadDetails.aspx?DownloadID=29506

 Rahul Gangwar

·         MVVM relies on the concept that View (XAML) will not contact your Model directly.

·         View will contact Model via ViewModel classes.

·         ViewModel talks with View using Data Binding and Commanding in XAML based applications.

·         ViewModel talks with Model via web service communication in case of Silverlight when we don’t use RIA Service.

MVVM with RIA:

·         RIA Service abstracts the web service layer from developer’s perspective and presents on client an identical set of classes that we have in model on server.

·         This simply means that under normal MVVM scenario we just change the communication between ViewModel and Model to go via the auto generated model classes on client side instead of explicitly calling web services (though internally it is still WCF communication).

·         In my opinion, this should not pose any problems to the MVVM pattern given that we keep the communication link same as above.

Cautions/DON’Ts:

·         We should never attach the auto generated client model classes of RIA with View directly. This link of View to Model should happen via ViewModel only. In a nutshell, these auto-generated model classes should never be considered as ViewModel.

·         We also have the availability of some RIA controls like DomainDataSource (in XAML). In case of MVVM we should NEVER use any such RIA controls in XAML since they break the very pillar of MVVM by contacting auto generated client side model of RIA directly (which is same as above point).

Rahul Gangwar

Today is May 20th 2010 and I am writing this post to discuss the working mechanism of Silverlight’s capability of utilizing GPU of your machine.

There are two kinds of graphics popular today: 

1. Bitmap Graphics:

Here the graphics is a file that stores the display information in the form of pixel information. Basically the term itself means map of bits. Each pixel stores information about how to light it up, in the data structure: Color Depth. This structure (color depth) can be of various size like 8bit, 16 bit, 32 bit etc. and larger the data structure the more information about the pixel can be stored. This determines the bit depth. So when you buy a scanner/camera kind of device you can relate what bit depth mean and what mega pixel means. Two devices having same megapixel but different bit depth will capture different quality of images.  

Now larger the number of pixels per inch you have for your image, lesser would be distortion if you try expanding the image. But as we store larger number of pixels, the size of image also grows large. This means for all practical purpose there is a limit to which you can expand your image to a larger size. Theoretically no matter how many pixels an image contains, expanding to larger size will definitely distort its quality. It is just that it won’t be visible to human eye if the expansion and number of pixels are maintained to a balanced level. Not only that the image quality decreases even if you shrink it beyond the allowed limit. That is why we use less pixel images if the display required is small (not primarily because of size, but for quality). So a high resolution image/video will not look good on a small device. You need to re-encode it!

2. Vector Graphics 

Here the graphics is store as mathematical formulas. Like lines, formulas for different shapes like curves or even custom algorithm is created to interpret the shape. This means that no matter how big/small display we need, the information will simply remain the same. Size of vector images will remain always same irrespective of how large/small it is displayed. The size changes with the complexity of information stored. If you want to display simple shapes like triangle, circle or even combination of such shapes, vector images are best. But if you ever try to convert a photograph that you clicked on that famous hill station, your resulting vector image would be larger in size and also in worse condition because vector graphics techniques are not so sophisticated to convert the graphics created by God into man made. No lens in the world could be better than your eye and no speaker in the world could be better than your ear! You might call that as a limiting factor or something mysterious :).  

Now coming back to Silverlight, the graphics that you display with Silverlight can be rendered using two ways: 

1. Using Graphics Card (GPU) (primarily when the graphics is bitmap).

2. Using Software that is run by CPU (primarily when the graphics is vector based).  

The XAML that you write in your SL application are displayed using CPU. They are all formulas that are calculated and rendered by CPU. When you instruct SL to use GPU we call that as GPU acceleration. Why “acceleration”, I don’t know. Probably “assisted” would have been better term.  

Mechanism:  

It is simple. You specify the UI elements that you want to display using GPU by adding the attribute CacheMode=”Bitmap”. Now all the UI elements are represented in form of object tree in Silverlight. Silverlight does the calculation to create the object tree and forms a bitmap for that object tree. Then, Silverlight gives this bitmap to GPU to display it and keeps it cached. Now, when the application screen refreshes, that part of the screen that you specified to be displayed by GPU is simply rendered by GPU and the rest part of the UI is rendered by SL using CPU. As a result, we are using less CPU and improving performance. Even if you are performing some animations like scaling, changing opacity etc. these are done by GPU and not by CPU.  

Why not Always?
This seems interesting so why not do it always using GPU? Few reasons (not in order of any priority):  

·         There are some animations that GPU cannot assist (like changing perspective (3D), animating on each pixel such as pixel shader effects). There is no point in using GPU acceleration in such scenarios.  

·         If the GPU has less memory and if the memory required by Silverlight is large, Silverlight falls back to CPU mode and you will hit performance. Also there is no way to query memory available in GPU from Silverlight code. However you can query other details like GPU vendor, driver version etc. through Analytics.GpuCollection structure.  

·         Cache Invalidations: If your UI element is changing a lot and that change cannot be assisted by GPU (another example is changing text of a UI element lie textbox), then cache invalidation happens and we hit on performance.  

Note:

<param name="enableGPUAcceleration" value="true" />

Also note that on Mac OSX GPU acceleration works only in full screen mode. Users need to have a DirectX9 (windows) or OpenGL2 (Mac) compatible video card. 

Testing:

You can test the GPU acceleration by adding another parameter to the plug-in as shown below:

<param name="enableCacheVisualization" value="true" />

This parameter shows all GPU accelerated graphics in natural colors while all non GPU accelerated graphics in Red color for testing purpose.

When you test CPU percentage, make sure you use lesser number of cores and more complex graphics to actually see the difference in performance.  

Beware Animation:  

Please make sure that the UI elements for which you have used GPU acceleration are using below animations only and not anything else: 

Scale, Rotate, Changing Opacity, Clipping (only if clip is rectangular). For all other types of animations like 3D, moving, pixel shaders, do not use GPU acceleration.  

Poor Joke:
If you like the post, you can pay pal me here.

Today is January 5th 2010 and I am writing this post to explain the mechanism of the Server Driven Paging feature of WCF Data Service and also a code guide on how to use this from Silverlight 4.0.

The Pattern:

Server Driven paging is one of the new features that have been added in WCF Data Service Version 1.5 CTP2 (formerly called as ADO.NET Data Service). The feature in simple terms means transmitting data in pages. This feature is important from performance point of view and also an important pattern used in applications where UI layer and data access layer are separated by cloud. I view this as a pattern used many places; the most common of being “chunking” approach used in large data transfer. Take example for WCF data transfer of a large file. We typically implement and recommend to use “chunking” pattern to transmit large data from one place to another because of many reasons. One reason could be to support “resume” functionality. So if data transfer is interrupted at 50%, we don’t need to start all over again next time. We can resume from 50%. Another reason could be to allow large data transfers where network policies put an upper bound on data transfers. One more possibility I can think of is unreliable network (which internet is) where packet loss is very common and would otherwise lead to long time to get a perfect data flow in one go. Server driven paging is the same pattern/algorithm applied to transfer large set of records between WCF Data Service and client so that it doesn’t put pressure on client (e.g, to render millions of records with small scroller in a single data grid, or storing large records in client memory after he waited for good 10 minutes or so). In short, paging makes the overall process faster on average. Server driven paging gives more performance since we save time in transmitting very large number of records over the wire and improve the end user experience.

The Mechanism:

The mechanism is simple and worth understanding to make decision about whether to use this feature or not:

Step 1: Client sends request to fetch data by firing query.

Step 2: Server Fetches result from Data Base for a given page. This step needs a little elaboration since here comes the important factor that might decide how effective SDP of WCF Data service is for your scenario:

When a user fires a query from client, the data service fetches the result based on page size and the page on which currently the user is. If we want to implement paging in our application from scratch, we usually follow one of the two approaches to achieve this:

•  Session based approach: We store the page on which a user currently is in the session. So we always know which page the user wants when he clicks next/previous. Pros: Sometimes in order to improve speed we also cache result set in session and simply return the required page and in fact for such cases, this is the only approach we use. We cannot afford to go to DB every time the request comes.
  Another benefit of this approach is that the client code is very simple, we simply have to tell whether we need next or previous page. On server side, we fetch records from DB that should be present on a particular page.
  Cons: Drawback is that the moment session expires, the page tracking is lost and it also requires us to scale server. Not a preferred approach for most of business scenarios that I have encountered.

• No session approach: Here we establish some unique key that can be used to calculate which records should be present on a particular page. An example would be a combination of “page number” and “page size” would give the records that should be present on a given page. Every time a new page is requested, we go to database to fetch the records for that page and return them. This approach is used where scalability is more important than speed. We don’t put pressure of storing calculated results in memory at server.

WCF Data Service uses “no session” approach and goes to DB every time a new page is fetched. It uses primary key as the counter to reach to a particular page. So when a next page request comes, it contains the primary key of the last record of the last page. WCF Data Service goes to Database, orders the query by primary key, fetches records of page size starting after the provided argument by client and returns result. The benefit of this approach is that you don’t depend on session for paging. This approach is also ideal for clients (especially thick clients like Silverlight) that might not talk to server for long time. Another benefit is that we don’t need to scale the server by not storing large set of data in session. The drawback of this approach is that it is not performance effective since it goes to DB for each request. Not only that, it uses orderby in query fired to DB which is even slower than an equivalent query not using “orderby”.

Step 3: WCF Data Service returns the data for the requested page. Along with the Data it also returns a URL that contains the URL to fetch next page.

Sample Code with Silverlight:

The following Sample Code has been developed using Visual Studio 2010 and Silverlight 4, but it will work fine with SL 3.0 as well since Data Service is all based on REST. Just that right client APIs need to be used and correct version of Data Service is needed at server side to enable Server Driven Paging. Contact me if you need a working sample code and if anything below did not worked.

• Create a Silverlight 4.0 Application
• Add Database to Server. Add Employee Table with FirstName, LastName and eMail (Primary Key). This is the table that we are going to page via server upon client request. Fill it with some data… maybe 5 records.
• Add DataLayer at server side using Entity Framework.
• Add ADO.NET Data Service at server side. Attach it to DataLayer, set its Access Rule, turn on ServerDriven Paging with Page size and Protocol version to be used by adding following statements in InitializeService() method of ADO.NET Data Service:

config.SetEntitySetAccessRule("*", EntitySetRights.All);

//enable server driven paging with PageSize

config.SetEntitySetPageSize("Employees", 2);

//This is required for server driven paging to work and maintain backword compatibility since SDP doesn't work with version 1 of data service

config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;

• Add DataGrid and next button on Silverlight client. Remember to autogenerate columns to true for datagrid.
• Fix port address for website and add service reference to WCF data service in SL client.
• On Silverlight client code behind, add proxy to data service and client type that will hold link to next URI on every request fired to data service. The code is similar to below:

//add proxy to wcf data service where we can fire queries

public DatabaseEntities myDB = new DatabaseEntities(new

Uri(@"https://localhost:53951/WebDataService1.svc/", UriKind.RelativeOrAbsolute));

//add the client type that will hold the link to next uri (next page of records) after every request

public DataServiceQueryContinuation<Employee> token = null;

• On PageLoad, execute query to get all records from server. Code is same as any WCF Data Service code and shown below:

public MainPage()

{

InitializeComponent();

//execute query to get all records from table

var q = from e in myDB.Employees

select e;

var eq = q as DataServiceQuery<Employee>;

eq.BeginExecute(new AsyncCallback(MyCallback), eq);

}

• On Callback, write code to collect response for two different conditions. The entire code behind is shown below. I have not included XAML since its trivial having a blank Datagrid with autogenerate columns to true and a button for next page:

public partial class MainPage : UserControl

{

//add proxy to wcf data service where we can fire queries

public DatabaseEntities myDB = new DatabaseEntities(new

Uri(@"https://localhost:53951/WebDataService1.svc/", UriKind.RelativeOrAbsolute));

//add the client type that will hold the link to next uri (next page of records) after every request

public DataServiceQueryContinuation<Employee> token = null;

public MainPage()

{

InitializeComponent();

//execute query to get all records from table

var q = from e in myDB.Employees

select e;

var eq = q as DataServiceQuery<Employee>;

//this is called once and the caller is DataServiceQuery<Employee>;

//so in callback it has to be collected in the same datatype which is what is written

//in when next querytoken is null (since we are still to collect it).

eq.BeginExecute(new AsyncCallback(MyCallback), eq);

}

public void MyCallback(IAsyncResult result)

{

//slot to hold returned result

IEnumerable<Employee> response = null;

if (token != null) //this is executed everytime we have clicked on next button

{

//since in next button click we used DatabaseEntities to fire query , we need to typecast it to the same type

var eq = result.AsyncState as DatabaseEntities;

//collect the response

response = eq.EndExecute<Employee>(result);

}

else//this "else" is executed only for the first call to WCF data service; we get token only after first call

{

//since the first call was made by DataServiceQuery<Employee>, we are typecasting it to the same type on first callback

var eq = result.AsyncState as DataServiceQuery<Employee>;

//collect the response

response = eq.EndExecute(result);

}

//bind the data to datagrid (but do it before typecasting response to get token

//or collect data first in separate instance and then typecast the returned result to get next token)

dataGrid1.ItemsSource = response.ToList();

//collect next page uri by typecasting the returned response

var qor = response as QueryOperationResponse<Employee>;

token = qor.GetContinuation();

}

private void button1_Click(object sender, RoutedEventArgs e)

{

//if there are more pages

if (token != null)

//fire the next page query to proxy of WCF Data Service

myDB.BeginExecute(token, new AsyncCallback(MyCallback), myDB);

}

}

Comments/error always welcome. Contact me if you need a working sample code and if anything below did not worked.

Today is October 15th 2009 and I am writing this post to express my understanding about parallel programming using Parallel Extensions (or Task based parallel programming); specifically to express why should we go for this approach.

Since the Parallel Extension announcement, I was excited to get my hands dirty with it but after working with Parallel Extensions, I had few initial questions that I thought everyone would have and there is less content available online that targets against benefits of Parallel extensions. Below are a couple of questions that I am targeting in this post.

• Is Threading going to die? Are Parallel Extensions replacements to Threading?
• Sometimes Threading gives me better performance than Task based programming. Why?

Before we come to these questions, it would be worth understanding options other than parallel extensions that we used to follow for parallel programming. Before Parallel Extensions, we would use primarily any of the below two modes for parallel programming:

Directly Creating Threads:
Here we create Threads directly in our code something similar to the following code:

Thread t = new Thread(DoSomeWorkMethod);
t.Start(someInputValue);

Benefits:
The benefit of this approach is that each of the thread will run at all times. .NET Framework’s scheduling algorithm will make sure that all the threads that we created will execute in similar to RoundRobin algorithm (though actual algorithm is different). This approach is especially useful in cases where I want to provide justice to all threads.
Another benefit associated with this approach is the fact that we have control over these created threads like aborting a thread in the middle.

Drawbacks:
The drawback of this approach comes into picture when we have more threads than the available CPUs (cores). As an example, if we have two cores and I am running 10 threads, context switching, invalidation of each thread’s cache etc. So we need to keep a balance between how many CPUs we have and how many Threads should we create for optimum performance.

Using Thread Pool:
.NET Framework has a thread pool (configurable) which limits the number of Threads that can be run at a time. Now when we want to execute something in a separate thread, we simply queue our work on a thread of thread pool instead of creating our own separate thread. Code looks something similar to below:

ThreadPool.QueueUserWorkItem(
DoSomeWorkMethod, someInputValue);

Benefits:
Thread Pool size for .NET Framework can be set by doing a simple calculation using number of CPUs the machine has. After this, Thread Pool makes sure to run only those number of threads that are optimal for a particular machine. This prevents any context switching and other overheads. So we get best performance.

Drawbacks:
The drawback of this approach is that developer doesn’t have much control over the Queued tasks over Thread Pool. In other words, once we have queued a work item, we do not get a reference to it and there is no explicit support for knowing when it completed, or for blocking until it completes or for cancelling it, or for identifying it in the debugger via some sort of ID etc. Plus, you are not guaranteed when your Task will start to run, so progress is unknown.

Using Parallel Extensions:
This approach is the one that we are here to discuss. Task based programming combines the benefit of both the approaches with some new additions. Using task Based programming, you create Tasks instead of Threads and start executing them. Programmatically they are almost identical to Thread Based programming with additions features. Here is a code snippet that uses the new Task class:

Task t = Task.Create(DoSomeWorkMethod,
someInputValue);

Benefits:

• With Task Based programming, by default, you can be sure that we are not actually decreasing the performance by doing excessive multitasking since Task Manager (central class in Parallel Extensions) takes care of this issue by queuing any additional tasks if all the existing CPUs are busy.
• With Task based programming you have complete control over the Task. You can pause, abort, wait, join etc.
• The scheduling algorithm is improved over Thread Pool approach with new features like “Task Stealing”.
• You get additional constructs to easily work with, like Parallel LINQ, Parallel For Loop etc.

Drawbacks:

Personally I see the only drawback is that you are not guaranteed when the Task will execute. So if you want to guarantee of starting your Task execution as soon as it is created, use plain old Threading Way. UI multitasking as an example should be done with Threading and not Thread Pool or Task based programming.

I will also share one interesting incident that happened while I was developing a Proof Of Concept. I was running the samples on a 4 Core Machines and created 2 Threads to do a Task and then 2 Tasks to do the same task and I could not see any improvement. Then I realized that if I am creating Threads which are less in number than available free CPUs I might actually get some greater performance sometimes, but that is not a guarantee. With task based programming, I can guarantee that either all my Tasks are running in parallel or all CPUs are busy executing my tasks.

I will try to cover benefits approach in future posts but for now, it is done.

I hope this post was useful. If you are a Microsoft Partner, please feel free to involve our team for all your development consultation needs by contacting us here. Comments, corrections; I Love them.

Rahul Gangwar

Hi,

Today is July 28th 2009 and I am writing this small post to share my understanding on Gemini and Business Intelligence arena and specifically around Analysis Service.

Talk about data in Microsoft and the very first thing that comes in mind is SQL Server. Microsoft hardly released SQL 2008 and we are looking forward for next version of SQL Server. That is very ambitious.

When it comes to Business Intelligence problem, things like Reporting, Analysis, data-mining come automatically forms part of the solution. But there are few patterns that should be considered for developing an efficient BI solution. Before talking about Analysis Service, let's understand how operations are different in OLTP vs OLAP databases.

In OLTP databases, data is written in form of rows (relational databases). So for example, if I want to store two records of two column structure having (name and email as columns) the storage will go in a fashion:
"Rahul" "rahul@contoso.com", "Sabiha" "sabiha@contoso.com".
Eeffectively, data is written over disk one row at a time. This makes typical OLTP transactions efficient  since I can quickly read/write a all columns of ONE record somewhere in middle of 50 million rows. But when it comes to reading ALL records (let's say email of all 50 Million records), relational databases fail miserably in terms of performance.

In OLAP databases, data is stored in a different fashion. It is stored in the form of columns rather that rows. So if I want to store the same set of records in columnar fashion, it would be stored this ways:
"Rahul" "Sabiha", "rahul@contoso.com" "sabiha@contoso.com".
The benefit and drawbacks are here.
If I want to read columns of ONE record somewhere in middle of 50 million rows, it will fail miserably in comparison to relational databases. But when it comes to reading a column of ALL records, this will be surprisingly fast (and this requirement is very common in Analysis: to read large number of records rather than one record).

So one thing is clear that columnar databases are better than relational databases for data analysis.

Another thing that need to be considered while designing analysis solution is the storage medium. Databases that use disk as the storage medium will perform very slow for performing analysis algorithms than the databases that use in memory medium as storage. Storing data in memory is fast because of the medium speed as well as saving CPU cycles. Although the durability property of ACID is lost in this process (since memory is volatile) but this still holds a good candidate for analysis solutions.  Even hybrid storage mechanisms can be considered depending on the requirement scenario.

Now comes Microsoft's project codename "Gemini". Gemini is next version of Analysis Service based on the concepts of using columnar database and in memory storage, making analysis of data hell fast. Gemini will be shipped in the next version of SQL Server (2008 R2/Kilimanjaro).
The good thing is that Excel users can use its capability by installing a plug-in. Hence the term "self service analysis". Every user in an organization will be able to analyse data and take decisions using simple Excel. More here:
https://www.microsoft.com/presspass/press/2008/oct08/10-06BI08PR.mspx 

I hope this gives a peek about what Gemini is and how it will work.

Comments/feedbacks welcome.

Rahul Gangwar