Dr. Alp Asutay for Modern Cloud Applications

TFS’in Özelleştirilmesi

Alp Asutay — Thu, 30 Mar 2017 13:32:40 GMT

PROBLEM TANIMI

Code Review Süreci, TFS 2015’in sunduğu code review süreci, checkout edilen kod için review request’i istenmesi ve isteğin incelenip cevap vermesi ile tamamlanmaktadır, Hali hazırda TFS’in sunduğu bu süreç code review yollanıp yollanmadığına ve sonuçlarının olumlu olumsuz olduğuna bakmaksınız checkin yapılmasına izin vermektedir, Checkin esnasında file’lar lock olduğu için changeset’e ait code review work item üzerinde bulunan dosyaların değişip değişmediği kontrol edilememektedir
Auto Review Süreci, Checkin sırasında Grup/Kişi/Folder/Team Project bazlı bazı kısıtlar istenmektedir, örnek kısıtlar; i. Forbidden Files, ii. Forbidden Folders, iii. Forbidden Extension-Folder, iv. Allowed Extensions, Bu kısıtların tanımlanması için TFS’in sistemi yeterli olmamaktadır, aksiyondan girilen herhangi bir proje yetki talebi bir otorite’den geçmeden alınabilmektedir.
TFS’in sunduğu Visual Studio ile entegre olabilen client side plugin’lerin kullanımı dağıtımda problem yaşabileceği için tercih edilmemektedir.
Çalışacak yapıya ait bazı konfigurasyonel paremetrelerin/yetkilerin tutulacağı bir yapıya ihtiyaç vardır, örnek; Codereview minimum kaç kişi tarafından onaylanmalı, Hangi team project’ler için code review süreci işletilmeli, Grup/Kullanıcı/Team Project/Folder/Yetki bilgileri,
Hazırlanacak yapının tek taraftan yönetilebilmesi adına work item policy ve comment policy gibi TFS’in sunduğu custom policy’lerinde yeni yapı üzerinde implemente edilmesi gerekmektedir,
Kullanılan branch yapısından dolayı Folder bazlı konfigurasyonlara ihityaç vardır TFS bunu destekleyememektedir. Team Project altındaki farklı development gruplarına ait farklı projeler bulunmaktadır
Gated checkinde unit test kontrolü yapılırken, tüm projede unit test kontrolü yapılması yerine sadece değişen dosyalara ait unit testlerin kontrolü yapılması adına, changeset’de bulunan dosyaların adı ve pathine ihtiyaç vardır.
Hazılanacak konfigurasyon yönetim veri sisteminin TFS’in default collection’ından farklı bir yerde tutulması gerekmektedir.

ÇÖZÜMLER 1) Code Review sürecinin yeniden tasarlanması, a. Checkin notification evet üzerinde ilgili dosyalara erişilemediği için çalışma gated checkin içerisinde yapıldı, b. Hali hazırda TFS’in code review süreci üzerine gated checkin sırasında araya girip code review kontrolü yapabilecek bir code activity projesi hazırlandı, c. Hazırlanan proje içerisinde checkin yapılmak istenen file’lara ait code review bulunmadığına bakılır, code review bulunuyorsa, code review’lara ait response’lar analiz edilir, olumlu response’lara ait sayı tanımlı minimum olumlu response sayısından fazla ise checkin’e izin verilir. d. Checkin izini sırasında file’lar üzerinde review sonrasında değişiklik olup olmadığına her bir dosyanın hash value’su kontrol edilerek bakılır. 2) Code Review öncesinde, checkin sırasında gated checkin öncesinde, checkin yapılan dosyalar ile alakalı kısıtları konumlandırmak için server side olan checkin notification eventi kullanıldı, a. Checkin notification eventini içeren bir proje hazırlandı, hazırlanan proje TFS Server üzerinde plugins folder’ina library olarak eklenerek çalıştırılması sağlandı, herhangi bir şekilde client’lara dağıtılmasının önüne geçildi, b. Kısıtlar implemente edilerek, herhangi bir kısıta uymayan dosyalar checkin sırasında red edilerek kullanıcıya bilgi verildi. c. Kısıtların tanımlarına ulaşabilmek için bir başka proje daha hazırlandı. 3) Server Side hazırlanan checkin notification projesi ve gated checkin projeleri ile client üzerinde ekstra herhangi bir proje yazılmasına veya dağıtılmasına gerek kalmadı. 4) Konfigurasyon parametrelerinin ve yetkilerin tutulması için bir veri tabanı tasarlandı. 5) TFS’in sunduğu custom policy’ler yeni hazırlanan yapı üzerinde de implemente edildi. 6) Yetkilerin folder bazlı verilmesi sağlandı. 7) Gated checkin sırasında değişen dosyalara ait bilgilerin drop folder’da saklanmasını sağlayacak bir code activity yazıldı, gated checkin projesine dahil edildi. 8) Hazırlanan veri tabanı TFS’in default veritabanından ayrıştırıldı. Veri Tabanı Tasarımı Konfigürasyon parametrelerini saklamak için TFSConfiguration tablosu hazırlandı, Grup ve User’lara ait bilgiler, CheckinGroups ve CheckinUsers tablolarında tutulmakadır, Yetkilere ait bilgiler, CheckinPermissionTypes tablosunda bulunmaktadır, CheckinUserFolder,CheckinUserFolderPermissions, CheckinGroupFolder ve CheckinGroupFolderPermissions tabloları ile permissionlar yönetilmektedir. Solution Tasarımı Hazırlanan .NET solution 4 farklı projeden oluşmaktadır. CodeReviewBuildTemplate projesi ile code activity’ler build sırasında kullanılabilecek hale gelirler, build sırasında alınması gereken BuildDetails gibi parametreler bu proje içerisinde verilir, CheckinNotification Projesi ile checkinNotificationEvent yakalanır ve gerekli restrictionlar işletilir, GatedCheckinReviewPolicy projesinde gatedCheckin’deki code review kontrol süreci işletilir, PolicyConfiguration projesi ile tüm konfigürasyon bilgileri ve yetkiler veri tabanından alınır.

Modern Entity Framework - Row & Column Based Security

Alp Asutay — Thu, 15 Dec 2016 11:03:39 GMT

Prepared by L.C Waikiki Team(TAYFUN.ESMER@Lcwaikiki.com) and Dr. Alp Asutay 1. Introduction The last few years have seen the rise of Entity Framework object-relational mapper. While getting more robust and improved with each version compared to previous ones, we have been introduced to interceptors starting with Entity Framework 6. Those interceptors allow us to intervene the queries executed against SQL, the result sets coming back from SQL and even the exceptions related to the executions. If you have decided to learn more about interceptors, you must have come across a few implementations on the internet like logging and soft delete concepts. But what we aim here is much more complex and advanced scenario which is going to serve for our security policies based on rows and columns of a table. At the end of the day, Row Based Security will enable us to implement restrictions on data row access. For example ensuring that employees can access only the data pertinent to their department, or a group of accountants can access the orders only with less than $10.000 total price. Column Based Security on the other hand, will enable us to implement restrictions on columns. It's the data will be restricted again as Row Based Security, but the target is all data under specified columns. 2. Solution & Implementation We all know that limiting data means filtering it, which is done with the “where” keyword. So basically we will apply predicates to the set of entities when queried, at application tier and independently of developers. There are two interceptor levels for the queries before they go out the gate and get executed in SQL. One is DbCommand Interceptor, which is the SQL level and carries DbCommand object which is the very same object SqlCommand derives from. That is not the field we want to work on by altering the SQL command before it gets executed. The other one is DbCommandTree Interceptor, which reveals itself from its name and makes it possible to work with expression trees and alter them with the help of visitor objects. That’s the path we will take to achieve our goal more elegantly. So, first things first. We are going to create policy expressions holding the main data which defines the target for security policy to be applied: environment, server, database, table (entity), column, field (property of the entity) and even the values or the source of them. You can see the data model below. By holding the environment information along with the server and database, we can apply different policies for a user on different environments such as test, prod, development etc. DbObject and DbObjectField tables are for the entity and its property. These policy securities can be associated with claims, claim groups and users specifically, and the information for that is held in AuthorizationRef and AuthorizationDefinitionRef fields in PolicySecurity table. The journey of the code written for the implementation of this solution begins as the user steps in our application, which is the Session_Start event for web applications. There we query all the row and column based security policies associated with the incoming user with all the data needed, transform them into DataPolicyExpression objects and store them in distributed cache which happened to be Redis Cache for us. Notice that the PropertyValueSource value is static with a value next to it which is "2" at this example. However, the source could be External meaning the data for the expression is going to be pulled from an object which implements the interface (IExternalPolicySecurityService) this solution exposes with a method returning Dictionary typed value, holding the key for that external values and the actual values (probably a set of IDs of other tables) how and wherever they are coming from. So in that case instead of a static value, we hold that key name of that external source. Once the concrete class implementing that interface is found on the web project, the method of it is invoked and all the data is returned for the security policy to be used most likely with IN operator instead of equal (=). That external value source option allows us to define policy with dynamic data which needs to be executed at runtime and involves other tables. Other than these two options, we can store our policies with internal value sources, which refers to endpoints of services. Instead of a concrete class implementing a specific interface, a web service can also be used for the same reason but different technical needs. As in the external value case, that service again will return a Dictionary typed value. Again in our case this internal value source working with web services always served us set of IDs of other tables to be used along with the IN operator. We have two separate interceptors for row based policies and column based policies. These two interceptors are registered and waiting for any query to catch before it is executed in SQL Server. At that point, when the TreeCreated event is triggered, we pull the data from the cache and see if the executing query holds any matching entity (table). If so, the policy information stored in DataPolicyExpression object is used to build DbFilterExpression with dynamic filters, to be attached to the query. Building that expression doesn’t happen for every query hit. Instead, when it is created for the first time, it’s stored in memory for the later hits for the same entity. That process could have done much earlier, like in the Session_Start event for every single user. But that could depend on the number of policies defined for each user, how complex the policies therefore the expression needs to be build are complex, and of course we cannot know for sure if the user is going to hit that table (entity) in that session at all. So the implementation may be different for some technical and performance purposes but at the end of the day with the help of the interceptors, we can apply new filters or replace the existing ones based on our policies. 3. Problems & Solutions Of course we had some troubles making our code work during this implementation. Firstly, the order for the interceptors to work mattered in a case where a column which both column and row based policies are applied for. Because the data was replaced with the default value of that type –empty string in our case-instead of the column name, again that default value was used in the where clause which caused the queries to have false conditions all the time. It looked something like this: SELECT Foo1, Foo2, N’’ FROM tb_Table WHERE N’’ = ‘UK’ Instead of WHERE Foo3 = ‘UK’, the column name was replaced by the default value which was put there due to a column based security for the user not having rights to see that specific column. We simply solved the problem by registering the interceptors in the right other. Another problem we faced was the query cache mechanism of Entity Framework. The problem rise when an entity is queried more than once. When a value is set in a filter parameter, that value is cached and reused for the further queries. This happens because the interceptors we use implements IDbCommandTreeInterceptor and the execution of the TreeCreated method happens only once for the query commands, being cached after. Even for a new instance of DbContext won’t make any difference, we’ll be stucked with the cached values for the parameters. This situation might be a problem for almost any kind of interceptor works, unless it is using a hard coded value such as true/false for soft delete purpose implementations. While there are a few different work arounds (using functions that produce queries with constants or using properties of a non-mapped objects and such) for this problem, we decided to go with the IEnumerable collection wich is always remain empty. Entity Framework does not cache queries that involves IEnumerable.Contains(T value) call against an in-memory collection, since the values of the collection are considered volatile. That method (Contains) call can be executed in a wrapped DbSet object or at a point of base repository like object for it so no matter what entity the query comes for, Entity Framework won’t replace it with the cached one.

Using Esri in Single Page Application frameworks

Alp Asutay — Fri, 01 May 2015 05:50:00 GMT

Prepared by MSc. İbrahim ULUDAĞ (ibrahim.uludag@turkcell.com.tr) and Dr. Alp Asutay Durandal is a modern cross-device, asynchronous cross-platform client framework written in JavaScript and designed to make Single Page Applications (SPAs) easy to create and maintain. This framework is built on latest technology libraries and web components including jQuery, Knockout and RequireJS as AMD loader. Esri's ArcGIS API is a geographic information system (GIS) for working with maps and geographic information. It is used for: Creating and Using Maps;
Compiling geographic data; analyzing mapped information; sharing and discovering geographic information; using maps and geographic information in a range of applications; and managing geographic information in a database. ArcGIS API uses Dojo as AMD Loader. Since they use different AMD loaders, using them together is not trivial. Esri states that the ArcGIS API can be only used with Dojo. Durandal suggests to replace RequireJS with Dojo but it is not desired to change one of the core library of Durandal. There is a workaround to use ArcGIS API in Durandal without replacing RequireJS. But there are some issues of this
technique.

You cannot use CDN. The files must be placed in the project.
Some effort and much care is required when going to production

The versions of RequireJS and ArcGIS API is important. This technique may not work for older versions. The minimum required version of RequireJS is 2.1.16, and 3.12 for ArcGIS API. Here are the steps: Download ArcGIS API 3.12 from ArcGIS web site. Extract and copy the files into your project. For example you can place ArGIS file bulk into Scripts folder. Add ArcGIS css to html page. It can be used either from CDN or local.
Configure RequireJS paths for dojo, dijit, dojox and esri requirejs.config({
    paths: {
        'text': '../Scripts/text',
        'durandal': '../Scripts/durandal',
        'plugins': '../Scripts/durandal/plugins',
        'transitions': '../Scripts/durandal/transitions'
        'dojo': '../Scripts/arcgis_js_v312_api/arcgis_js_api/library/3.12/3.12/dojo',
        'dijit': '../Scripts/arcgis_js_v312_api/arcgis_js_api/library/3.12/3.12/dijit',
        'dojox': '../Scripts/arcgis_js_v312_api/arcgis_js_api/library/3.12/3.12/dojox',
        'esri': '../Scripts/arcgis_js_v312_api/arcgis_js_api/library/3.12/3.12/esri'
    }
}); For production, Grunt is used. Add the paths as empty in the Gruntfile.js. var requireConfig = {
      baseUrl: 'app/',
      paths: {
         "jquery": 'empty:',
         "knockout": 'empty:',
         'bootstrap': 'empty:',
         'text': '../Scripts/text',
         'durandal': '../Scripts/durandal',
         'plugins': '../Scripts/durandal/plugins',
         'transitions': '../Scripts/durandal/transitions',
         'dojo': 'empty:',
         "dijit": "empty:",
         "dojox": "empty:",
         "esri": "empty:",
      },
      catchError: true
}; Remove custom-almond in your main file by adding almondFilter and add to tasks. (Configure src and dest according to your configuration), require.js file will be added later for release mode. replace: {
  almondFilter: {
src: ['build/main.js'],
   dest: 'build/mainalmondFilter.js',
   replacements: [{
     from: /var requirejs, require, define;(.|[rn])*define("../Scripts/almond-custom", function(){});/g,
     to: ''
}]}}, grunt.registerTask('default', [
       ....
      'durandal:main',
      'replace:almondFilter'
]); After following these steps, ArcGIS API can be used in Durandal SPA projects. But there are still steps before going to production and it is important to follow below steps for each release. Because ArcGIS API makes approximately 200 HTTP requests and this affects the performance of the application. These requests must be combined into one file. Generating production from ArcGIS web site causes problems. Because it adds dojo to script. Since RequireJS is used as AMD Loader, they conflict with each other. The production code will be generated manually. The step are as below:

Open Fiddler. Navigate to all your map pages and consume every action you have implemented. For example, if you have written onlick event, make sure you trigger it. The reason is to load all AMD modules.
In fiddler, sort by name url. Select all URLs in the ArcGIS folder. Export by File > Export Sessions > Selected Sessions > Raw.
Merge all the content of the exported files into one file. Copy the merged file into project. (e.g. ~/Scripts/all-arcgis.js). You can use MergeFileContents.zip for merging.
Make the appropriate change in your cshtml file.

@if (HttpContext.Current.IsDebuggingEnabled)
{

}
else
{



}