The Azure Manifesto https://docs.microsoft.com/archive/blogs/the_azure_manifesto/feed.xml Wed, 17 Apr 2019 03:54:07 GMT en-US hourly 1 Blockchain 101 - Developing using Azure Services https://docs.microsoft.com/archive/blogs/the_azure_manifesto/blockchain-101-developing-using-azure-services Mon, 15 Aug 2016 15:52:52 GMT https://blogs.msdn.microsoft.com/the_azure_manifesto/?p=55 BlockApps - an Ethereum environment used to quickstart authoring of Smart Contracts. But how does one actually start developing on the blockchain? For quick prototyping, a BlockApps Bloc webserver is useful. Once development has begun, you can replace the local server with Azure's BaaS offerings. This post will provide you with both the tools to get started and demonstrate a complete sample application. The sample project, BlockMobile, uses a private Ethereum Blockchain as a backend and Xamarin.Android for interaction on the frontend. Additionally, the project uses an AngularJS webapp and an Azure Mobile Service app to provide a complete demo environment.

What You Will Learn

  • Microsoft and Blockchain
  • Smart Contracts
    • What and Why?
    • How? Using Solidity
  • Using BlockApps to interact with Blockchain
  • Setting Up a Frontend
  • Realworld Use Cases

Video Tutorial and Information

https://youtu.be/ZiZgH8l8f_g

BlockMobile

BlockMobile is provided as an open source Blockchain project that is shown in the video above. You can read more about it on Github. The project will provide you with sample Smart Contracts, a Xamarin.Android app, and an AngularJS webapp. To run the project, follow the instructions provided.]]> Java and DNS Caching https://docs.microsoft.com/archive/blogs/the_azure_manifesto/java-and-dns-caching Tue, 20 Oct 2015 11:54:00 GMT https://blogs.msdn.microsoft.com/the_azure_manifesto/2015/10/20/java-and-dns-caching/ I ran into some problems with a VM on Azure trying to resolve an outside URL. The problem was that even though the URL existed and was assigned, the Azure DNS server wouldn't resolve it and therefore my java code would return an Unknown Hostname exception. 

I wrote some code to debug this and quickly realized that the problem wasn't really the Azure DNS and rather something within Java. The behavior is when a name doesn't get resolved  for whatever reason the "negative" result gets cached by Java and therefore any subsequent queries for the name doesn't get queried. So I wrote a little Java code to explore this more. 

The key thing is, prior to making the call for the nameserver we must first  set the cache TTL value. There are two of these; 1. for the positive or successful queries and 2. for the negative or the failed queries. The values for each can be found at here. 

networkaddress.cache.ttl (default: -1)
Indicates the caching policy for successful name lookups from the name service. The value is specified as as integer to indicate the number of seconds to cache the successful lookup. A value of -1 indicates "cache forever".

networkaddress.cache.negative.ttl (default: 10)
Indicates the caching policy for un-successful name lookups from the name service. The value is specified as as integer to indicate the number of seconds to cache the failure for un-successful lookups. A value of 0 indicates "never cache". A value of -1 indicates "cache forever".

        java.security.Security.setProperty("networkaddress.cache.ttl", successfulCacheValue);

        java.security.Security.setProperty("networkaddress.cache.negative.ttl", failedCachedValue);

Just in case you want to confirm the set value you can read it like this: 
    
System.out.println(
        "Positive cache: " + java.security.Security.getProperty("networkaddress.cache.ttl") + "\n" +
        "Negative cache: " + java.security.Security.getProperty("networkaddress.cache.negative.ttl")
);
Next attempt resolve the name. In my case the url was passed-in as a command line argument.
try {
        InetAddress inetAddress = InetAddress.getByName(args[1]);
        displayData(args[1], inetAddress);
} catch (UnknownHostException e) {
        System.out.println("Unknown hostname");
        e.printStackTrace();
}
 
public static void displayData(String hostName, InetAddress inetAddress) {
        System.out.println("Hostname: " + hostName);
        System.out.println("Canonical Host Name:" + inetAddress.getCanonicalHostName());
        System.out.println("Host Name:" + inetAddress.getHostName());
        System.out.println("Host Address:" + inetAddress.getHostAddress());
}
 
And that's it. Happy coding.. 
]]> Gluster on Azure - Part 2 https://docs.microsoft.com/archive/blogs/the_azure_manifesto/gluster-on-azure-part-2 Mon, 26 Jan 2015 11:41:00 GMT https://blogs.msdn.microsoft.com/the_azure_manifesto/2015/01/26/gluster-on-azure-part-2/ This is part 2 of installing and configuring Gluster on Azure. For information on setting up the VM’s on Azure installing Gluster check out this article.

Once we’ve installed Gluster we need to configure it. The first step is to ensure that all machines can see each other. In order to do this we need to get the Internal IP Address (internal to Azure) of each server. You can get this information from the Dashboard of each server VM or by running the ifconfig command on each VM.

1. From each server make sure you can Ping all other servers in the cluster to ensure that you have the correct IP addresses.

2. Once the IP addresses are confirmed edit the hosts file to map the IP’s to a simple name. We do this on all machines in the cluster.

 

[root]# vi /etc/hosts

3. Add the IP addresses to the hosts file and save the file:

 

#VM IP        #FQDN   #NAME 
xxx.xx.xxx.94 gluster1 gluster1 
xxx.xx.xxx.19 gluster2 gluster2

4. Now that we have the Hosts file set we can start setting the connections. We do this using the “gluster peer probe” command. We do this for all the servers that we want to connect including the local server itself.

 

[root]# gluster peer probe gluster1 
peer probe: success
[root]# gluster peer probe gluster2 
peer probe: success

 

5. Now we can check the connections and ensure that the state is “Connected” on all machines.


[root]# gluster peer status Hostname: xxx.xx.xxx.19 Uuid: 77d27dac-6544-47b9-8d50-29945008b24e State: Peer in Cluster (Connected)

 

6. Once all machines are connected we can create volume that essentially connects all drives together.  Here’s the parameter of how this command works.


gluster volume create NEW-VOLNAME [stripe COUNT] [replica COUNT] [transport <tcp | rdma>] NEW-BRICK ...
[root]# gluster volume create glusteredData stripe 2 transport tcp  gluster1:/mnt/gluster gluster2:/mnt/gluster force

 

So the above creates a volume called “glusteredData” across 2 servers. The bricks (servers in the gluster) reference the mount points that we created.

 

9. The last step in the process is to start the volume.


[root]# gluster volume start glusteredData

 

10. Test the gluster, you can run:


[root]# gluster volume info 
Volume Name: glusteredData 
Type: Stripe 
Volume ID: eb0948e5-11b3-484c-8855-bdc167be8f85 
Status: Started 
Number of Bricks: 1 x 2 = 2 
Transport-type: tcp 
Bricks: 
Brick1: gluster1:/mnt/sharedDisk 
Brick2: gluster2:/mnt/sharedDisk

 

 [Connecting to the Gluster from a Client]

Similiar to the Gluster-server setup we need to install the glusterfs on the client.

<optional> You can copy the server’s Hosts file to the client. This way you can use the machine name to reference the Gluster point.

Next create a mount point in the /mnt/nas

 

mount -t glusterfs HOSTNAME-OR-IPADDRESS:/VOLNAME MOUNTDIR
[root]# mount -t glusterfs gluster1:/glusteredData /mnt/nas

 

and that’s it. You can check the size of the drive by:


[root]# df -h 
FilesystemSizeUsedAvailUse%Mounted on 
/dev/sda1 31G1.2G29G4% / 
none 4.1k 0 4.1k 0% /sys/fs/cgroup 
udev 866M 4.1k 866M 1% /dev 
tmpfs 176M 345k 176M 1% /run 
/dev/sdb1 74G 55M 71G 1% /mnt 
gluster1:/glusteredData 1.5T 1.2T 220G  85% /mnt/nas
]]> Gluster on Azure - Part 1 https://docs.microsoft.com/archive/blogs/the_azure_manifesto/gluster-on-azure-part-1 Mon, 26 Jan 2015 11:31:00 GMT https://blogs.msdn.microsoft.com/the_azure_manifesto/2015/01/26/gluster-on-azure-part-1/ How do you store and share large files, and I mean really large files, on Linux VMs on Azure? Maybe it’s best if I describe it this way.. I have several large files with sizes from 500GB to 1TB+ with a total size of 100TB+. I want to be able to share these files across multiple Linux VM’s in Azure and access them using the standard I/O calls of Linux.

Azure has several storage solutions including Azure Blob and Azure File Share. The Azure Blob has maximum file size of 1TB if you store the files as page blobs or 200GB if you store the files as blocks with a total maximum size of several hundred terabytes. It’s important to note that when you attach a disk to your VM’s the disk is actually created on the Azure blob (keep this in mind because it will come in handy later in this article).  But unfortunately once you attach a disk to a Linux VM, you can’t attach the same disk to another VM.

Azure File Share is a little better as it does in fact lets you share files across multiple VMs using the standard SMB 2.1 protocol. Microsoft Azure virtual machines and cloud services can share file data across application components via mounted shares, and on-premises applications can access file data in a share via the File storage API. Since a File storage share is a standard SMB 2.1 file share, applications running in Azure can access data in the share via file I/O APIs.  It’s important to note that the maximum file size on the share is 1TB and the total maximum share size is 5TB.

So here’s the problem again: How do you share a file that’s 1TB+ across multiple VM’s running Linux?

The Gluster File System (GlusterFS) is an open source distributed file system that can scale out in building-block fashion to store multiple petabytes of data. Through out this article I will walk you through step by step instructions on your setup GlusterFS on Azure.

In this article I will setup 1 storage account and 3 Linux VM’s running Ubuntu 14.04. Two of the Ubuntu VM’s will host my Gluster file system and the last VM will act as the client which will access the files on the Gluster servers. Of course you’re not limited to this configuration and can add or remove machines both as a Gluster server and as a Client.

A sample Gluster Topology in Azure

A sample Gluster Topology in Azure

1. The first step in the process is to create a storage account. From the Azure Management Portal, Click onNew, Data Services, Storage, Quick Create. Type in the name of your storage account as well as selecting the location etc.

Create a New Storage account

Create a New Storage account

2. We need to create several Linux VM’s (I chose Ubuntu 14.04) by selecting: New, Compute, Virtual Machines, From Gallery. Once the wizard starts, select Ubuntu from the left menu and then choose Ubuntu Server 14.04 LTS. Click on the Next arrow to go to next page.

3. Here you input the regular standard VM setup things including the VM name, username, password etc. Take a note of how you’ll be connecting to the VM (via username and password vs. using certificate). This method of authentication will be used to connect to the machine using SSH. Once you’re done with this configuration click on the Next arrow.

4. On this page the important piece is to select the storage account that we created in Step 1 and finish the wizard.

Select Storage Account

Select Storage Account in VM wizard

5.  Repeat the above 4 steps for all your VMs.

6. Once finished go to the dashboard page for each server. On this page take a note of the server ports and configuration on the right hand side. On the bottom of the page select Attach Disk. On the screen input the disk size. I am chose 750GB for testing but you can attach any disk up to 1TB. The size restriction is because the disks are stored as blob files in the Azure blob storage which limits the file sizes to 1TB. Repeat this step for all your other servers.

VM Info and Disk

VM Info and Disk

7. I use Putty to connect to the Linux VM’s and for simplicity I inputted all the connection info (found on the right-hand side of the Dashboard page of each VM) into Putty. Connect and login to your first VM server.

[Setting up the newly attached disk]

8. In order to initialize and mount the new hard drive, you must find the drive name. If you go to the Azure site here, you will see that it says to run

 

# sudo grep SCSI /var/log/messages

or if you’re running Ubuntu run:

 

# sudo grep SCSI /var/log/syslog

Unfortunately neither of these worked for me. But finding the newly added drive is fairly easy.  The drive that you attached is normally the last drive in the /dev directory that starts with “sd” followed by a letter (not a letter and a number but just a letter). For me, it was sdc. So in order to initialize the drive we need to run the following:

NOTE: You have to run the following commands as a root or a super user. (run sudo su or you can type sudo to the beginning of all commands).
 
[root]# fdisk /dev/sdb 
Command (m for help): m 
(Enter the letter "m" to get list of commands) 
Command action 
a toggle a bootable flag 
b edit bsd disklabel 
c toggle the dos compatibility flag 
d delete a partition 
l list known partition types 
m print this menu 
n add a new partition 
o create a new empty DOS partition table 
p print the partition table 
q quit without saving changes 
s create a new empty Sun disklabel 
t change a partition's system id 
u change display/entry units 
v verify the partition table 
w write table to disk and exit 
x extra functionality (experts only) 

Command (m for help): n 
Command action e extended 

p primary partition (1-4) p 
Partition number (1-4): 1 
First cylinder (1-9729, default 1): 
Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-9729, default 9729): 
Using default value 9729 

Command (m for help): w 
(Write and save partition table)

 

9. Next we need to format the disk.

 

[root]# mkfs.ext4 -L dataDisk /dev/sdc1

10. We need to mount our formatted drive.

 

[root]# mount /dev/sdc1 /mnt/sharedDisk

And that’s it for the disks. Again make sure you do the above steps on all your servers.

 

[Installing GlusterFS]

11. Before installing Gluster we need to setup its repository. For the latest Gluster version and package information check out this site. Since I am using Ubuntu I can use the Ubuntu package manager. The first step is to add the Gluster PPA. This way I don’t need to worry about manually downloading and building the source code. This also makes updating Gluster a lot easier.

 

add-apt-repository ppa:semiosis/ubuntu-glusterfs-3.5 apt-get update

Finally, install the packages:

 

apt-get install glusterfs-server

To ensure that Gluster has been installed properly run the following:

 

glusterfs --version

Repeat the above steps on all your servers that will be part of the Gluster cluster.

]]>