Freddys Blog

Moving day...

FreddyDK — Fri, 08 Mar 2019 10:04:48 GMT

The MSDN blog platform is under retirement and I have decided to move my blog to be mastered in https://freddysblog.com The blog will also be syndicated to https://community.dynamics.com/business/b/freddysblog Meaning, that this is the final blog post on my MSDN blog. Over and out Freddy Kristiansen Technical Evangelist

What's new in NavContainerHelper 0.5.0.1

FreddyDK — Thu, 31 Jan 2019 14:05:54 GMT

Run As Administrator - or not!

NavContainerHelper now supports running as a standard user, but it does require you to add some permissions to this user:

Full Control to the folder C:\ProgramData\NavContainerHelper and all subfolders and files.
Modify permissions to the file C:\Windows\System32\drivers\etc (if you want to use the -updateHosts switch on New-NavContainer).
Full Control to the docker engine pipe as described in Tobias Fensters blog post here.

After installing or updating the NavContainerHelper, you can run the command Check-NavContainerHelperPermissions

Check-NavContainerHelperPermissions

During initialization of NavContainerHelper, it automatically runs Check-NavContainerHelperPermissions with the -silent parameter. This will check the permissions for the user and display any missing permissions. Note that if you are running as administrator, you do have the right permissions. You can also run Check-NavContainerHelperPermissions yourself and the output could look like this: You can ignore the hosts file permission check by adding -IgnoreHosts. Which also indicates that I can run

Check-NavContainerHelperPermissions -Fix

Check-NavContainerHelperPermissions -Fix -IgnoreHosts

to fix the permissions for me: Note that PowerShell might ask for permissions to run some scripts elevated in order to assign these permissions. After that, permissions are OK: The script for checking and fixing permissions can be found here: https://github.com/Microsoft/navcontainerhelper/blob/master/Check-NavContainerHelperPermissions.ps1

Manually assigning permissions

If for some reason the script doesn't work on your machine, you can also assign the permissions manually.

Full Control to the folder C:\ProgramData\NavContainerHelper and all subfolders and files.

Navigate to C:\ProgramData and open properties for the NavContainerHelper folder and add permissions for the user, it should look like this:

Modify permissions to the file C:\Windows\System32\drivers\etc

Only if you are planning to use the -updateHosts flag on New-NavContainer (where the hosts file is automatically updated with the IP number of the container), you need this permission. It should look like this:

Full Control to the docker engine pipe

Tobias Fenster has written a blog post and created a small PowerShell module, which fixes this issue. Please read Tobias Fensters blog post here.

Invoke-ScriptInNavContainer

In the new NavContainerHelper you will find a function called Invoke-ScriptInNavContainer and it does exactly like the name says - it invokes a script in a NavContainer.

Invoke-ScriptInNavContainer -containerName test -scriptblock { Get-NavServerUser NAV }

If you are running as an administrator, NavContainerHelper will create a PsSession to the container and invoke the script in a session. If you are running as an end user, NavContainerHelper will save the script in a shared folder and use docker exec to run the script in the container. The return value from the script will also be serialized into a file in the shared folder and deserialized in the script. Note: You can transfer Secure strings to the script (they will be transferred using a random encryption key), but currently Secure strings returned from the script is not supported/cannot be unpacked. Let me know if this is needed and I will consider. The biggest difference is really that outputting this to Host in your script will go to the Host if you are running as administrator and it will be returned as a return value if running as an end user. Example:

$a = Invoke-ScriptInNavContainer -containerName test -scriptblock { Write-Host "This is a test" }

When run as administrator, this will output the string "This is a test" to the Host and $a will be empty. When you run this as an end user, nothing will output to the Host and $a will contain "This is a test". My recommendation is that you avoid outputting to Host in the Scriptblock. All places within NavContainerHelper, where a script is invoked in a NavContainer has been replaced by a call to Invoke-ScriptInNavContainer.

Using the best container image

After releasing ltsc2019 images and NavContainerHelper 0.4.3.2, I could see that a number of people still would spin up 2016 images, simply because they were already downloaded. The policy was, that if you reference an image (eg. microsoft/dynamics-nav) without specifying a tag and without specifying -alwayspull - and you had that image downloaded, then NavContainerHelper would reuse that. This is still going to happen - but now you will see a note when starting the container, which indicates that you aren't using the optimal image: adding -useBestContainerOS will force NavContainerHelper to use the optimal image and even re-platform the image if needed.

New function Get-NavContainerImageTags

Try to run this script:

$imageName = "mcr.microsoft.com/businesscentral/onprem"
$tags = (Get-NavContainerImageTags -imageName $imageName).Tags | Where-Object { $_.contains("w1-ltsc2019") }

$tags will now contain all tags of ltsc2019 images with w1. While writing this blog post, that is:

13.0.24630.0-w1-ltsc2019
13.1.25940.0-w1-ltsc2019
13.2.26556.0-w1-ltsc2019
13.3.27233.0-w1-ltsc2019
cu1-w1-ltsc2019
cu2-w1-ltsc2019
cu3-w1-ltsc2019
latest-w1-ltsc2019
rtm-w1-ltsc2019
w1-ltsc2019

Grabbing all tags will right now return 572 image tags, which is 3 platforms (default (which is the same as ltsc2016), ltsc2016 and ltsc2019), 4 CU's, 2 CU tags (version number and CU) - and all of that times 20 localizations. On top of that there are a number of latest tags.

New function Get-NavContainerImageLabels

I have often found that I would like to know whether there is a new version of a specific image without pulling it. Get-NavContainerImageLabels will download the labels section of the image manifests without pulling the image.

Get-NavContainerImageLabels -imageName "mcr.microsoft.com/businesscentral/onprem:cu4"

will return nothing until cu4 ships.

Get-NavContainerImageLabels -imageName "mcr.microsoft.com/businesscentral/onprem:cu3

will (today) return

country : W1
created : 201901171250
cu : 
eula : https://go.microsoft.com/fwlink/?linkid=861843
legal : http://go.microsoft.com/fwlink/?LinkId=837447
maintainer : Dynamics SMB
nav : 
osversion : 10.0.14393.2665
platform : 13.0.27183.0
tag : 0.0.9.0
version : 13.3.27233.0

Indicating that the cu field in the labels isn't set, which is an error that will be fixed with the next rebuild round. When the images are being rebuild, the created date will also change. Also if you want to know the labels of the latest Business Central On Prem image:

Get-NavContainerImageLabels -imageName "mcr.microsoft.com/businesscentral/onprem:latest-ltsc2019"

Finally, if you want to have all Business Central tags, sorted by image creation date:

$imageName = "mcr.microsoft.com/businesscentral/onprem"
$tags = (Get-NavContainerImageTags -imageName $imageName).Tags | Where-Object { $_.contains("w1-ltsc2019") }
($tags | % { Get-NavContainerImageLabels -imageName "${imageName}:$_" } | Sort-Object -Descending -Property created).version | Select -Unique

Which won't return the images ordered by version number... If you want to see all versions of NAV 2018, ordered by version number, you need to

$imageName = "microsoft/dynamics-nav"
$tags = (Get-NavContainerImageTags -imageName $imageName).Tags | Where-Object { $_.startswith("11.0") } | Where-Object { $_.contains("w1-ltsc2019") }
$labels = $tags | % { Get-NavContainerImageLabels -imageName "${imageName}:$_" }
($labels | Sort-Object -Descending -Property version).version

You should see:

11.0.26893.0
11.0.26401.0
11.0.25466.0
11.0.24742.0
11.0.24232.0
11.0.23572.0
11.0.23019.0
11.0.22292.0
11.0.21836.0
11.0.21441.0
11.0.20783.0
11.0.20348.0
11.0.19846.0
11.0.19394.0

Enjoy Freddy Kristiansen Technical Evangelist

Process Isolation for containers in Windows 10

FreddyDK — Sun, 13 Jan 2019 08:02:56 GMT

For a few months we have known that Docker for Windows would get support for process isolation under Windows 10. Arend-Jan Kauffmann explained how to use nightly builds from Docker to get the feature early and test it. I of course jumped on this and have been running a nightly build of Docker since December 6th. A few days ago the feature was released in the edge release of Docker Desktop.

No updates?

I tried to check for updates, but this didn't reveal any new release. I replaced the docker executables with the original files and tried again - still no updates. I checked the edge release notes: https://docs.docker.com/docker-for-windows/edge-release-notes/ - no updates. I did some investigation and found that this was probably caused by Docker for Windows being renamed to Docker Desktop and the version numbering schema has changed.

This worked for me!

I decided to uninstall docker and install the edge release from here: https://hub.docker.com/editions/community/docker-ce-desktop-windows Unfortunately this also didn't go as smooth as expected. I ran into this issue: https://success.docker.com/article/dockerforwin-install-fails-on-installationmanifestjson - but fortunately, I could resolve this by following the resolution in this blog post. After installing, the About Docker dialog shows that I am running engine 18.09.1, which is the first version supporting the process isolation.

NavContainerHelper support (as of today)

NavContainerHelper 0.4.3.0 or newer will default the isolation mode to process when running Windows 10 1809 and Docker 18.09.1 (or a daily build with support for process isolation) and will display this in the output

Proof

Inspecting the processes on the host and in the container reveals that you indeed are running process isolation:

Recommendation

If you are running Windows 10, I recommend you to update to 1809 and update Docker to the 18.09.1 release. Running NAV/BC containers in process isolation is a HUGE win over hyperv isolation.

Enjoy

Freddy Kristiansen Technical Evangelist

Clean up after yourself Docker, your mom isn't here!

FreddyDK — Tue, 11 Dec 2018 15:04:26 GMT

Docker is an amazing tool. As Dynamics 365 Business Central or Microsoft Dynamics NAV developers we can spin up any version of the platform and app in minutes and use our favourite tools to work with it. Using the layering techniques from Docker, these images will be stored on disc as efficient as possible and every time I start up a new container, it reuses the disc space and your running container just becomes another layer on top of your image. Really really smart, but it does take up some disc space and it isn't very good at cleaning up.

Folders

Docker will store all images and containers in a folder called C:\ProgramData\Docker. You can modify the location of this folder when configuring Docker by changing a setting called data-root. Read more here: https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-docker/configure-docker-daemon NavContainerHelper will store extracted image files and object sources under C:\ProgramData\NavContainerHelper. I typically run Docker on Azure VMs, which doesn't live long. When I need a new version, I use http://aka.ms/getbc to spin up a new VM. I never really had any issues with these folders taking up space. I have however gotten the question from a number of people running Docker locally: How do I clean up after Docker? and after Windows 10 1809 came out and after Windows 10 supports process isolation (if you use Docker insider builds) I have started using my laptops more and more and I decided that I would investigate how this is done.

The process is simple

What we need to do is simple:

Remove all running Containers
Remove all images, build caches and other things Docker related.
Empty the NavContainerHelper folder
Pull the images you need

It is the how-to, which becomes complicated - especially #2. You might have tried to delete the folder manually or tried to delete individual layers, just to find that you get a ton of errors and if you finally succeed deleting things, you might have ruined your host operating system and only a reinstall will get you back up running. The reason for this is that the folders contains a lot of symbolic links (when they say that they are sharing the OS with the host, they mean it) and you need to be a backup/restore service in order to touch/delete these files. No user can do so, not even the administrator. So who can?

Docker-Ci-Zap can!

Docker-Ci-Zap is an open source project, written in a programming language called go. Read more about go here! It uses the Host Compute Service Shim, another go project, used of the Docker Host foundation. It basically enumerates the layes and destroys them. Disclaimer: There is absolutely NO guarantee that this tool works and it might cause your computer to need a reinstall. But it worked for me. So, I have built a script around this tool. A script, which performs the other 3 steps and as step 2, automatically downloads Docker-Ci-Zap and calls it with your Docker Data Dir. The script will detect that Docker is installed and determine that you are running Windows Containers and detect your Docker Root Folder. If you successfully run the script, you should see something like: And both your Docker folder and your NavContainerHelper folder should now be empty. If you encounter an error like: It is likely because you need a restart of your computer and a re-run of the script.

NOTE: This script will delete all Containers and Images

The Script can be copied from below or downloaded from this URL: https://raw.githubusercontent.com/Microsoft/nav-arm-templates/master/CleanupAfterDocker.ps1

$ErrorActionPreference = "Stop"
$WarningActionPreference = "Stop"

# Specify which images to download
$ImagesToDownload = @("microsoft/dynamics-nav:generic"
                      "mcr.microsoft.com/businesscentral/onprem",
                      "microsoft/dynamics-nav:2018")

$navContainerHelperFolder = "C:\ProgramData\NavContainerHelper"

$currentPrincipal = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
if (!($currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator))) {
    throw "This script must run with administrator privileges"
}

Write-Host "Checking Docker Service Settings..."
$dockerService = (Get-Service docker -ErrorAction Ignore)
if (!($dockerService)) {
    throw "Docker Service not found / Docker is not installed"
}

if ($dockerService.Status -ne "Running") {
    throw "Docker Service is $($dockerService.Status) (Needs to be running)"
}

$dockerInfo = (docker info)
$dockerOsMode = ($dockerInfo | Where-Object { $_.StartsWith('OSType: ') }).SubString(8)
if ($dockerOsMode -ne "Windows") {
    throw "Docker is not running Windows Containers"
}

$dockerRootDir = ($dockerInfo | Where-Object { $_.StartsWith('Docker Root Dir: ') }).SubString(17)
if (!(Test-Path $dockerRootDir -PathType Container)) {
    throw "Folder $dockerRootDir does not exist"
}

Write-Host -Foregroundcolor Red "This function will remove all containers, remove all images and clear the folder $dockerRootDir"
Write-Host -Foregroundcolor Red "The function will also clear the contents of $navContainerHelperFolder."
Write-Host -Foregroundcolor Red "Are you absolutely sure you want to do this? (This cannot be undone)"
Write-Host -ForegroundColor Red "Type Yes to continue:" -NoNewline
if ((Read-Host) -ne "Yes") {
    throw "Mission aborted"
}

Write-Host "Running Docker System Prune"
docker system prune -f

Write-Host "Removing all containers (forced)"
docker ps -a -q | % { docker rm $_ -f 2> NULL }

Write-Host "Stopping Docker Service"
stop-service docker

Write-Host "Downloading Docker-Ci-Zap"
$dockerCiZapExe = Join-Path $Env:TEMP "docker-ci-zap.exe"
Remove-Item $dockerCiZapExe -Force -ErrorAction Ignore
(New-Object System.Net.WebClient).DownloadFile("https://github.com/jhowardmsft/docker-ci-zap/raw/master/docker-ci-zap.exe", $dockerCiZapExe)
Unblock-File -Path $dockerCiZapExe

Write-Host "Running Docker-Ci-Zap on $dockerRootDir"
Write-Host -ForegroundColor Yellow "Note: If this fails, please restart your computer and run this script again"
& $dockerCiZapExe -folder $dockerRootDir

Write-Host "Removing Docker-Ci-Zap"
Remove-Item $dockerCiZapExe

Write-Host "Starting Docker Service"
Start-Service docker

if (Test-Path $navContainerHelperFolder -PathType Container) {
    Write-Host "Cleaning up $navContainerHelperFolder"
    Get-ChildItem $navContainerHelperFolder -Force | ForEach-Object { 
        Remove-Item $_.FullName -Recurse -force
    }
}

if ($ImagesToDownload) {
    Write-Host -ForegroundColor Green "Done cleaning up, pulling images for $os"

    $os = "ltsc2016"
    if ((Get-CimInstance win32_operatingsystem).BuildNumber -ge 17763) { $os = "ltsc2019" }

    Write-Host -ForegroundColor Green "Done cleaning up, pulling images for $os"
    # Download images needed
    $imagesToDownload.Split(',') | ForEach-Object {
        if ($_.EndsWith('-ltsc2016') -or $_.EndsWith('-1709') -or $_.EndsWith('-1803') -or $_.EndsWith('-ltsc2019') -or
            $_.EndsWith(':ltsc2016') -or $_.EndsWith(':1709') -or $_.EndsWith(':1803') -or $_.EndsWith(':ltsc2019')) {
            $imageName = $_
        } elseif ($_.Contains(':')) {
            $imageName = "$($_)-$os"
        } else {
            $imageName = "$($_):$os"
        }
        Write-Host "Pulling $imageName"
        docker pull $imageName
    }
}

I hope you find the explanation and the script helpful. If the tool and the script doesn't work for you, there isn't much I can do about it. Enjoy Freddy Kristiansen Technical Evangelist

Business Central Build Numbers

FreddyDK — Wed, 05 Dec 2018 12:00:07 GMT

Now and then, I get questions about the build numbers - what do they all mean? I always try to tell people that they don't really need to know but for some reason, that just makes people more curious. So instead of answering the same question over and over again - here is what I know... As you might know, I wrote this blog post a while back, trying to describe that you shouldn't really care about the build numbers, you should always go for the latest unless instructed otherwise. But let me try to shed some light over the version numbering of these Docker Images. In order to do so, I will have to talk about public and insider builds separately, but before I do that, let me shortly describe what they are doing in Windows (as our strategy isn't that different).

Windows Build numbers

Windows 10 and Windows Server uses the same build numbers, i.e. 10.0... The major and the minor version number doesn't change and the build numbers are public (https://www.microsoft.com/en-us/itpro/windows-10/release-information), the revision not so much. Build 17763 is Windows 10 Fall Update 2018, also called 1809 and build 17763 is also Windows Server 2019. Build 17134 is Windows 10 Spring Update 2018, also called 1803. etc. With Business Central we also use the major and the minor version numbers.

Business Central Build numbers

In October 2018 we shipped Business Central Fall 2018 release.

Fall 2018 release, update 19, 13.0, rtm

There are a lot of names for the Business Central Fall 2018 release, lets explain them:

Fall 2018 release - obviously because the release is a major release from fall of 2018.
Internally (and sometimes externally) this is called update 19 because it is month 19 since we started our online services and this the 19th update to our SaaS offerings.
Dynamics NAV 2018 was 11.0, Business Central Spring 2018 was 12.0, Business Central Fall 2018 is 13.0
rtm is only used for the on-premises release of Business Central as this was the first release of that

Business Central Fall 2018 release is major release 13, minor release 0 and build number 24630 was the golden build. Putting these numbers together gives 13.0.24630.0 which indeed is the version number for Business Central Fall 2018. If you want to pull the on-premises Docker image for this release you can use:

docker pull mcr.microsoft.com/businesscentral/onprem:13.0.24630.0[-country][-os]

where

country is one of the countries shipped for on-premises: w1, at, au, be, ch, cz, de, dk, es, fi, fr, gb, is, it, na, nl, no, nz, ru, se. Default is w1.
os is either ltsc2016 or ltsc2019. Default is ltsc2016.

The on-premises release is also tagged with rtm instead of 13.0.24630.0. If you want to pull the sandbox Docker image for this release you can use:

docker pull mcr.microsoft.com/businesscentral/sandbox:13.0.24630.0[-country][-os]

where

country is one of the countries shipped online: w1, at, au, be, ca, ch, de, dk, ee, es, fi, fr, gb, hk, hr, is, it, jp, kr, lt, lv, mx, nl, no, nz, pl, pt, rs, se, si, tw, us. Default is w1.
os is either ltsc2016 or ltsc2019. Default is ltsc2016.

For sandbox builds you will also find a country called base. Currently, this is the on-premises w1 database running in sandbox mode. This image is build as foundation for all the country versions. The sandbox release is not tagged with any extra release tags

Update 20, 13.1, cu1

One month after Business Central Fall 2018 release, in November, we shipped the first minor update. Internally, this is known as update 20. As it is a minor release, the minor part of the version number is updated to 1 and for on-premises we also call this cu1. The build number for update 20 golden build was 25940 giving us this image for the onprem Docker image:

docker pull mcr.microsoft.com/businesscentral/onprem:13.1.25940.0[-country][-os]

and this image for the sandbox Docker image:

docker pull mcr.microsoft.com/businesscentral/sandbox:13.1.25940.0[-country][-os]

country and os have the same meaning as with the fall release, but we are likely to add new countries at minor updates as well.

Update 21, 13.2, cu2

In December 2018, we will ship update 21. Another minor release, as cu2 on-premises. I do not have the build numbers for this yet, but you probably get the picture by now.

What about revisions?

For Business Central online, we also ship revisions or hotfixes. These hotfixes will keep the same Major and Minor version number and also the same build number. It will however add a revision number. The revision number is not a counter starting from zero - it is an internal build number added as revision number to indicate that it is a newer revision. We do NOT build Docker images of all these revisions as we also don't roll out all revisions to our Business Central online servers. For Fall release, we did create Docker images of 3 revisions: 24844, 25242 and 25789. You probably already guessed that this means that the latest sandbox Docker image you can get from update 19/13.0 is:

docker pull mcr.microsoft.com/businesscentral/sandbox:13.0.24630.25789[-country][-os]

For update 20, we have created Docker images of 2 revisions: 26108 and 26323, meaning that currently the latest sandbox Docker image available is:

docker pull mcr.microsoft.com/businesscentral/sandbox:13.1.25940.26323[-country][-os]

which is the same version you get by using:

docker pull mcr.microsoft.com/businesscentral/sandbox:[country][-][os]

These images are ONLY available for sandbox - we will not release revisions of the on-premises Docker images.

Build numbers of insider builds

If you are in the ReadyToGo program, you have access to Microsoft Collaborate and should have access to our insider builds. Note: When you signed up for ReadyToGo, you signed a Non Disclosure Agreement (NDA) preventing you from telling about what you see and find in the insider builds. We (Microsoft) want to be the one disclosing our new features, but we would like you to know about it early and be ready with your offerings before we ship. You should be working on finding ways to leverage the innovations in the product instead of blogging about them. Insider builds come in two flavours: bcsandbox and bcsandbox-master

bcsandbox contains functionality, which will ship in our next minor release. (While writing this blog post, this is January)
bcsandbox-master contains functionality, which will ship in our next major release. (While writing this blog post, this is Spring 2019)

Now you might wonder why the functionality from bcsandbox won't be in the December release (13.2 / cu2)? The reason is simple, we branched out for the 13.2 / cu2 release a few days ago and are running final tests and quality assurance before shipping - this does take some time. Even more confusing is it, that the version number of the current bcsandbox insider build (right now) is 13.2.26500.0. The reason is again simple, we haven't updated the version number yet in the branch. And to complete the confusion. The version number of the current bcsandbox-master insider build (right now) is 14.0.26496.0, indicating that bcsandbox-master is older than bcsandbox. The reason is as usual simple, the build number is a number series shared between all branches (Dynamics NAV 2017 and up) and it just means that the daily bcsandbox-master build was build before the daily bcsandbox build. You might have noticed that the Major and Minor version number has been updated to 14.0 though. Revision is always 0 as we are not shipping the insider builds and the build number will just be updated. If you are working on an app for AppSource, you should consider setting up daily builds checking that your app doesn't break on the next minor update (bcsandbox). You should also consider setting up weekly builds to check that your app doesn't break on the next major update (bcsandbox-master). This is one of the next blog posts in the CI/CD series. Remember that you will always get the latest bcsandbox image by using:

docker pull bcinsider.azurecr.io/bcsandbox:[country][-os]

and the latest bcsandbox-master by using:

docker pull bcinsider.azurecr.io/bcsandbox-master:[country][-os]

For the list of countries, available you should check Microsoft Collaborate (until we ship)

More version numbers

If you run docker inspect on a Docker image, you will under the Labels section under Config find not one, not two, not three, but four version numbers. Before getting that question, let me also describe the meaning of these

     "osversion": "10.0.17763.134"
     "tag": "0.0.8.0"
     "version": "14.0.26445.0"
     "platform": "13.0.26325.0"

osversion is the version of the Operating System. Which Windows Server Core version was used to build this image. Read more here. tag is the version of the Generic Image used to build this mage. You will find the source for the generic image here together with information about what's included in the version. version is the version of Business Central. This version can be used as the tag to pull this image at a later time. This is the version number this blog post is all about. platform is the version number of the platform (version of the Business Central binaries). These are following a different version numbering and the same platform version might be used for several releases of Business Central (especially on insider builds). We are likely to specify on our monthly developer preview blog posts which platform version is needed for the features included and we need to know the platform version when you ask questions on AL issues. Note also that when an issue has been fixed, the AL Bot will add a reply to the issue with a text like this: “ The fix for this issue has been checked in to the master branch. It will be available in the bcinsider.azurecr.io/bcsandbox-master Docker image starting from platform build number xxxxx.” This means that you can check whether the fix is in your version of docker image. Unfortunately we cannot give any build version or date when this the fix will be in bcsandbox-master.

Now you know... - but do you really need the info?

If you are developing a per tenant customization for an online tenant, you should always be using the latest public sandbox version - as this is what is being rolled out on our online servers. If your extension doesn't run on the latest sandbox, it will not run on an online tenant. If you are developing an app for AppSource, you should be developing either on the latest public sandbox version or using insider builds. If you are using insider builds you should follow the insider builds (at least monthly). If you are lifting a Dynamics NAV vertical (https://docs.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/readiness/readiness-embed-apps) you might need to use a specific build, but you should be in contact with Microsoft on which build to use. If you are developing for on-premises, they should use rtm or cu1 tags - not the version numbers and you should use the onprem container, NOT the sandbox containers. Here you also have the option of downloading the DVD.

Enjoy

Freddy Kristiansen Technical Evangelist

NAV and Business Central Docker images now available for Windows Server 2019

FreddyDK — Sat, 01 Dec 2018 01:21:57 GMT

As of today, all NAV and Business Central Docker images are available for Windows Server 2019 as well as for Windows Server 2016! It doesn't sound like a big deal, but believe me, it is...:-)

What does that mean?

As you probably know, Microsoft ships major updates to Windows every 6 months and every 2-3 years a version will be marked as LTSC (Long Term Servicing Channel). For Windows Server, only the LTSC builds will include a UI, which allows you to connect to the Desktop of the Server. All updates between the LTSC releases will only ship as new versions of Windows Server Core, giving you only a Command prompt and/or PowerShell prompt. You can read more about the Windows Server Semi Annual Channel here and what that means to Windows 10 here. Up until today, specific images containing NAV or Business Central have only been available based on Windows Server Core 2016. Generic images (images, which can be used to run NAV or Business Central if you have an install media or files extracted from another image) are available for most Semi annual builds and you can select to use these by using -useBestContainerOS on New-NavContainer. This blog post explains how to do this. As of today, specific images containing NAV or Business Central are also available based on Windows Server Core 2019.

Image Tags

Some sample image names for NAV and Business Central:

microsoft/dynamics-nav:2018-cu4-dk
mcr.microsoft.com/businesscentral/onprem:cu1-de
mcr.microsoft.com/businesscentral/sandbox:13.1.25940.26323

If you use docker pull and docker run on these images, you will get a ltsc2016 (Windows Server Core 2016) version of the images. That is the same behavior as before today. If you append -ltsc2016, you will get exactly the same images:

microsoft/dynamics-nav:2018-cu4-dk-ltsc2016
mcr.microsoft.com/businesscentral/onprem:cu1-de-ltsc2016
mcr.microsoft.com/businesscentral/sandbox:13.1.25940.26323-ltsc2016

If you append -ltsc2019, you will get the same versions of NAV/Business Central, but based on ltsc2019 (Windows Server Core 2019)

microsoft/dynamics-nav:2018-cu4-dk-ltsc2019
mcr.microsoft.com/businesscentral/onprem:cu1-de-ltsc2019
mcr.microsoft.com/businesscentral/sandbox:13.1.25940.26323-ltsc2019

Using docker pull and docker run, you need to determine yourself which version of the image to pull and run.

Compatibility

One reason for using ltsc2019 images is compatibility. It is my experience, that the further away we came from Windows Server 2016, the more issues we would see when people were running Windows 10 and using ltsc2016 images. Almost all issues that I have been unable to resolve for people have been due to Windows 10 1803 running Docker and ltsc2016 images.

Image Size

Another reason for using ltsc2019 images is image size. After having pulled a number of images, we can inspect the sizes they occupy:

On WindowsServerCore, the ltsc2019 version is 7.02GB smaller.
The Generic layer on ltsc2019 is another 2.39GB smaller than the ltsc2016 equivalent, making it 9.41GB smaller in total.
Even the W1 layer is smaller. 300MB is saved on ltsc2019, making the image 9.71GB smaller in total.
The country layer is only a fraction smaller. Still the combined savings of 9.73GB is massive.

Note that these sizes are after extraction of the compressed images. The download sizes are way smaller, but the difference between the download size of ltsc2016 images and ltsc2019 images are even bigger. As an example, the download size of the ltsc2016 version of NAV 2018 CU5 DK is 8,159,631,982 bytes and the ltsc2019 equivalent is 3,208,188,273 bytes. The ltsc2016 image is more than 2½ times the size of the ltsc2019 image.

NavContainerHelper 0.4.2.2

A new version of NavContainerHelper will try to help you get the right image for your host. If you specify that you want to use the Danish CU1 version of Business Central on prem, you would specify mcr.microsoft.com/businesscentral/onprem:cu1-dk as the imagename. With NavContainerHelper, you should still just specify the imagename like this and NavContainerHelper will then help you determine the best container OS. The strategy for NavContainerHelper to determine which imagename to use is as follows:

If the imagename doesn't specify platform (ltscxxxx), then calculate the best imagename (append -ltsc2016 for Windows Server 2016 or -ltsc2019 for Windows Server 2019)
If you have not specified -alwaysPull, use the best imagename or the imagename in that order if they exist on the host
If the images didn't exist on the host or you have specified -alwaysPull, try to pull the best imagename and if that fails, revert back to the imagename specified

You can also specify -useBestContainerOS if you want to allow NavContainerHelper to use a generic image if that is better, in fact I recommend that you always do that. This allows NavContainerHelper to spin up a better generic image matching your OS in the future as well. -useBestContainerOS won't do anything if you are already using the best container os for your host.

ARM templates

When using http://aka.ms/getbc, http://aka.ms/getnav or any of the other ARM templates to spin up Azure VMs with Business Central or NAV, you will now have an option to select your operating system: Since the ARM templates uses the NavContainerHelper to create the container, you do not have to specify platform version after the image name. Just specify mcr.microsoft.com/businesscentral/onprem:cu1-dk and you will get the version matching the operating system you selected. The total time it takes to spin up the Azure VM, initialize the VM, pull the image and run the image varies depending on a number of things, but I did a test of spinning up the same image on these 3 host operating systems and the total time for initialization was:

Windows Server 2016 = 42 minutes
Windows Server 2019 = 22 minutes
Windows Server 2019 with Containers = 20 minutes

The primary difference between 2 and 3 is, that #3 has docker pre-installed and the WindowsServerCore 1809 pre-downloaded. This normally works just fine, but I have kept option #2 if for some reason #3 isn't running the latest Docker or isn't updated. #1 is kept as an option until we decide to stop building ltsc2016 images.

Issues?

If you encounter any issues with the images, with NavContainerHelper or with the ARM templates, please file the issues on GitHub:

It is much better than emailing me. An email might disappear in the inbox, a GitHub issue will also notify me with an email and it will be there for other people to see, comment on and learn from.

What next

First of all - it's late Friday evening - or rather early Saturday morning. I will enjoy my weekend and then next week I will continue the blog post series on Continuous Integration and Continuous Delivery - part #6 coming up... Enjoy Freddy Kristiansen Technical Evangelist

Developing Business Central Extensions (part 5) - YAML

FreddyDK — Sat, 17 Nov 2018 09:17:36 GMT

This is part 5 of a blog post series. Part 1 contains all the prerequisites, part 2 is about cloning the project and get your sandbox environment up running, part 3 is about build agents, and building your project in Azure DevOps and part 4 is about branch policies and working with VSCode and Git.

YAML, or not YAML, that's the question!

In part 3 of this blog post series, we created a build pipeline based on a file called CI.yml. This file is using YAML to describe the build pipeline and is actually very easy to read if you open it in VS Code. I actually did not start out using YAML, I started out using the visual designer to create and work on my build pipeline and for some time, I was arguing with myself which approach would be the best. The Visual Designer has some obvious advantages for newbies like me, but YAML seemed to be the right way when I want to have the build pipeline saved as part of the repo (as code) and be able to transfer this to other repositories. There are other reasons why one would need to use the Visual Designer, so it is really up to you whether you want to go with YAML or the Visual Designer. I have included a few reasons for and reasons against using YAML, then you can decide for yourself. In this blog post I will also describe how you within 15 minutes can convert the CI YAML pipeline to a Visual Designer Pipeline - it is really just a lot of Copy/Paste work.

Reasons for using YAML

You like code and markup editing over visual designer editing (wysiwyg)
Pipeline is checked in as a file in code and follows branching and forking
Changes to code and changes to pipeline can be checked in together as part of a pull request
I have not found a good way to reuse a build pipeline created using the visual designer across projects
- The export format from the Visual Designer (.json) is very much tied to the repository and the branch and doesn't seem suited to be used as a template
- Saving the pipeline as a template also doesn't make the template available in other projects

Reasons against using YAML

You like visual designer editing (wysiwyg) over code and hate markup editing
If your source code isn't stored in Azure DevOps (could be GitHub, Subversion, Bitbucket or external Git), then you cannot use Yaml
If a change to the code requires a change in the pipeline, you will have to hand-carry the change into the repo
Steep learning curve creating your first YAML pipeline (if you don't have the sample from the template repo)

Whether you use the Visual Designer or YAML, you can save the pipeline as a template and reuse it, but it is still not reusable in other repositories.

YAML isn't YAML

As I wrote earlier, I started out using the Visual Designer and switched to YAML. I thought conversion would be as easy as clicking View YAML on my Visual Designer pipeline but when things sounds to good to be true,. they usually are. There seem to be a bug (or maybe it is only when using PowerShell tasks). The YAML I got out of my Visual pipeline couldn't be imported later. Instead of waiting for somebody to fix the bug, I did a manual conversion and now my master is in YAML. A full description of the YAML format is available here: http://aka.ms/yaml very very helpful when converting and coding in YAML.

CI.yml

While writing this blog post, the CI.yml file looks like this:

resources:
- repo: self

steps:
- task: PowerShell@2
  displayName: 'Defaulting container version'
  inputs:
    errorActionPreference: silentlyContinue
    targetType: inline
    script: 'if ("$(version)" -eq "")  { Write-Host "Version not defined, using current container version"; Write-Host "##vso[task.setvariable variable=version]current" } else { Write-Host "Using $(version) container version" }'

- task: PowerShell@2
  displayName: 'Login to bcinsider repository'
  condition: and(succeeded(),or(eq(variables['version'],'nextminor'),eq(variables['version'],'nextmajor')))
  inputs:
    targetType: inline
    script: 'docker login "bcinsider.azurecr.io" -u "$(bcInsiderUsername)" -p "$(bcinsiderPassword)"'

- task: PowerShell@2
  displayName: 'Install NavContainerHelper'
  inputs:
    targetType: filePath
    filePath: 'scripts\Install-NavContainerHelper.ps1'

- task: PowerShell@2
  displayName: 'Create Build Container'
  inputs:
    targetType: filePath
    filePath: 'scripts\Create-Container.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -licenseFile "$(LicenseFile)"'

- task: PowerShell@2
  displayName: 'Compile App'
  inputs:
    targetType: filePath
    filePath: 'scripts\Compile-App.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -buildProjectFolder "$(Build.Repository.LocalPath)" -buildArtifactFolder "$(Build.ArtifactStagingDirectory)" -appFolders @("app")'
    failOnStderr: true

- task: PowerShell@2
  displayName: 'Sign App'
  condition: and(succeeded(),ne(variables['CodeSignPfxFile'],''),ne(variables['CodeSignPfxPassword'],''))
  inputs:
    targetType: filePath
    filePath: 'scripts\Sign-App.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -buildArtifactFolder "$(Build.ArtifactStagingDirectory)" -appFolders @("app") -PfxFile (ConvertTo-SecureString -String "$(CodeSignPfxFile)" -AsPlainText -Force) -PfxPassword (ConvertTo-SecureString -String "$(CodeSignPfxPassword)" -AsPlainText -Force)'

- task: PowerShell@2
  displayName: 'Publish App'
  inputs:
    targetType: filePath
    filePath: 'scripts\Publish-App.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -buildArtifactFolder "$(Build.ArtifactStagingDirectory)" -appFolders @("app") -skipVerification:("$(CodeSignPfxFile)$(CodeSignPfxPassword)" -eq "")'

- task: PowerShell@2
  displayName: 'Import TestToolkit'
  inputs:
    targetType: filePath
    filePath: 'scripts\Import-TestToolkit.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force)))'

- task: PowerShell@2
  displayName: 'Compile Test App'
  inputs:
    targetType: filePath
    filePath: 'scripts\Compile-App.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -buildProjectFolder "$(Build.Repository.LocalPath)" -buildArtifactFolder "$(Build.ArtifactStagingDirectory)" -appFolders @("test")'
    failOnStderr: true

- task: PowerShell@2
  displayName: 'Publish Test App'
  inputs:
    targetType: filePath
    filePath: 'scripts\Publish-App.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -buildArtifactFolder "$(Build.ArtifactStagingDirectory)" -appFolders @("test") -skipVerification'

- task: PowerShell@2
  displayName: 'Run Tests'
  inputs:
    targetType: filePath
    filePath: 'scripts\Run-Tests.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -TestResultsFile (Join-Path "$(System.DefaultWorkingDirectory)" "TestResults.xml") -test "unittests"'

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: XUnit
    testResultsFiles: TestResults.xml

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: app'
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)\app'
    ArtifactName: app

- task: PowerShell@2
  displayName: 'Remove Build Container'
  condition: always()
  inputs:
    targetType: filePath
    filePath: 'scripts\Remove-Container.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force)))'

Very straightforward. As you can see, the YAML file consists of a series of tasks. Every task has a type, a version, a display name, conditions and a number of properties. If the Task is a PowerShell task, then it can be an inline script, meaning that it contains the PowerShell code directly or it can be a filePath, where it references a PowerShell script in the repository. If you are using filePath, you can specify a number of parameters to transfer to the script. The script will have to have a Param section, matching the parameters from the build pipeline. Example, the Param section of the Create-Container script looks like:

Param(
    [ValidateSet('AzureDevOps','Local')]
    [string] $run = "AzureDevOps",
    [ValidateSet('current','nextminor','nextmajor')]
    [string] $version = "current",
    [ValidateSet('bld','dev')]
    [string] $type = "bld",
    [Parameter(Mandatory=$true)]
    [pscredential] $credential,
    [String] $licenseFile = ""
)

Matching the parameters transferred in CI.yml:

-version "$(version)"
-credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) 
-licenseFile "$(LicenseFile)"'

It is possible to run things in parallel, but in our case, the pipeline is really a single threaded thing, which cannot be parallelized.

Multiple Apps

If your project consists of multiple apps, which have dependencies, the idea is to catch these dependencies in the build pipeline. We might at a later time add functionality to automatically detect and build dependencies but for now, you will have to capture these in the pipeline. You can transfer several app folders to the Compile App tasks (if the apps are not inter-dependent) or you can have multiple Compile App steps if the apps depend on other apps, which then have to be published (much like the App and the Test App). No, it is not ideal, but it is possible.

"Converting" CI.yml to a pipeline in the Visual Designer

It isn't really converting, it is creating a new pipeline using the visual designer and copying the information from the YAML file. I have decided NOT to include a CI.json (pipeline export format) in the template repository, instead you will have to create a new empty pipeline using the visual designer, link the Azure Key Vault variable group and add a number of tasks to the pipeline.

The first step

Let's have a look at the very first step in the pipeline, which task is to set the version pipeline variable to current, if it hasn't been set already:

- task: PowerShell@2
  displayName: 'Defaulting container version'
  inputs:
    errorActionPreference: silentlyContinue
    targetType: inline
    script: 'if ("$(version)" -eq "")  { Write-Host "Version not defined, using current container version"; Write-Host "##vso[task.setvariable variable=version]current" } else { Write-Host "Using $(version) container version" }'

In the Visual Designer you add a Task to the Agent Job and search for PowerShell (not Azure PowerShell, not Service Fabric PowerShell, not PowerShell on Target Machines - just PowerShell). Add this task and fill out the properties:

The number after PowerShell in the - task line determines the version of PowerShell (here 2)
The displayName goes into Display Name
errorActionPreference (defaults to stop) goes into ErrorActionPreference
If targettype is inline, type inline is selected and the script goes into the scripts editor.

The above step in the Visual Designer will look something like this:

Conditions

The second step in the pipeline is logging into the bcinsider repo, if you are building against nextminor or nextmajor. It looks like

- task: PowerShell@2
  displayName: 'Login to bcinsider repository'
  condition: and(succeeded(),or(eq(variables['version'],'nextminor'),eq(variables['version'],'nextmajor')))
  inputs:
    targetType: inline
    script: 'docker login "bcinsider.azurecr.io" -u "$(bcInsiderUsername)" -p "$(bcinsiderPassword)"'

This task is also a PowerShell task and the only difference to the first step is that this step only executes if the conditions are met. The conditions here are:

If all previous steps succeeded and version is either nextminor or nextmajor.

Then we want to login to the bcinsider registry using the username and password from the key vault. In the Visual Designer, you find the conditions under Control Options:

Script Path

The third step in the pipeline will install the NavContainerHelper module if not already installed. It looks like

- task: PowerShell@2
  displayName: 'Install NavContainerHelper'
  inputs:
    targetType: filePath
    filePath: 'scripts\Install-NavContainerHelper.ps1'

Again, a PowerShell task, but in this case, it is not an inline script, instead the targetType is a filePath and the FilePath refers to a script in the script folder. The conditions are default, meaning that all previous tasks have to succeed before this task is run.

Arguments

The forth step in the pipeline is creating the build container. This also uses a filePath but this one comes with arguments:

- task: PowerShell@2
  displayName: 'Create Build Container'
  inputs:
    targetType: filePath
    filePath: 'scripts\Create-Container.ps1'
    arguments: '-version "$(version)" -credential ([PSCredential]::new("$(Username)", (ConvertTo-SecureString -String "$(Password)" -AsPlainText -Force))) -licenseFile "$(LicenseFile)"'

The arguments are using the exact same syntax as the aguments in the Visual Designer, just remove the '.

Other properties

For the remaining tasks, a few other properties are used, but these maps easily into the Visual Designer as well (like FailOnStderr -> Fail on Standard Error under Advanced). Note that the Remove Build Container task is using always() in conditions, meaning that it will run even if a previous task has failed, even if the build was cancelled.

Publishing Test Results

After running the tests, the tests results needs to be published. As the run tests task will store the test results in XUnit format in the file TestResults.xml, the task for publishing the tests looks like:

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: XUnit
    testResultsFiles: TestResults.xml

In the Visual Designer, it is really just as simple

Publishing Build Artifact

If everything succeeded, we need to publish the build artifact (our app) where it can be accessed after the build. For that we use the Publish Build Artifact task

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: app'
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)\app'
    ArtifactName: app

Like when publishing tests results, the Visual Designer task is just as easy

What's next

This blog post really didn't get us any further in the process of setting up CI/CD, but it does explain why I selected YAML as the format of pipelines and when we get to create integration builds with minor or major builds of Business Central a little later, these things should be clear. In the next post, I will cover how to automatically deploy the daily build of your extension to an Azure VM, where you can demo or test the latest version at all times. Enjoy Freddy Kristiansen Technical Evangelist

Windows Server 2019 and SQL Server 2017

FreddyDK — Thu, 15 Nov 2018 11:27:08 GMT

Windows Server 2019 (1809) finally shipped (again) and with that we have a new LTSC (Long Term Servicing Channel). This means that we will begin creating images for Windows Server 2019 as soon as our infrastructure is ready for that.

-UseBestContainerOS

While waiting for 1809 images, you can use -UseBestContainerOS when running images with NavContainerHelper and it will automatically detect the best generic image to use for your host OS, whether that is Windows Server 2016, Windows Server 2019, Windows 10 1709, Windows 10 1803 or Windows 10 1809. It will even detect that you are running an insider version of Windows Server and fall back to Windows Server 2019 generic image.

SQL Server 2017

With the release of Windows Server 2019, I tried to just rebuild the generic image based on the new Windows Server Core 1809 image. SQL Server 2016 has always been used in the images, but it did cause problems with 1809, so I decided to switch to SQL Server 2017, which installs and works. A few issues though encountered during testing:

DAC missing

SQL Server 2016 came with a version of Dac Framework pre-installed. SQL Server 2017 doesn't. Dac was used for importing .bacpac files in the container. I have changed this code to do a Just-In-Time download and install of Dac Framework 18.0 when needed.

Export-NavApplication

When switching to multitenant setup, the container uses Export-NavApplication. This worked out of the box for SQL Server 2016, but for some reason SQL Server 2017 failed. After a lot of investigations, I found that setting -ServiceAccount on Export-NavApplication to NT AUTHORITY\SYSTEM and adding the user to the database solved the issue. This also required the Export-NavContainerDatabasesAsBacpac in NavContainerHelper to remove this user on export.

Cmdlets expecting "old" version of DLLs

The Cmdlets for Service Tier administration in old NAV and Business Central versions are expecting earlier versions of SQL Server DLL's:

Microsoft.SqlServer.Smo
Microsoft.SqlServer.SmoExtended
Microsoft.SqlServer.ConnectionInfo
Microsoft.SqlServer.SqlEnum

By placing dependentAssembly sections the AssemblyBinding section in PowerShell.exe.config in the container, specifying that any attempt to load old versions of these DLLs, should take the new version:

This should make all CmdLets work with the new SQL Server.

Export-NavContainerDatabasesAsBacpac

This function is placed in the NavContainerHelper, but it also used to install a newer version of the Dac Framework (than the one included in SQL Server 2016). This function has been changed to also use the new version (18.0) and it will just re-use the version installed in the container if possible. This fix is included in NavContainerHelper 0.4.1.0.

So, what's there now

You will need NavContainerHelper 0.4.1.0 for supporting Windows Server 2019 and SQL Server 2017. While writing this blog post, the generic images have been updated:

microsoft/dynamics-nav:generic-ltsc2016 is the generic image for Windows 10 1607, Windows ServerCore 1607 or Windows Server 2016. Based on Windows ServerCore ltsc2016 and the foundation of all ltsc2016 NAV and Business Central containers
microsoft/dynamics-nav:generic-1709 is the generic image for Windows 10 1709 release or Windows ServerCore 1709. Based on Windows ServerCore 1709.
microsoft/dynamics-nav:generic-1803 is the generic image for Windows 10 1803 release or Windows ServerCore 1803. Based on Windows ServerCore 1803.
microsoft/dynamics-nav:generic-ltsc2019 is the generic image for Windows 10 1809, Windows ServerCore 1809 or Windows Server 2019. Based on Windows ServerCore 1809.

microsoft/dynamics-nav:generic still points to microsoft/dynamics-nav:generic-ltsc2016. The version number of the new Generic image is 0.0.8.0.

What will be there tomorrow

All future images (NAV and Business Central) will be build on the new ltsc2016 image with SQL Server 2017.

What will be there in a week from now

All NAV and Business Central images have been rebuilt to use the new ltsc2016 image with SQL Server 2017.

And finally, what will be there in a few weeks

All images are available as ltsc2019 as well as ltsc2016. In order to download the 1809 version you will have to append -ltsc2019 if using docker run. New-NavContainer will automatically detect that your host is Server 2019 and download the right image for that. Remember that you can use ltsc2019 already today by specifying -useBestContainerOS to New-NavContainer or you can extract the files from a ltsc2016 container using Extract-FilesFromNavContainerImage and use them to run ltsc2019 image. All dates are subject to change... Enjoy Freddy Kristiansen Technical Evangelist

Developing Business Central Extensions (part 4) - Branch Policies

FreddyDK — Tue, 13 Nov 2018 09:30:40 GMT

This is part 4 of a blog post series. Part 1 contains all the prerequisites, part 2 is about cloning the project and get your sandbox environment up running and part 3 is about build agents, and building your project in Azure DevOps.

Branch Policies

At this time, we have an Azure DevOps project with a build pipeline, which we can kick off manually. We have not setup any branch policies, meaning that anybody with write access to the repository can check in code, even code which will break the build and cause instability of your product. It's time to setup branch policies to ensure quality. Open your Azure DevOps organization (eg. https://dev.azure.com//) and select Repos -> Branches. Select the master branch, click the more actions symbol (...) and choose Branch Policies. You should now be able to setup branch policies for your branch The first thing you will notice is, that setting any required policy will enforce the use of pull requests and will disallow direct checkins to the master branch. It will also prevent the branch from unintended or evil deletion. You can read much more about the policies here: https://docs.microsoft.com/en-us/azure/devops/repos/git/branch-policies. What I want, is to ensure that my CI build pipeline runs whenever somebody checks something in and that the checkin cannot be completed if the build isn't successful. This is called Build validation and by adding a build policy with our CI pipeline from part 3, we should be ready to go.

Working with VS Code and Git

When working with a project under source code management in VS Code, you will notice some nice and very helpful features. I am using workspaces, meaning that I have added the app and the test folder to a workspace and I am opening that in VS Code, giving me a nice overview like: Lets try to change the HelloWorld.al with a very simple change (add an exclamation mark to Hello World and Save the file) You will see a number of things changing

The Source Control symbol shows that there is one modification
The HelloWorld.al file got a different color and is flagged with M (Modified)
In the bottom left corner you can see that you are working directly on the master branch and the * marks that you have unstaged changes

if you click the Source Control symbol to see your changes, you can select your changed files to see what changes you actually did A few other things you could do to files

If you create a new file, it will be flagged with U (untracked), which means that this file is unknown to the repository.
If you delete a file, it will be deleted and disappear from the file list, but show up in the Source Control list of changes with a D (deleted)

The files listed under changes in the Source Control area are unstaged changed, meaning that you have not yet determined whether you want to checkin this change or not. Clicking the + symbol next to the file will move the change to the staged changes. Clicking the + next to changes will move all changes to the staged changes. Staged changes are changes you intend to commit, but haven't yet. The * marker next to the master branch (bottom left corner) changes to a +, meaning that you have staged changes. Note, that if I add another exclamation mark to the Hello World string and save the file, I will have the HelloWorld.al file in both staged and unstaged changes list: This means that you are tracking changes - not files and now you have both a * and a + next to the master branch (bottom left corner). You can now unstage your change, which will remove the change from the staged changes area, but not change your file. You unstaged change will just be both exclamation mark on the master branch and you will not have any staged changes. You can also stage the second change, which will merge both changes and you will now have two exclamation marks in the staged changes area. You can discard changes from unstaged changes. If you want to discard changes from staged changes, you will need to unstage the changes and then discard them. So, staging or unstaging changes doesn't change any files, it only determines how Git looks at your changes. Having staged my changes and checked that my changes are indeed ready to be checked in, it is time to Commit. Write a message in the Commit Message bar and press Ctrl+Enter. After committing the changes you can close the comparison window and you will see But wait… - did I just commit changes to the master branch? I shouldn't be able to do that, right? What's happening??? The answer is simple. As you probably know, Git is a distributed source control system and every developer machine has a full copy of the source tree. What we did here was to commit to my local copy of the master branch, but I didn't push my changes to the Azure DevOps repository yet. Actually, you can see that in the bottom left corner there is an indicator signaling that you have one change, which needs to be pushed to Azure DevOps. If other developers have made changes to the project, there might also be a number next to the Down Arrow. So, in order for me to push my changes and pull changes from other people is to click the Synchronize button. When trying to synchronize, I get a strange message Clicking the Open Git Log reveals the reason for this: Pushes to this branch are not permitted; you must use a pull request to update this branch. So how do I create a pull request??? A pull request is a request to take changes from one branch and merge them into another branch. Problem is that I committed my changes to the local master branch. I need to uncommit those and put them in another branch. Fortunately, VS Code can help us here. In the Source Control section press the action menu (…) and select Undo Last Commit. Undoing the commit will leave my changes unstaged again and my local master branch is clean. Now, I can click the master branch symbol in the lower left corner to open the list of existing branches and select Create new branch. Enter the name of the new branch, press Enter and you should see In the lower left corner you will now see that you are in the Exclamation branch and the icon to the right (an arrow into the cloud) means that your branch is purely local and have no representation in Azure DevOps. Clicking the arrow into the cloud symbol will create the branch in the cloud and you will now be able to see it in your Azure DevOps Repos -> Branches page: We can now press the + symbol next to the file to stage the changes and commit the changes to our local Exclamation branch. Still the changes are purely local, but now, you can synchronize your changes to Azure DevOps. After synchronizing changes to Azure DevOps, VS Code will look like and viewing the branches again on Azure DevOps will show that our branch is one change ahead and we can now create a pull request. There is an extension for VS Code, which allows you to do Pull Requests directly from VS Code, but I haven't tried this. Read more here: https://code.visualstudio.com/blogs/2018/09/10/introducing-github-pullrequests Creating a pull request allows you to add code reviews, assign work item etc. After creating the pull request, a build is kicked off and you can monitor that build by clicking the build in progress link on the right side (under required policies) You can select to set Auto-complete, meaning that if all reviewers accepts the change and the build is successful, the pull request will be merged. In the autocomplete window, you can select to remove the feature branch and squash changes (meaning that all the checkins in the feature branch will be checked into master as one checkin) After completion we can see the status of the pull request and the build under Pull requests -> completed. and if we look under branches, the Exclamation branch has been deleted. In VS Code, we need to remove the orphaned Exclamation branch manually. Click the branch symbol in the lower left corner and checkout the master branch by selecting master. Press Ctrl+Shift+P and choose Git Delete Branch… and delete the orphaned Exclamation branch. You are now ready to create a new feature branch and work on the next set of changes.

What's next

Now we have setup branching policies and we have been through a very long and visual walkthrough of VS Code and Git, but if you are not used to Source Control and Git, it really shows how changes moves from untracked. staged, committed in your local repository to being pushed to the remote repository. Next post is about YAML (I guess this stands for Yet Another Markup Language) and how pipelines are defined. Enjoy Freddy Kristiansen Technical Evangelist

Developing Business Central Extensions (part 3) - Build Pipeline

FreddyDK — Mon, 12 Nov 2018 15:47:48 GMT

If you haven't read part 1 and part 2, you should do so before continuing here. Part 1 contains the prerequisites and part 2 is about cloning the project and get your sandbox environment up running.

Create a build agent

In order to setup continuous integration, we need a machine which can perform the actual build asks for us, the build agent. A build agent consists of a Windows Server 2016 with some software installed: Docker, NavContainerHelper and the Azure DevOps Agent. You will also need to configure the build agent to access your organization. I have created an ARM template, with which you can create a build agent. The Url to this template is http://aka.ms/getbuildagent or you can create a Windows Server 2016 and install the components manually:

Docker EE (https://docs.docker.com/install/windows/docker-ee/)
NuGet Package Provider 2.8.5.201 or higher (https://www.nuget.org)
NavContainerHelper (https://www.powershellgallery.com/packages/navcontainerhelper)
Azure DevOps Agent 2.141.1 installed and configured (https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=vsts)

The script, which performs these steps for the ARM template can also be downloaded and used. You will find it here: https://github.com/Microsoft/nav-arm-templates/blob/master/initbuildagent.ps1 In both cases you will need a Personal Access Token for your Azure DevOps Organization in order to setup a build agent. Open your Azure DevOps organization (eg. https://dev.azure.com//) and click your avatar and select Security In the Personal Access Tokens list, you can revoke tokens, regenerate them or edit scope, expiration date and more. Select New Token, give the token a name, an expiration date, click show all scopes to reveal Agent Pools and select Read & manage. Click Create and you will see the token one time and one time only. You will need to copy the token. Note: You can use the same token for multiple agents. For the automated build agent setup, open http://aka.ms/getbuildagent and specify size, location, your organization and the Personal Access Token. After 15-20 minutes, you should see your new agent popping up (https://dev.azure.com//_settings/agentpools), ready to serve you:

Access your secrets in the build system

In part 1, we created an Azure KeyVault, with a number of secrets (password, license file etc.). We need these secrets in the build and fortunately, Azure DevOps has a nice way of getting to those. In Azure DevOps, under your Project, select Pipelines -> Library. Click + Variable group to create a variable group. Name the variable group and select Link secrets from Azure key vault as variables. Specify the subscription (you will need to authenticate), select the key vault and add the variables These variables will now be available for the build tasks if you link the variable group to the pipeline.

Create a build pipeline

In Azure DevOps, under your Project, select Pipelines -> Builds. You will likely see that you have no build pipelines: Click New pipeline and click Use the visual designer. Select your repository: Continue and select the YAML template under Configuration as code. Select the Default Agent pool and select the CI.yml file (included in the repository) Select Variables, Variable Groups, Link variable group and Link your variable group you created above to the pipeline. Click Save & queue and queue a build. You can click the link of the build to see the log... Note: The first time you run a build on the agent, it will take significantly longer because it needs to pull the Business Central Docker image. After some time, you should see your build successfully completed and one failing test.

A few words about Tests

Update February 27th - the helloworld template repository has been updated with a new way of running tests - a special blog post on test execution will be added to this series soon. I should mention here, that the way tests are executed in the repository while writing this blog post is still work in progress. Currently the setup just invokes a Codeunit in a container and there are multiple reasons why this isn't sufficient. We are working on making a better approach to this and I will include this in a later blog post.

What's next

Now, we have our source in Azure DevOps and we have local and hosted builds running. In the next Blog Post we will setup checkin policies with code reviews and pull requests - and we will create a few more pipelines for testing our app against insider builds. Enjoy Freddy Kristiansen Technical Evangelist