Windows 2016 with Containers

Windows 2016 has a feature to add Windows and Hyper-V containers, both with their own advantages and limitations but I am not going to go over that in detail here. Below is a diagram that shows the architecture of each implementation and as we can see straight away that the Kernel is shared when using Windows Server Containers, hence it can only run Windows based instructions. This implementation doesn’t provide any security boundaries between containers as it exposes instructions of a container to the host and to all other VM’s, I wont go to comparision.

Windowscontainer

What I wanted to go over is a recent deployment that I have gone through which experienced some unusual behaviours.

Using Windows Server 2016 with Containers image from Azure gallery, provisioned a new Windows 2016 host, but that host on completion was missing Host Network Service (HNS network adaptor), this adaptor will be used for any communication that is external to the host, for example accessing the Internet.

The host was showing that Docker host network was bound to an adaptor that wasn’t showing in Network Connections and as you know that we don’t have access to see the status of that network adaptor on the host VM.

Using commands below I managed to clear Docker network settings and bring that network adaptor back online.

Using Powershell ran under admin credentials:

Stop-service hns

Stop-service docker

Get-ContainerNetwork | Remove-ContainerNetwork -force

Get-NetNat | Remove-NetNat

Start-Service docker

 

Your containers should have Internet/External access.

 

Leave a comment

Gain access to SQL

sql_locks

Have you had instances when your DBA left you with a DB that no one has SA access rights to?

For any instance of Microsoft SQL install, including SQL Express, there is a mode which would enable a local admin on that SQL server machine to gain/assign access to DB.

 

There are few steps which you have to follow:

1. Stop all SQL services

2. Start a command prompt with Admin permissions

3. Navigate to:
C:\Program Files\Microsoft SQL Server\MSSQL14.SQLEXPRESS\MSSQL\Binn

4. Run:
sqlservr.exe -m “Microsoft SQL Server Management Studio” -s SQLEXPRESS or any SQL instance name

5. Start SQL Server Management Studio as local admin and assign relevant groups/users access to DB

Leave a comment

StorSimple on site appliance factory reset

StorSimpleWith a recent client, we had to re-register the StorSimple physical appliance to a new instance of StorSimple device manager instance, the old instance was deleted and no longer available on Microsoft Azure (ASM).

If you attempt to run Invoke-HcsSetupWizard on the appliance, it would error with the appliance is already registered to another instance and the only way to register it to a new instance is to factory reset the device.

While resetting, the device hang up on phase 3 and wouldn’t go past that stage. After speaking to Microsoft Support, they have advised to shutdown both controllers and re-attempt to run the factory reset again which was successful in this case.

Just keep in mind that you will be losing all your device configuration and any data that you have accumulated on the device after the reset, ensure you update device after the reset.

Leave a comment

DevTest Labs and Auto-start

DevTestIconLargeWhile working with DevTest Labs on Microsoft Azure, it’s a good idea to make sure you have Auto-start and Auto-shutdown policies to keep cost down on resource utilisation while not being used.

The mistake or misconception I had form customers and I see this continuously that they forget to opt the VM in or out of the policy when scheduling these machines to shutdown or start at certain times.

This needs to be enforced on a per VM basis, see below.

optin

, , ,

Leave a comment

Application Gateway bug – Jan 2017

bug-512I have been working with Microsoft lately on an issue that I was experiencing with an Azure application gateway (appGW) deployment that require both internal and external interfaces handling traffic over HTTPS.

If you try to attach the same AppGW front end port to internal and external front end configuration, this would cause the appGW to misbehave. In my scenario I had a rule attached to external interface for handling incoming traffic but no rule attached on the internal interface and as a consequence the internal interface started processing traffic while the external interface was rejecting all connections (not even 502 error! would you believe!). Just to note that all my appGW deployments are scripted using PowerShell/JSON.

Microsoft managed to replicate this internally and issued a bug report, they are working on it as we speak but without ETA currently.

I had to drop my second listener (internal) in order to bring the appGW back to it’s expected behaviour!

, , , ,

2 Comments

The myth of Azure Application Gateway – Part 2

5db46fc5-d9c1-44ae-a0f6-4ae1d2301395In part 1 of this article I have gone through creating Azure Applications Gateways (AGW) using Powershell which is a powerful way of deploying resources on Azure, using recursive functions and methods you could build a complex solution in few lines. Unlike Powershell, JSON is a static solution. It gives you a way of creating a baseline deployment, I still haven’t found a way of controlling sub-resources (which isn’t governed by the JSON Copy() function).

In this article I will go over the same deployment using a JSON template, this only serve as a reference point for your deployment. I will use the same Azure AGW configuration I used in part 1 to keep both deployments consistent.

When building Azure AGW, same prerequisites apply to the JSON method. There is a subtle difference which I will go over when I get to that point.

So this time I have prepared a Visio diagram to simplify our deployment and understand what we are trying to achieve – see below.

appgateway-blog

The diagram shows a Web server hosting four websites running a combination of HTTP HTTPS with authentication (using Basic authentication) or Anonymous access.

The only thing different in comparison to previous Powershell deployment is the steps we take to process SSL certificates and passing them over to the JSON template.

$pfxCert = Get-Content "pathToYourSSL.pfx" -Encoding Byte
$pfxCert = [System.Convert]::ToBase64String($pfxCert)
The above steps will ensure SSL cert is passed on based on Base 64 Binary which is what is expected by the JSON parameter.
As part of the deployment I have also created a custom probe that target an HTTPS site which accepts anonymous authentication, otherwise the default probe would fail causing the whole site to be unavailable. Custom probes are a good way in overcoming issues around sites that require NTLM or basic authentication (but not form based authentication).
Make sure your Application Gateway can talk to the backend server, if you have firewalls or even IIS Request Filtering could cause the connection to fail, ensure appropriate ports are allowed both ways.
So without further ado I have attached the full code below for your leisure ….
{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "applicationGatewayName": {
      "type": "string",
      "minLength": 1
    },
    "location": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Region"
      }
    },
    "virtualNetworkName": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Application Gateway vnet name"
      }
    },
    "virtualNetworkResourceGroup": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Application Gateway vnet resource group name"
      }
    },
    "subnetName": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Application Gateway subnet name"
      }
    },
    "publicIPAddressName": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "PIP resource name"
      }
    },
    "certData": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "SSL data in binary Base64 format"
      }
    },
    "certPassword": {
      "type": "securestring",
      "minLength": 1,
      "metadata": {
        "description": "SSL certificate password"
      }
    },
    "authCert": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Public authentication certificate - in binary Base64 format"
      }
    },
    "Authhostname": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Backend hostheader/Host SNI names used as part of your HTTPS request"
      }
    },
    "Noauthhostname": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Backend hostheader/Host SNI names used as part of your HTTPS request"
      }
    },
    "backendIPAddress": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Backend web server IP address"
      }
    }
  },
  "variables": {
    "applicationGatewayID": "[resourceId('Microsoft.Network/applicationGateways',parameters('applicationGatewayName'))]",
    "capacity": 2,
    "skuName": "Standard_Medium",
    "subnetRef": "[concat(variables('vnetID'),'/subnets/',parameters('subnetName'))]",
    "vnetID": "[resourceId(parameters('virtualNetworkResourceGroup'),'Microsoft.Network/virtualNetworks',parameters('virtualNetworkName'))]"
  },
  "resources": [
    {
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[parameters('publicIPAddressName')]",
      "location": "[parameters('location')]",
      "properties": {
        "publicIPAllocationMethod": "Dynamic"
      },
      "tags": {
        "displayName": "[parameters('publicIPAddressName')]"
      }
    },
    {
      "apiVersion": "2016-06-01",
      "name": "[parameters('applicationGatewayName')]",
      "type": "Microsoft.Network/applicationGateways",
      "location": "[resourceGroup().location]",
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', parameters('publicIPAddressName'))]"
      ],
      "properties": {
        "sku": {
          "name": "[variables('skuName')]",
          "tier": "Standard",
          "capacity": "[variables('capacity')]"
        },
        "sslCertificates": [
          {
            "name": "appGatewaySslCert",
            "properties": {
              "data": "[parameters('certData')]",
              "password": "[parameters('certPassword')]",
              "publicCertData": "[parameters('authCert')]"
            }
          }
        ],
        "authenticationCertificates": [
          {
            "name": "PublicCert",
            "properties": {
               "data": "[parameters('authCert')]"
            }
          }

        ],
        "gatewayIPConfigurations": [
          {
            "name": "appGatewayIpConfig",
            "properties": {
              "subnet": {
                "id": "[variables('subnetRef')]"
              }
            }
          }
        ],
        "frontendIPConfigurations": [
          {
            "name": "appGatewayFrontendIP",
            "properties": {
              "PublicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',parameters('publicIPAddressName'))]"
              }
            }
          }
        ],
        "frontendPorts": [
          {
            "name": "appGWFEHttps",
            "properties": {
              "Port": 443
            }
          },
          {
            "name": "appGWFEHttp",
            "properties": {
              "Port": 80
            }
          }
        ],
        "probes": [
          {
            "name": "CustomProbe",
            "properties": {
              "protocol": "Https",
              "host": "auth.simplesite.com:443",
              "path": "/noauth/iisstart.htm",
              "interval": 5,
              "timeout": 120,
              "unhealthyThreshold": 2
            }
          }
        ],
        "backendAddressPools": [
          {
            "name": "appGatewayBackendPool",
            "properties": {
              "BackendAddresses": [
                {
                  "IpAddress": "[parameters('backendIPAddress')]"
                }
              ]
            }
          }
        ],
        "backendHttpSettingsCollection": [
          {
            "name": "appGWBEHttpSettings",
            "properties": {
              "Port": 80,
              "Protocol": "Http",
              "CookieBasedAffinity": "Disabled"
            }
          },
          {
            "name": "appGWBEHttpsSettings",
            "properties": {
              "Port": 443,
              "Protocol": "Https",
              "CookieBasedAffinity": "Disabled",
              "probe": {
                "id": "[concat(variables('applicationGatewayID'), '/probes/CustomProbe')]"
              },
              "authenticationCertificates": [
                {
                  "id": "[concat(variables('applicationGatewayID'), '/authenticationCertificates/PublicCert')]"
                }
              ]
            }
          }
        ],
        "httpListeners": [
          {
            "name": "appGatewayHttpListener",
            "properties": {
              "FrontendIPConfiguration": {
                "Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
              },
              "FrontendPort": {
                "Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGWFEHttp')]"
              },
              "Protocol": "Http",
              "SslCertificate": null
            }
          },
          {
            "name": "appGatewayHttpsListenerNoAuth",
            "properties": {
              "FrontendIPConfiguration": {
                "Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
              },
              "FrontendPort": {
                "Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGWFEHttps')]"
              },
              "Protocol": "Https",
              "hostName": "[parameters('Noauthhostname')]",
              "SslCertificate": {
                "Id": "[concat(variables('applicationGatewayID'), '/sslCertificates/appGatewaySslCert')]"
              },
              "RequireServerNameIndication": "true"
            }
          },
          {
            "name": "appGatewayHttpsListenerAuth",
            "properties": {
              "FrontendIPConfiguration": {
                "Id": "[concat(variables('applicationGatewayID'), '/frontendIPConfigurations/appGatewayFrontendIP')]"
              },
              "FrontendPort": {
                "Id": "[concat(variables('applicationGatewayID'), '/frontendPorts/appGWFEHttps')]"
              },
              "Protocol": "Https",
              "hostName": "[parameters('Authhostname')]",
              "SslCertificate": {
                "Id": "[concat(variables('applicationGatewayID'), '/sslCertificates/appGatewaySslCert')]"
              },
              "RequireServerNameIndication": "true"
            }
          }
        ],
        "requestRoutingRules": [
          {
            "Name": "HTTPrule",
            "properties": {
              "RuleType": "Basic",
              "httpListener": {
                "id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpListener')]"
              },
              "backendAddressPool": {
                "id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool')]"
              },
              "backendHttpSettings": {
                "id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGWBEHttpSettings')]"
              }
            }
          },
          {
            "Name": "HTTPSruleNoAuth",
            "properties": {
              "RuleType": "Basic",
              "httpListener": {
                "id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpsListenerNoAuth')]"
              },
              "backendAddressPool": {
                "id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool')]"
              },
              "backendHttpSettings": {
                "id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGWBEHttpsSettings')]"
              }
            }
          },
          {
            "Name": "HTTPSruleAuth",
            "properties": {
              "RuleType": "Basic",
              "httpListener": {
                "id": "[concat(variables('applicationGatewayID'), '/httpListeners/appGatewayHttpsListenerAuth')]"
              },
              "backendAddressPool": {
                "id": "[concat(variables('applicationGatewayID'), '/backendAddressPools/appGatewayBackendPool')]"
              },
              "backendHttpSettings": {
                "id": "[concat(variables('applicationGatewayID'), '/backendHttpSettingsCollection/appGWBEHttpsSettings')]"
              }
            }
          }
        ]
      }
    }
  ]
}
I know I haven’t written a detailed article around how each section of the JSON template code work together. I have tested this code in my own environment and I know it works for the web server I have for the backend. If you require help using the above template, please hit me with a message and I will try to clear some of the information around the code and usage cases.

5 Comments

The myth of Azure Application Gateways – Part 1

5db46fc5-d9c1-44ae-a0f6-4ae1d2301395Azure Application Gateways is a layer 7 reverse proxy service offered as a PaaS to general public. It supports SSL offloading, which means you can terminate your SSL connection at the Application Gateway and connect to the backend server using HTTP traffic or initiate a new SSL connection to your backend service.

This is all well and good, simple and painless if you have a single backend server with a single website. The complexity of the solution increases as the backend start leveraging more of the IIS functionalities such as Windows/NTLM authentication, SNI and host headers or various SSL certificates used for each sub-site (if you have multiple sites running on the same IIS server).

Before even starting to look at designing your Azure Application Gateway, there are few guidelines you will need to follow:

  • You should have an empty default site.
  • If using both HTTP/HTTPS protocols on any of the sub-sites, the default website should be listening on both 80 and 443.
  • In the case of HTTPS the default site will need to be loaded with a single SSL certificate that will primarily be used by the Application Gateway to authenticate against the server.
  • Not running SNI on default website.
  • If you are running NTLM or Windows authentication on any of the sites (except form based authentication) then you will need a site/page that allow anonymous authentication to be used for Application Gateway custom probe.
  • Use IP address for the backend pool rather than FQDN.

The above will save you a lot of hassle while implementing and configuring your Application Gateway to work with your backend web server.

Microsoft have fixed few issues we were experiencing recently with Application Gateways around SSL and custom probes.

There are two ways available to deploy an Application Gateway, Powershell or JSON template. The latter is preferable to ensure consistency at each deployment. This article is in two parts, in this article I will be using Powershell to deploy an Application Gateway.

Prerequisites:

  • SSL private key in PFX format for all sites using SSL
  • SSL public key in CER format for default site
  • IP address of the backend web server
  • Front and backend listening port
  • Site/page with anonymous access if requiring authentication

Powershell code below would deploy an Application Gateway listening on two ports (80,443). The backend consists of four sites with SNI and host headers enabled, two sites run under port 80, one of them require basic authentication. Another two sites run under port 443 bound with self-signed SSL cert for testing, one of the sites has basic authentication turned on. This would test the four common scenarios of a typical deployment.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
Login-AzureRmAccount
Get-AzureRmSubscription | Select Subscription* | out-gridview -Title "Select your Azure Subscription" -OutputMode Single | Select-AzureRmSubscription 

$ApplicationGWName = "ApplicationGatewayName"
$AppGatwayPIPName = "ApplicationGatewayPublicIPName"
$vnetName = 'vNETName'
$VNetResourceGroup = 'vNETResourceGroup'
$AppGWSubnetName = 'ApplicationGatewaySubnetName'
$ResouceGroupName = "ApplicationGatewayResourceGroupName"
$Location = "UK South"
$BackendIPaddresses =  "BackendServerIP"

New-AzureRmResourceGroup -Name $ResouceGroupName -Location $Location

#Prompt for Certificate password
$SecurePassword = Read-Host -AsSecureString  "Enter Certificate password"
$BSTR = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($SecurePassword)
$UnsecurePassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR)

#Retrive Virtual Network and Application Gateway information
$vnet = Get-AzureRmvirtualNetwork -Name $VnetName -ResourceGroupName $VNetResourceGroup
$gwSubnet = Get-AzureRmVirtualNetworkSubnetConfig -Name $AppGWSubnetName -VirtualNetwork $vnet

#Create Public IP address for Application Gateway
$publicip = New-AzureRmPublicIpAddress -ResourceGroupName $ResouceGroupName -Name $AppGatwayPIPName -Location $Location -AllocationMethod Dynamic
#$publicip = Get-AzureRmPublicIpAddress -Name $AppGatwayPIPName -ResourceGroupName $ResouceGroupName

#Create an application gateway IP configuration, this setting configures what subnet the application gateway uses
$gipconfig = New-AzureRmApplicationGatewayIPConfiguration -Name 'appGatewayIpConfig' -SubnetId $gwSubnet.Id -Verbose

#Create a front-end IP configuration, this setting maps the public ip address to the front-end of the application gateway
$fipconfig = New-AzureRmApplicationGatewayFrontendIPConfig -Name 'appGatewayFrontendIP' -PublicIPAddress $publicip

#Configure the back-end IP address pool with the IP addresses of the backend web servers
$pool = New-AzureRmApplicationGatewayBackendAddressPool -Name 'appGatewayBackendPool' -BackendIPAddresses $BackendIPaddresses

#Configure the front-end IP port for the public IP endpoint
$fp = New-AzureRmApplicationGatewayFrontendPort -Name 'appGatewayFrontendPorthttps' -Port 443
$fp1 = New-AzureRmApplicationGatewayFrontendPort -Name 'appGatewayFrontendPorthttp' -Port 80

#Configure the certificate for the application gateway
$cert = New-AzureRmApplicationGatewaySslCertificate -Name wildcard -CertificateFile '.\selfsignedcert.pfx'

#Create the HTTPS listener for the application gateway. Assign the front-end ip configuration, port, and ssl certificate to use
$listener1 = New-AzureRmApplicationGatewayHttpListener -Name authHTTPS -Protocol Https -FrontendIPConfiguration $fipconfig -FrontendPort $fp `
                                                       -HostName "auth.simplesite.com" -RequireServerNameIndication true -SslCertificate $cert

$listener2 = New-AzureRmApplicationGatewayHttpListener -Name noauthHTTPS -Protocol Https -FrontendIPConfiguration $fipconfig -FrontendPort $fp `
                                                       -HostName "noauth.simplesite.com" -RequireServerNameIndication true -SslCertificate $cert

$listener3 = New-AzureRmApplicationGatewayHttpListener -Name authHTTP -Protocol Http -FrontendIPConfiguration $fipconfig `
                                                       -FrontendPort $fp1 -HostName "auth.simplesite.com"

$listener4 = New-AzureRmApplicationGatewayHttpListener -Name noauthHTTP -Protocol Http -FrontendIPConfiguration $fipconfig `
                                                       -FrontendPort $fp1 -HostName "noauth.simplesite.com"

#Upload the certificate to be used on the ssl enabled backend pool resources - authentication certificate
$authcert = New-AzureRmApplicationGatewayAuthenticationCertificate -Name 'sanauthcert' -CertificateFile '.\publickey.cer'

#Configure the application gateway back-end http settings. Assign the certificate uploaded in the preceding command to the http settings
$poolSetting1 = New-AzureRmApplicationGatewayBackendHttpSettings -Name 'wildcard' -Port 443 -Protocol Https `
                                                                 -CookieBasedAffinity Enabled -AuthenticationCertificates $authcert

$poolSetting2 = New-AzureRmApplicationGatewayBackendHttpSettings -Name 'unsecured' -Port 80 -Protocol Http -CookieBasedAffinity Enabled

#Create a load balancer routing rule, Basic Round Robin
$rule1 = New-AzureRmApplicationGatewayRequestRoutingRule -Name 'authHTTPrule' -RuleType basic -BackendHttpSettings $poolSetting2 `
                                                         -HttpListener $listener3 -BackendAddressPool $pool
$rule2 = New-AzureRmApplicationGatewayRequestRoutingRule -Name 'noauthHTTPrule' -RuleType basic -BackendHttpSettings $poolSetting2 `
                                                         -HttpListener $listener4 -BackendAddressPool $pool
$rule3 = New-AzureRmApplicationGatewayRequestRoutingRule -Name 'authHTTPSrule' -RuleType basic -BackendHttpSettings $poolSetting1 `
                                                         -HttpListener $listener1 -BackendAddressPool $pool
$rule4 = New-AzureRmApplicationGatewayRequestRoutingRule -Name 'noauthHTTPSrule' -RuleType basic -BackendHttpSettings $poolSetting1 `
                                                         -HttpListener $listener2 -BackendAddressPool $pool

#Configure the instance size of the application gateway
$sku = New-AzureRmApplicationGatewaySku -Name Standard_Small -Tier Standard -Capacity 1

#Create the Application Gateway
$appgw = New-AzureRmApplicationGateway `
                -Name $ApplicationGWName `
                -SslCertificates $cert `
                -ResourceGroupName $ResouceGroupName `
                -Location $Location `
                -BackendAddressPools $pool `
                -BackendHttpSettingsCollection $poolSetting1,$poolSetting2 `
                -FrontendIpConfigurations $fipconfig `
                -GatewayIpConfigurations $gipconfig `
                -FrontendPorts $fp,$fp1 `
                -HttpListeners $listener1,$listener2,$listener3,$listener4 `
                -RequestRoutingRules $rule1,$rule2,$rule3,$rule4 `
                -Sku $sku `
                -AuthenticationCertificates $authcert `
                -Verbose

To be continued …..

4 Comments