PwnageChecker

Download from App Store

Download Source at Github

ScreenShot

I have recently submitted the PwnageChecker App I create a few months ago to App Store and early this week it was approved. Pwnagechecker is an iOS app built using Swift. It allows users to easily check whether an account has been compromised in a known data breach on any iOS device. It leverages the https://haveibeenpwned.com API created by Troy Hunt, which aggregates public leaked data from breaches and makes them readily searchable.

App Description

Adobe, LinkedIn, Snapchat, Sony, these are just some of the many high profile data breaches in recent years. Have you ever wondered if your account has been compromised in one of these data breaches? With PwnageChecker, you can easily check whether your account has been compromised in a known data breach.

Features:

  • Enter the account or email address to check whether it’s been breached.
  • Have an elderly at home or less technical friend you want to help? Check whether their email addresses has been compromised by selecting their contacts.
  • Browse known data breaches that are loaded into the service.
  • Subscribe to get notification when there is a data breach on your account (notification provided by http://haveibeenpwned.com)

Bootswatch Theme Preview Google Chrome Extension

Bootswatch offers many beautiful free themes for bootstrap. Recently I created a Google Chrome Extension that makes it easy to preview any existing bootstrap sites in one of these Bootswatch themes.

Install Extension From Chrome Web Store

Download Source From GitHub

How To Use

Install the extension from Chrome Web Store. Once the extension is installed, you can click on the icon icon to bring up the the following popup. From the popup, select the a theme and hit the Apply Theme button to apply theme on current page.

extension popup

How It Works

The extension has the following components:

  • A popup page and script for user interaction. It presents available themes and let user select desired theme.
  • A content script to apply a theme on current web page. This script is injected to the web page. It runs in a special context that has access to the page’s DOM. More on about content script later.
  • A background script for communication with the content script and organizing common functions and variables.

Content Script

Content script runs in a special environment of isolated world. It has access to the DOM of the page, but not any javascript functions or variables created by the page. It also does not have access to the functions and variables defined by the extensions’ pages. To communicate with the parent extension, we need to use message passing.

Message Passing

Since content scripts runs in isolated world and not part of the parent extension, it can only use message passing to communicate with the extension. For simple one time requests like the ones we have in this extension, we can use runtime.sendMessage or tabs.sendMessage to send messages, and runtime.onMessage to receive messages. For long-lived connections, chrome extension api offers other methods you can learn more about here

The Code

This is the extension’s manifest.json. Only the activeTab permission is specified as we will only inject content script to web page in the active tab.

{
  "manifest_version": 2,
  "name": "Bootswatch Theme Preview",
  "short_name": "BootswatchThemePreview",
  "description": "Preview a bootstrap site with a bootswatch theme.",
  "version": "1.0.1",
  "background": {
    "scripts": ["js/background.js"]
  },  
  "icons" : {
    "16" : "img/icon16.png",
    "48" : "img/icon48.png",
    "128": "img/icon128.png"
  },
  "browser_action": {
    "default_title": "Preview this page with a bootswatch theme.",
    "default_popup": "popup.html"
  },  
  "permissions" : [
    "activeTab"
  ]
}

This is the popup.js. When user clicks on the icon, the popup.html will be loaded and execute this script. This script serves two purposes. First, it creates the UI elements on the popup page. Second, it responds to the apply theme button’s click event and inject our content script to the current web page. When the button is clicked, it calls chrome.tabs.query to get the active tab. In the callback function (most of chrome extension api are asynchronous), it saves the selected theme to background script and calls chrome.tabs.executeScript to inject the content.js script to the web page.

document.addEventListener('DOMContentLoaded', function () {
    createBootswatchSelect();
    createBootswatchThumbnails();
    
    // When apply button is clicked
    document.getElementById("apply-button").addEventListener("click", function(){
        // Get the active tab
        chrome.tabs.query({'active': true, 'lastFocusedWindow': true}, function (tabs) {
            // Save the selected them
            var bootswatchSelectList = document.getElementById("bootswatch-theme-select");
            var bootswatchThemeName = bootswatchSelectList.options[bootswatchSelectList.selectedIndex].value;
            chrome.extension.getBackgroundPage().saveUserPreference(bootswatchThemeName);
            
            // And inject content script to the active tab.
            chrome.tabs.executeScript({
                file: 'js/content.js'                
            });
        });        
    });
});

/**
 * Create the select list for bootstrap themes.
 */
function createBootswatchSelect() {
    var themes = chrome.extension.getBackgroundPage().bootswatchThemes.themes;
    var userThemeName = chrome.extension.getBackgroundPage().getUserBootswatchThemeName();
    var container = document.getElementById("bootswatch-theme-container");

    //Create and append select list
    var selectList = document.createElement("select");
    selectList.id = "bootswatch-theme-select";
    selectList.className = "form-control";
    container.appendChild(selectList);

    //Create and append the options
    for (var i = 0; i < themes.length; i++) {
        var theme = themes[i];
        var option = document.createElement("option");
        option.value = theme.name;
        option.text = theme.name + " | " + theme.description;   
        if (theme.name == userThemeName) {
            option.selected = true;
        }
        selectList.appendChild(option);        
    }
}


/**
 * Create the thumbnails
 */
function createBootswatchThumbnails() {
    var themes = chrome.extension.getBackgroundPage().bootswatchThemes.themes;
    var container = document.getElementById("bootswatch-thumbnail-container");
    
    //Create and append the thumbnail images
    for (var i = 0; i < themes.length; i++) {
        (function() {
            var theme, div, a, img;
            theme = themes[i];
            
            div = document.createElement("div");
            div.className = "col-xs-3";
            
            a = document.createElement("a");
            a.className="thumbnail";
            a.href = "#";    
            
            img = document.createElement("img");
            img.className = "img-responsive";
            img.src = "img/" + theme.name + ".png";
            
            container.appendChild(div);
            div.appendChild(a);
            a.appendChild(img);
            
            // change the select list selected item when a thumbnail is clicked on.
            a.addEventListener("click", function(){
                console.log(theme.name);
                document.querySelector('#bootswatch-theme-select [value="' + theme.name + '"]').selected = true;
            });             
        }());  
    }
}

This is the content.js. It uses chrome.runtime.sendMessage api to communicate with the background script. When it receives a reply in the callback function, it gets the selected bootswatch theme’s css cdn, and append a stylesheet element on the current web page.

/**
 * Send a message to background script to get the saved theme and apply it to current page.
 */
chrome.runtime.sendMessage({method: "getUserBootswatchTheme"}, function(response) {    
    var link = document.getElementById("injected-bootswatch-theme");
    if (link) {
        // We have already injected a bootswatch theme on this page, update the path instead of creating additional link
        link.href = response.theme.cssCdn;
    }
    else {
        var link = document.createElement( "link" );
        link.href = response.theme.cssCdn;
        link.type = "text/css";
        link.rel = "stylesheet";
        link.id = "injected-bootswatch -theme";
        document.body.appendChild(link);  
    }
});

This is the background script. It listens to messages from the content script and responds with the saved theme.

...
/**
 * Listen and respond to messages sent from content script
 */
chrome.runtime.onMessage.addListener(function(request, sender, sendResponse) {
  if (request.method == "getUserBootswatchTheme") {
    // respond with preferred theme.
    sendResponse({ theme: getUserBootswatchTheme() });
  }
});

/**
 * get user's preferred bootswatch theme.
 */
function getUserBootswatchTheme() {
  var themeName = getUserBootswatchThemeName();
  for (var i = 0; i < bootswatchThemes.themes.length; i++) {
      if (bootswatchThemes.themes[i].name == themeName) {
        return bootswatchThemes.themes[i];
      }
  }
  console.log("no user bootswatch theme is found");
}

/**
 * get the name of user's preferred bootswatch theme.
 */
function getUserBootswatchThemeName() {
  var name = localStorage["UserBootstrapThemeName"];
  if (name == undefined) {
    name = "Readable";
  }
  return name;  
}

/**
 * save user's preferred theme to local storage.
 */
function saveUserPreference(bootswatchThemeName) {
  localStorage["UserBootstrapThemeName"] = bootswatchThemeName;
}

Conclusion

That’s it. This is a relatively simple and yet useful extension that demonstrate how to use a content script to make changes to an existing page. Now you can preview your bootstrap sites in one of the bootswatch themes without making any changes. When you decide you like the theme, all you have to do is to download the css file from bootswatch.com and replace the one in your project to apply it permanently.

References

Chrome Message Passing
Chrome Content scripts

REINSTALLMODE vs ReinstallModeText

REINSTALLMODE

This is a MSI Property. On MSDN:

The REINSTALLMODE property is a string that contains letters specifying the type of reinstall to perform. Options are case-insensitive and order-independent. This property should normally always be used in conjunction with the REINSTALL property. However, this property can also be used during installation, not just reinstall. By default the REINSTALLMODE is “omus”.

ReinstallModeText

This property is added automatically in an Install Shield project. It’s not the same as ReinstallMode. It’s only related in the following way. During reinstall, the Maintenance Type Dialog will copy the value from ReinstallModeText to ReinstallMode.

maintenancetypedialog

REINSTALLMODE vs ReinstallModeText

A couple weeks ago I was helping another developer who tried to solve an issue with an installation package. When the installer is run, certain dlls are not being updated. These dlls have the following characteristics:

  • The dlls in the package and on the target system are both versioned and the version numbers are the same.
  • The dlls in the package have newer modified date than the dlls on the target system.

MSDN has a nice flowchart and documentation that describes the file replacement rules for versioned dlls. The relevant information to the story is that, when an installer calculates whether a versioned file will be copied to target system, only the version number and the language will be considered. The modified date of the files will not be considered. Since the version numbers and the language of the dlls did not change, the files were not copied.

There are two ways to resolve this
1. The developer who created the versioned dlls should increment the version number when the dll is changed. In my opinion, this should be done regardless of the installer issue.
2. Change the REINSTALLMODE property from the default of omus to emus. The e option specifies that a file will be copied if the file is missing, or is an equal or older version.

Option 2 was chosen at the time because there are many dlls in the package that could have this issues, and checking all the dlls and recompiling the dlls with incremented version number can’t be done quickly enough to resolve the installer issue. The developer proceeded to update the install shield project. In the install shield project’s property manager, the ReinstallModeText is already defined with omus and REINSTALLMODE is not defined anywhere.

reinstallmodetext

An unsuspecting developer could easily be mistaken and just update that ReinstallModeText property, and that was what happened. After spending some time to investigate this, we found out the root cause. Everything worked as expected after we manually added the REINSTALLMODE property and set it to emus

Hope sharing this little story will help someone saves some time.

Troubleshooting IIS Process Crash

Recently I was asked to help on troubleshooting occasional crashes of the IIS w3wp.exe process on a production site. Other than having the same windows application events generated at the time of the crash, the crashes do not have any obvious pattern and do not appear to be related to the traffic level. After performing a memory dump analysis using windbg, it turned out the crash was caused by a decryption function that was decrypting an empty string.

Problem

The following is the code that caused the exception.

Public Shared Function strDecrypt(ByVal encryptedString As String) As String
    encryptedString = encryptedString.Replace(" ", "+")
    Dim pstrDecrKey As String = "renamedFakeKey"
    Dim byKey As Byte() = {}
    Dim IV As Byte() = {18, 52, 86, 120, 144, 171, 205, 239}
    Dim inputByteArray As Byte() = New Byte() {encryptedString.Length}

    byKey = System.Text.Encoding.UTF8.GetBytes(pstrDecrKey.Substring(0, 8))
    Dim des As DESCryptoServiceProvider = New DESCryptoServiceProvider()
    inputByteArray = Convert.FromBase64String(encryptedString)
    Dim ms As MemoryStream = New MemoryStream()
    Dim cs As CryptoStream = New CryptoStream(ms, des.CreateDecryptor(byKey, IV), CryptoStreamMode.Write)
    cs.Write(inputByteArray, 0, inputByteArray.Length)
    cs.FlushFinalBlock()
    Return System.Text.Encoding.UTF8.GetString(ms.ToArray())
End Function

Solution

The fix is simply to check the input for empty string and return immediately

If string.IsNullOrEmpty(encryptedString) Then
    Return string.Empty
End If

Additional Information

This problem appears to have existed for a while as it was reported here back in 2009 and mentioned here in 2011.

Note that this problem only happens on 64 bit OS but not 32 bit OS. If you upgraded a web server OS from a old windows 32 bit OS to a newer windows 64 bit OS, and the site suddenly experienced random crashes, check if you have similar code and verify if you are having the same problem.

The fix is trivial once the cause is known, but finding out the root cause was a bit more involved. I shared below the process I went through to troubleshoot this.

At the time of the crash, three windows event log entries are generated.

Application Error Event 1023

Log Name:      Application
Source:        .NET Runtime
Date:          2/24/2016 12:53:02 PM
Event ID:      1023
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      xxxxxx
Description:
.NET Runtime version 2.0.50727.5485 - Fatal Execution Engine Error (000007FEF099600A) (80131506)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name=".NET Runtime" />
    <EventID Qualifiers="0">1023</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2016-02-24T19:53:02.000000000Z" />
    <EventRecordID>137453</EventRecordID>
    <Channel>Application</Channel>
    <Computer>xxxxxx</Computer>
    <Security />
  </System>
  <EventData>
    <Data>.NET Runtime version 2.0.50727.5485 - Fatal Execution Engine Error (000007FEF099600A) (80131506)</Data>
  </EventData>
</Event>

Application Event 1001

Log Name:      Application
Source:        Windows Error Reporting
Date:          2/24/2016 12:53:02 PM
Event ID:      1001
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      xxxxxx
Description:
Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0

Problem signature:
P1: w3wp.exe
P2: 7.5.7601.17514
P3: 4ce7afa2
P4: mscorwks.dll
P5: 2.0.50727.5485
P6: 53a11d6c
P7: c0000005
P8: 00000000006b9089
P9: 
P10: 

Attached files:

These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_w3wp.exe_b75d5a4315af67c6bf1c7ab752f52b4cfb718_611caf08

Analysis symbol: 
Rechecking for solution: 0
Report Id: 3626349d-db30-11e5-b7d1-00155d023016
Report Status: 4
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Windows Error Reporting" />
    <EventID Qualifiers="0">1001</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2016-02-24T19:53:02.000000000Z" />
    <EventRecordID>137455</EventRecordID>
    <Channel>Application</Channel>
    <Computer>xxxxxx</Computer>
    <Security />
  </System>
  <EventData>
    <Data>
    </Data>
    <Data>0</Data>
    <Data>APPCRASH</Data>
    <Data>Not available</Data>
    <Data>0</Data>
    <Data>w3wp.exe</Data>
    <Data>7.5.7601.17514</Data>
    <Data>4ce7afa2</Data>
    <Data>mscorwks.dll</Data>
    <Data>2.0.50727.5485</Data>
    <Data>53a11d6c</Data>
    <Data>c0000005</Data>
    <Data>00000000006b9089</Data>
    <Data>
    </Data>
    <Data>
    </Data>
    <Data>
    </Data>
    <Data>C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_w3wp.exe_b75d5a4315af67c6bf1c7ab752f52b4cfb718_611caf08</Data>
    <Data>
    </Data>
    <Data>0</Data>
    <Data>3626349d-db30-11e5-b7d1-00155d023016</Data>
    <Data>4</Data>
  </EventData>
</Event>

Application Error Event 1000

Log Name:      Application
Source:        Application Error
Date:          2/24/2016 12:53:02 PM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      xxxxxx
Description:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: mscorwks.dll, version: 2.0.50727.5485, time stamp: 0x53a11d6c
Exception code: 0xc0000005
Fault offset: 0x00000000006b9089
Faulting process id: 0x%9
Faulting application start time: 0x%10
Faulting application path: %11
Faulting module path: %12
Report Id: %13
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2016-02-24T19:53:02.000000000Z" />
    <EventRecordID>137454</EventRecordID>
    <Channel>Application</Channel>
    <Computer>xxxxxx</Computer>
    <Security />
  </System>
  <EventData>
    <Data>w3wp.exe</Data>
    <Data>7.5.7601.17514</Data>
    <Data>4ce7afa2</Data>
    <Data>mscorwks.dll</Data>
    <Data>2.0.50727.5485</Data>
    <Data>53a11d6c</Data>
    <Data>c0000005</Data>
    <Data>00000000006b9089</Data>
  </EventData>
</Event>

The information in the event log is limited. To find out what went wrong, I had to dig a little deeper and perform a memory dump analysis.

Collect Memory Dump
First I had to collect a memory dump when the IIS process crashes. In my case, the web server was already configured with WER to do that. The location of the memory dump was indicated in the evnet 1001 event log entry event.

A couple tools can be used to collect memory dump:
WER
DebugDiag

Analyze Memory Dump
To analyze the memory dump, I use use WinDbg. A list of common windbg commands can be found here

Download WinDbg if you do not already have it. Start WinDbg and go to File > Open Crash Dump to load the memory dump.

WinDbg

Enter the following command which will display information about the current exception or bug check in verbose format.

!analyze -v

This gives me the following output.

Loading unloaded module list
....................................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(5a8.172c): Access violation - code c0000005 (first/second chance not available)
ntdll!ZwWaitForSingleObject+0xa:
00000000`7797d9fa c3              ret
0:070> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

*** WARNING: Unable to verify checksum for mscorlib.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System.Web\aa5e82ba0882a68ec638bc04d3679a53\System.Web.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Web.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System\c7fb84e825f6604d7f4684ab96cbd148\System.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System.Data\326df77841f23d0e0039374a7d233416\System.Data.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Data.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\Microsoft.VisualBas#\6301f41d5d4d86722d00e611f0899490\Microsoft.VisualBasic.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for Microsoft.VisualBasic.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System.Xml\d09a5530f1283b469957bf146e2f4d65\System.Xml.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Xml.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System.Web.Services\0ec14eb4a93723f6f5f24154fa9def53\System.Web.Services.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Web.Services.ni.dll
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System.Web.Extensio#\9286bb3f5e8fbfb834fed7588a2271bd\System.Web.Extensions.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Web.Extensions.ni.dll
*** ERROR: Module load completed but symbols could not be loaded for System.Web.Extensions.ni.dll

DUMP_CLASS: 2

DUMP_QUALIFIER: 400

CONTEXT:  (.ecxr)
rax=00000000050ebfb0 rbx=0000000000000000 rcx=00000000ffffffff
rdx=0000000000000001 rsi=0000000000000000 rdi=0000000180a027e8
rip=000007fefa3c9089 rsp=000000000a82d380 rbp=0000000000000000
 r8=0000000000000000  r9=0000000000000000 r10=e35a172cdd8cf770
r11=000000000a82d2e8 r12=000000000a82d478 r13=0000000000000001
r14=0000000080090005 r15=00000000054670d0
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
mscorwks!COMCryptography::_DecryptData+0x329:
000007fe`fa3c9089 0fb63401        movzx   esi,byte ptr [rcx+rax] ds:00000001`050ebfaf=??
Resetting default scope

FAULTING_IP: 
mscorwks!COMCryptography::_DecryptData+329
000007fe`fa3c9089 0fb63401        movzx   esi,byte ptr [rcx+rax]

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 000007fefa3c9089 (mscorwks!COMCryptography::_DecryptData+0x0000000000000329)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: 00000001050ebfaf
Attempt to read from address 00000001050ebfaf

PROCESS_NAME:  w3wp.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE_STR:  c0000005

EXCEPTION_PARAMETER1:  0000000000000000

EXCEPTION_PARAMETER2:  00000001050ebfaf

READ_ADDRESS:  00000001050ebfaf 

FOLLOWUP_IP: 
mscorwks!COMCryptography::_DecryptData+329
000007fe`fa3c9089 0fb63401        movzx   esi,byte ptr [rcx+rax]

WATSON_BKT_PROCSTAMP:  4ce7afa2

WATSON_BKT_PROCVER:  7.5.7601.17514

PROCESS_VER_PRODUCT:  Internet Information Services

WATSON_BKT_MODULE:  mscorwks.dll

WATSON_BKT_MODSTAMP:  53a11d6c

WATSON_BKT_MODOFFSET:  6b9089

WATSON_BKT_MODVER:  2.0.50727.5485

MODULE_VER_PRODUCT:  Microsoft® .NET Framework

BUILD_VERSION_STRING:  6.1.7601.19045 (win7sp1_gdr.151019-1254)

MODLIST_WITH_TSCHKSUM_HASH:  857813e725729c7dd498f79da93398e4b08dead1

MODLIST_SHA1_HASH:  5a768968a550e35d43040d297ed764b41596e31f

NTGLOBALFLAG:  0

APPLICATION_VERIFIER_FLAGS:  0

PRODUCT_TYPE:  3

SUITE_MASK:  272

DUMP_FLAGS:  d96

DUMP_TYPE:  0

APP:  w3wp.exe

From Faulting module name: mscorwks.dll, version: 2.0.50727.5485 and
FAULTING_IP: mscorwks!COMCryptography::_DecryptData+329, We can tell that the crash happens in the mscorwks.dll and in the DecryptData function. Doing a bit of googling led to this blog post. I also found the strDecrypt function in the website’s code base and I verified on a test site that passing empty string to the function will crash the iis process.

It’s been a few weeks after fixing the code and no crashes so far.

Running ASPNET 5 and .NET Core on Linux

I have been hearing quite a bit about the new ASPNET 5 (vNext) and .NET Core for a while now. However, the frequent breaking changes between beta versions was too much for me to want to try it out. With ASPNET 5 and .NET Core in release candidate and a 1.0 release in sight, I thought it’s time to give it a try.

The super basics on ASPNET 5 and .NET Core

A one line summary on ASPNET 5 and .NET Core to set the stage.

ASP.NET 5 is a lean and composable framework for building web and cloud applications. ASP.NET 5 is fully open source
.NET Core is a cross-platform implementation of .NET that is primarily being driven by ASP.NET 5 workloads, but also by the need and desire to have a modern runtime that is modular and whose features and libraries can be cherry picked based on the application’s needs.

Choosing the OS

ASPNET 5 and CoreCLR can run on Windows, Mac, or Linux. I choose try it on Linux as it’s the most interesting option to me. I think one can safely assume that ASPNET 5 will run relatively smooth on Windows compared to other OS options. Moreover, if I can get it to work on another OS, it should only work better in the windows environment.  Mac is interesting, but it’s use case is mostly as a development environment. Linux on the other can be both a viable development environment as well as an attractive hosting option.  In my test, I used Ubuntu Server 14.04 as it’s one of the most popular linux distributions and it’s easy to use.

Choosing between Mono Runtime vs .NET Core Runtime

Developers are going to have a lot more options in the new ecosystem. Even on Linux, there are two options for .NET runtime to choose from: mono runtime or coreclr. Mono is an open source implementation of .NET framework by Xamarin and has its own mono runtime. It has been around 10+ years and supports ( some ) web and desktop .NET applications to run on Linux. .NET Core is a a cross-platform implementation of .NET by Microsoft and coreclrt is it’s runtime.

Picking between mono runtime and coreclr on linux is like picking between the full CLR and coreclr on windows. The full CLR supports any third party libraries available today. The coreclr while leaner and modular, requires libraries to specifically target the .NET Core. As a result, the amount of libraries supported in .NET Core will be limited in the beginning, but should improve over time as adoption increases.

One of the main goals of ASPNET 5 is about lightweight and composable framework, and I do think it’s worth pursuing. I am going to try out coreclr.

Setting up the environment

Getting the necessary components installed is surprisingly easy using the getting started page on asp.net site.

Install the .NET Version Manager (DNVM)

dnvm is the .NET Version Manager, a set of command line utilities to update and configure which .NET Runtime to use.

curl -sSL https://raw.githubusercontent.com/aspnet/Home/dev/dnvminstall.sh | DNX_BRANCH=dev sh && source ~/.dnx/dnvm/dnvm.sh

Install the .NET Execution Environment (DNX)

The .NET Execution Environment contains the code required to bootstrap and run an application. This includes things like the compilation system, SDK tools, and the native CLR hosts.

#install dnx prerequisite
sudo apt-get install libunwind8 gettext libssl-dev libcurl4-openssl-dev zlib1g libicu-dev uuid-dev

#use dnvm to install .net core
dnvm upgrade -r coreclr

At this point, if you run dnvm list to list the installed runtime, you should see something similar to:

Active Version              Runtime Architecture OperatingSystem Alias
------ -------              ------- ------------ --------------- -----
  *    1.0.0-rc1-update1    coreclr x64          linux           default

Notice only the coreclr runtime is installed. The getting started page on asp.net site also has instruction to install the mono runtime. Since I am interested in trying out the coreclr, not installing mono runtime will actually make it easier for me to verify it's working in coreclr.

Install Libuv

Libuv is a multi-platform asynchronous IO library that is used by Kestrel, a cross-platform HTTP server for hosting ASP.NET 5 web applications.

sudo apt-get install make automake libtool curl
curl -sSL https://github.com/libuv/libuv/archive/v1.4.2.tar.gz | sudo tar zxfv - -C /usr/local/src
cd /usr/local/src/libuv-1.4.2
sudo sh autogen.sh
sudo ./configure
sudo make
sudo make install
sudo rm -rf /usr/local/src/libuv-1.4.2 && cd ~/
sudo ldconfig

Running the HelloMVC sample project

Once the environment is set, it's also easy to run the Hello World MVC project.

#install git if it's not already installed
sudo apt-get install git

#clone the aspnet 5 repo
git clone https://github.com/aspnet/home

#change directory into the HelloMvc sample
#1.0.0-rc1-update matches the runtime version and could be different for you depends on what runtime you installed
cd home/samples/1.0.0-rc1-update1/HelloMvc/

#restore the packages required by HelloMvc project
dnu restore

#start the app
dnx web

If you browse to http://my-ip-address:5004, you should see the running sample application

HelloMvc

dnu restore command

dnu restore command looks at the dependencies of the application and download them, adding them to the apps packages directory. The dependencies is defined in project.json file. In the HelloMvc project, the dependencies are:

"dependencies": {
"Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final",
"Microsoft.AspNet.IISPlatformHandler": "1.0.0-rc1-final",
"Microsoft.AspNet.Diagnostics": "1.0.0-rc1-final",
"Microsoft.AspNet.Mvc": "6.0.0-rc1-final",
"Microsoft.Extensions.Logging.Console": "1.0.0-rc1-final"
},

dnx web command

dnx web command runs the application.

In earlier documentation, you might see references to dnx kestrel command instead. If you try to run the dnx kestrel command. You will get this error message: Error: Unable to load application or execute command 'kestrel'. Available commands: web. The reason is that web and kestrel are special commands expected to be defined in project.json. In the latest version of the HelloMvc sample, only web command is defined. In earlier beta version of the sample has both web and kestrel defined.

Open the latest version of the HelloMvc project.json file in 1.0.0-rc1-update1/HelloMvc directory, here only the web command is defined. Also pay attention to the server.urls argument, the value is http://*:5004, this explains why the sample runs on port 5004.

commands: {
 web: Microsoft.AspNet.Server.Kestrel --server.urls http://*:5004
},

Open an earlier version of the HelloMvc project.json file in1.0.0-beta7/HelloMvc, you will see that both web and kestrel commands are defined.

commands: {
 web: Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.WebListener --server.urls http://localhost:5001,
 kestrel: Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.Kestrel --server.urls http://localhost:5004
},

Running MusicStore sample project

Getting the HelloMvc sample to run was good, but I need to see a non trivial application running before I want to invest more time in this. My next test is getting the AspNet MusicStore sample project to run. This project has authentication and EF7, which makes it a good target. This project uses InMemory provider for database access as only SqlServer, SqlLite and InMemory providers are available at the moment.

Following similar steps we did for HelloMvc project

#clone the musicstore repo
git clone https://github.com/aspnet/MusicStore.git

#change the directory into the sample project
cd MusicStore/src/MusicStore

#restore package
dnu restore

#run the application
dnx web

At this point, I see the following in the console which indicated the application is running. It looks promising.

ubuntu@:~/MusicStore/src/MusicStore$ dnx web
Hosting environment: Production
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.

However, if I  try to access it, it get the following scary looking exception.

<img class="alignnone size-full wp-image-660" src="https://binaryexplorer.files.wordpress.com/2015/12/musicstore.png&quot; alt="MusicStore" width="1179" height="937" />

fail: Microsoft.AspNet.Server.Kestrel[13]
An unhandled exception was thrown by the application.
System.InvalidOperationException: Unable to resolve service for type 'Microsoft.Extensions.CompilationAbstractions.ILibraryExporter' while attempting to activate 'Microsoft.AspNet.Mvc.Razor.Compilation.RoslynCompilationService'.
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.PopulateCallSites(ServiceProvider provider, ISet`1 callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.CreateCallSite(ServiceProvider provider, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetResolveCallSite(IService service, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetServiceCallSite(Type serviceType, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.PopulateCallSites(ServiceProvider provider, ISet`1 callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.CreateCallSite(ServiceProvider provider, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetResolveCallSite(IService service, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetServiceCallSite(Type serviceType, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.PopulateCallSites(ServiceProvider provider, ISet`1 callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.CreateCallSite(ServiceProvider provider, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetResolveCallSite(IService service, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetServiceCallSite(Type serviceType, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.PopulateCallSites(ServiceProvider provider, ISet`1 callSiteChain, ParameterInfo[] parameters, Boolean throwIfCallSiteNotFound)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.Service.CreateCallSite(ServiceProvider provider, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetResolveCallSite(IService service, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetServiceCallSite(Type serviceType, ISet`1 callSiteChain)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType, ServiceProvider serviceProvider)
at System.Collections.Concurrent.ConcurrentDictionaryExtensions.GetOrAdd[TKey,TValue,TArg](ConcurrentDictionary`2 dictionary, TKey key, Func`3 valueFactory, TArg arg)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.AspNet.Mvc.Razor.Internal.MvcRazorMvcViewOptionsSetup.ConfigureMvc(IServiceProvider serviceProvider, MvcViewOptions options)
at Microsoft.Extensions.OptionsModel.OptionsCache`1.CreateOptions()
at System.Threading.LazyInitializer.EnsureInitializedCore[T](T&amp;amp;amp;amp;amp; target, Boolean&amp;amp;amp;amp;amp; initialized, Object&amp;amp;amp;amp;amp; syncLock, Func`1 valueFactory)
at Microsoft.AspNet.Mvc.ViewEngines.CompositeViewEngine..ctor(IOptions`1 optionsAccessor)
at lambda_method(Closure , ServiceProvider )
at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.AspNet.Mvc.ViewResult.&amp;amp;amp;amp;lt;ExecuteResultAsync&amp;amp;amp;amp;gt;d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.&amp;amp;amp;amp;lt;InvokeResultAsync&amp;amp;amp;amp;gt;d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.&amp;amp;amp;amp;lt;InvokeResultFilterAsync&amp;amp;amp;amp;gt;d__44.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.&amp;amp;amp;amp;lt;InvokeAllResultFiltersAsync&amp;amp;amp;amp;gt;d__43.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.&amp;amp;amp;amp;lt;InvokeResourceFilterAsync&amp;amp;amp;amp;gt;d__38.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.&amp;amp;amp;amp;lt;InvokeAsync&amp;amp;amp;amp;gt;d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.&amp;amp;amp;amp;lt;InvokeActionAsync&amp;amp;amp;amp;gt;d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Builder.RouterMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Authentication.AuthenticationMiddleware`1.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Session.SessionMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Session.SessionMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Diagnostics.ExceptionHandlerMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNet.Diagnostics.ExceptionHandlerMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Diagnostics.StatusCodePagesMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Hosting.Internal.RequestServicesContainerMiddleware.&amp;amp;amp;amp;lt;Invoke&amp;amp;amp;amp;gt;d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNet.Hosting.Internal.HostingApplication.&amp;amp;amp;amp;lt;ProcessRequestAsync&amp;amp;amp;amp;gt;d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.AspNet.Server.Kestrel.Http.Frame`1.&amp;amp;amp;amp;lt;RequestProcessingAsync&amp;amp;amp;amp;gt;d__3.MoveNext()

After some googling, the solution turned out to be a simple one. The sample requires the latest runtime. Run the following to update it to latest.

#upgrade the runtime to latest version
#-r specifies the runtime we are upgrading is coreclr
#-u use the unstable feed to get the latest
dnvm upgrade -u -r coreclr

#verify the version
dnvm list

#dnvm list output indicates that we have version 1.0.0-rc2-16319 and it's the active runtime
Active Version              Runtime Architecture OperatingSystem Alias

------ -------              ------- ------------ --------------- -----

       1.0.0-rc1-update1    coreclr x64          linux           

  *    1.0.0-rc2-16319      coreclr x64          linux           default

MusicStore
Run the dnx web command again and access the site. Yay! I am greeted with the home page of the music store. Browsing around the site and adding items to cart is also working.

Conclusion

In the blog post, we configured the .net runtime environment on linux. We also setup the HelloMvc and MusicStore sample application to run on the coreclr runtime. There were small hiccups but the overall experience is a pleasant one.

The libraries available on the Core CLR are still limited. But it looks like more are coming soon. A quick look at a couple libraries that I plan to use already has some support:
* Npgsql’s (PostgreSQL data access provider) development branch already fully support CoreCLR.
* AWS SDK also has beta support for the CoreCLR.

References:

Display Activity Indicator with Overlay

ActivityIndicatorWithOverlay

It’s a common task to provide visual cue to the user when a task is in progress. Apple UIKit has the UIActivityIndicatorView UI component for this purpose. The component appears as a “gear” that is either spinning or stopped  uiactivityindicator.  In this post, we will extend it:

  • Show a transparent overlay over the whole screen or part of the screen. The overlay will prevent user interaction with the covered controls while the task is in progress.

  • Create the animating UIActivityIndicatorView programmatically in the center of the overlay to indicate a task is being processed

  • Show additional text in the overlay to provide additional information

Create the Overlay

We create the overlay as a UIView control.

The frame size of the overlay will be the same as the target it’s covering.

let overlay = UIView(frame: overlayTarget.frame)

Position the overlay to completely cover its target by setting its center to be the same as its target.

overlay.center = overlayTarget.center

Set the overlay alpha to 0 (completely transparent) initially. We will gradually change it to 0.5 later to animate the display of the overlay.

overlay.alpha = 0
overlay.backgroundColor = UIColor.blackColor()

Add the overlay to its target. Bring the overlay to the front to make sure it covers the target and blocks user interaction with  other controls on the target.

overlayTarget.addSubview(overlay)
overlayTarget.bringSubviewToFront(overlay)

Animate the display of the overlay by gradually changing the alpha value

UIView.beginAnimations(nil, context: nil)
UIView.setAnimationDuration(0.5)
overlay.alpha = overlay.alpha > 0 ? 0 : 0.5
UIView.commitAnimations()

Create the UIActivityIndicatorView

let indicator = UIActivityIndicatorView(activityIndicatorStyle: UIActivityIndicatorViewStyle.White)
indicator.center = overlay.center
indicator.startAnimating()
overlay.addSubview(indicator)

Create the Loading Text Label

When loading text is given, we will create a UILabel to show it. We call sizeToFit to resize the label to fit the text. We position the label right below the UIActivityIndicatorView in the overlay.

if let textString = loadingText {
    let label = UILabel()
    label.text = textString
    label.textColor = UIColor.whiteColor()
    label.sizeToFit()
    label.center = CGPoint(x: indicator.center.x, y: indicator.center.y + 30)
    overlay.addSubview(label)
}

To Use

Include the LoadingIndicatorView.swift file.

Call one of the show() methods to show the loading indicator before the long running task.

// Show indicator to cover entire screen
LoadingIndicatorView.show()
// Show indicator to cover entire screen with custom text
LoadingIndicatorView.show("Loading")
// Show indicator to cover target view
LoadingIndicatorView.show(target)
// Show indicator to cover target view with custom text
LoadingIndicatorView.show(target, "Loading")

Call the hide() method to hide the loading indicator after long running task.

LoadingIndicatorView.hide()

Download Source And Demo

Source on GitHub

Hidden Attribute causes UnauthorizedAccessException in File.Copy

unauthorized-access-exception

If you are getting “System.UnauthorizedAccessException: Access to the path ‘destFileName’ is denied” when you call File.Copy to copy and overwrite a file, the two more obvious possible causes are documented on MSDN:

Exception Condition
UnauthorizedAccessException

The caller does not have the required permission.

-or-

destFileName is read-only.

I ran into a third possible cause last week. If the destination file has the hidden file attribute, it will also cause the UnauthorizedAccessException. I wrote a short code snippet to quickly verify it.

private static void Main(string[] args) {
    string fromFile = string.Format("from-{0:hhmmss}.txt", DateTime.Now);
    string toFile = string.Format("to-{0:hhmmss}.txt", DateTime.Now);
    File.Create(fromFile).Close();
    File.Create(toFile).Close();

    // try copy without hidden attribute
    try {
        Console.WriteLine("Try copy without hidden attribute>>>");
        File.Copy(fromFile, toFile, true);
        Console.WriteLine("File successfully copied");
    }
    catch (UnauthorizedAccessException ex) {
        Console.WriteLine("File not copied. System.UnauthorizedAccessException. {0}", ex.Message);
    }

    // add hidden attribute to dest file
    File.SetAttributes(toFile, File.GetAttributes(toFile) | FileAttributes.Hidden);

    // try copy with hidden attribute
    try {
        Console.WriteLine("\nTry copy with hidden attribute>>>");
        File.Copy(fromFile, toFile, true);
        Console.WriteLine("File successfully copied.");
    }
    catch (UnauthorizedAccessException ex) {
        Console.WriteLine("File not copied. System.UnauthorizedAccessException. {0}", ex.Message);
    }
}
/*
Outout:
Try copy without hidden attribute>>>
File successfully copied

Try copy with hidden attribute>>>
File not copied. System.UnauthorizedAccessException. Access to the path 'to-101146.txt' is denied.
*/

Troubleshooting Windows System Process CPU Spike

In this blog post, we will look at a case of Windows system process CPU spike and the procedure to troubleshoot it.

In general, if the system process on your Windows system is having high CPU usage, you want to

  1. Determine the thread in the system process that taking up the CPU resource and Identify the driver or subsystem the thread is running for.
  2. Once you determine the source of the issue, you can formulate a fix. Depends on your situation, this could be disabling the offending component. Or if you are lucky, obtaining and applying an update from the vendor of the offending component.

Introduction:
My colleague recently recommended a customer to make a configuration change for a software we are supporting. After the change is made, the customer noticed a CPU spike from the System process. (It’s actually not related to the configuration change, but the customer started paying attention after the change and noticed the CPU spike). The only way to make sure it’s not related to the config change is to figure out what caused the CPU spike.

What is System Process
Mark’s blog post has a good explanation as well as some very helpful tips on how to troubleshoot System Process CPU Spike.

“The System process is special because it doesn’t host an executable image like other processes. It exists solely to host operating system threads for the memory manager, cache manager, and other subsystems, as well as device driver threads.  These threads execute entirely in kernel mode”

Confirming the CPU spike

A quick look at the Task manager confirms the problem. task-manager-showing-cpu-spike

Identify the thread in system process that is taking up CPU resource

Using process explorer, we can look at the threads running in the system process (In process explorer, right click on system process > properties > Threads tab). In my case, process explorer shows that there are a few threads taking up a few percents of CPU each.
process-explorer 

It also indicates the start address and it points to the srv2.sys driver. This is the Microsoft SMB 2.0 server driver.

Coming up with a solution
Now that I know the source of the problem, a search on high CPU problem related to srv2.sys quickly leads me to this knowledge base article (2732618) – High CPU usage on a file server that is running Windows Vista, Windows 7, Windows Server 2008 or Windows Server 2008 R2 with ABE enabled. The cause section of the article says “This issue occurs because there are many access check requests when ABE enumerates a folder that contains many subfolders.”

I confirm that the machine is running Windows 2008 R2. It’s a file server with at least one shared folder that has ABE enabled. And we are having CPU spike issue. The solution would be to install the hot fix in the knowledge base article.

Link

Using PostgreSQL with Entity Framework in ASP.NET MVC

In this blog post we are going to learn how to use PostgreSQL with Entity Framework in an ASP.NET MVC application.

Introduction:

I wanted to investigate using Entity Framework with PostgreSQL in an ASP.NET MVC application quickly. Instead of writing a trivial demo that simply read and write data from the database, I decided a better approach would be to convert an existing ASP.NET MVC application that already uses Entity Framework (with another database) to work with PostgreSQL. I decided to use MvcMusicStore. It is one of best documented tutorial ASP.NET MVC applications. This is the description on its project site: “MVC Music Store is a tutorial application built on ASP.NET MVC. It’s a lightweight sample store which demonstrates ASP.NET MVC using Entity Framework”.

By going through the process of converting a working application, I can concentrate on the area that are specific to using Entity Framework with PostgreSQL. I can also easily perform basic testing to verify the changes I make by just running the application. The rest of this post documents that steps I have gone though to convert the MvcMusicStore.

What you will need:

  • A working installation of PostgreSQL. I use PostgreSQL version 9.3.
  • A development environment where you can compile and run ASP.NET MVC application.

Instructions:

Step 1: Setup MvcMusicStore
Download “MvcMusicStore”. Unzip the folder and open the project. At this point, you should be able to compile the project. If not, there is something wrong with your development environment, resolve it now before you make more changes that could complicate it further. The project is configured to use SqlServerSe If it’s installed, you should be able to run the application.
MVC-Music-Store-Thumb

Step 2: Prepare the PostgreSQL database.
MvcMusicStore uses EntityFramework Code First.
In the global.asax.cs file, it specifies the SampleData class as the database initializer.

protected void Application_Start()
{
    // specify database initializer
    System.Data.Entity.Database.SetInitializer(new MvcMusicStore.Models.SampleData());

    AreaRegistration.RegisterAllAreas();

    RegisterGlobalFilters(GlobalFilters.Filters);
    RegisterRoutes(RouteTable.Routes);
}

Take a look at the SampleData class, notice that it inherits from DropCreateDatabaseIfModelChanges. This means that the application will re-create and re-seed the database when the schema changes.

// inherits from DropCreateDatabaseIfModelChanges
public class SampleData : DropCreateDatabaseIfModelChanges
{
    // The seed method will seed the database with initial data
    protected override void Seed(MusicStoreEntities context)
    {
        var genres = new List
        {
            new Genre { Name = "Rock" },
             new Genre { Name = "Jazz" },
             new Genre { Name = "Metal" },
             new Genre { Name = "Alternative" },
             new Genre { Name = "Disco" },
             new Genre { Name = "Blues" },
             new Genre { Name = "Latin" },
             new Genre { Name = "Reggae" },
             new Genre { Name = "Pop" },
             new Genre { Name = "Classical" }
         };
// more code not shown here .......

Unfortunately database migration and creation is not yet supported in Npgsql Entity Framework. We will have to create the database and seed the data manually. a) Create a database in your PostgreSQL server. Name the database MvcMusicStore. b) Next, we need to create the tables and seed them with data. The MvcMusicStore download contains a \MvcMusicStore-Assets\Data\MvcMusicStore-Create.sql file that works for MSSQL. We can use it as the base and adapt it for postgreSQL. You can use the finished script here MvcMusicStore-Create-PostgreSQL. The file has documentation on what was changed from the original script.

Step 3: Install ADO.NET provider and Entity Framework provider for postgreSQL.
You will have multiple options here. We will use Npgsql.
The MvcMusicStore download uses EntityFramework 4.1, which is two versions older than the current version (Entity Framework 6). Let’s upgrade it to the latest first via Nuget.

Install-Package EntityFramework

Next, Install Npgsql PostgreSQL Entity Framework provider. This will also install its dependency which includes the Npgsql ADO.NET Provider.

Install-Package Npgsql.EntityFramework

Step 4: Update web.config to tell the run time about our database connection and Entity Framework configuration.

a) Update the connection string like below. Remember to replace the information in the connection string to the values in your environment.

<connectionStrings>
<add name="MusicStoreEntities" 
connectionString="Server=[myserver];Database=MusicStore;
User Id=[myusername];Password=[mypassword];" providerName="Npgsql" />
</connectionStrings>

Note: Do not change the name of the connection string name. The name MusicStoreEntities matches the project’s DbContext class name. This is how Entity Framework figures out which connection string to use.

b) Update the entityFramework element as follow

<entityFramework>
<!--<defaultConnectionFactory
type="System.Data.Entity.Infrastructure.LocalDbConnectionFactory, EntityFramework">
<parameters>
<parameter value="v11.0" />
</parameters>
</defaultConnectionFactory>-->
<defaultConnectionFactory type="Npgsql.NpgsqlFactory, Npgsql" />
<providers>
<provider invariantName="System.Data.SqlClient"
type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
<provider invariantName="Npgsql"
type="Npgsql.NpgsqlServices, Npgsql.EntityFramework" />
</providers>
</entityFramework>

c) Add the system.data element as follow.

<system.data>
<DbProviderFactories>
<add name="Npgsql Data Provider" invariant="Npgsql"
support="FF" description=".Net Framework Data Provider for Postgresql"
type="Npgsql.NpgsqlFactory, Npgsql" />
</DbProviderFactories>
</system.data>

Step 5: Modify the MvcStoreEntities (DbContext) class to configure the table names the Entities mapped to.
PostgreSQL creates data tables in the public schema by default. This is different than the default Entity Framework convention. Override the OnModelCreating method to specify the new table name mapping.

protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
    modelBuilder.Entity().ToTable("Artist", "public");
    modelBuilder.Entity().ToTable("Album", "public");
    modelBuilder.Entity().ToTable("Cart", "public");
    modelBuilder.Entity().ToTable("Order", "public");
    modelBuilder.Entity().ToTable("OrderDetail", "public");
    modelBuilder.Entity().ToTable("Genre", "public");
}

Step 6: Disable migration
Comment out the following line in the global.asax.cs file. Otherwise you will get error since Npgsql does not support migration and database creation.

//System.Data.Entity.Database.SetInitializer(new MvcMusicStore.Models.SampleData());

Run

Compile and run the application again. You are now running MvcMusicStore with Entity Framework 6 against a postgreSQL database.

Download

A working demo is available at GitHub

Update (2015-12-20)

The original post was written almost two years ago. The MvcMusicStore was based on ASPNET MVC 3 and ASPNET Membership. There are some clarifications and update I would like to make:

  • The original MvcMusicStore uses two databases: one for the store, another for the membership. The blog post only covers converting the store database to PostgreSQL.

  • The original MvcMusicStore uses ASPNET Membership, you probably want to use the new ASPNET Identity instead. (“The ASP.NET Identity system is designed to replace the previous ASP.NET Membership and Simple Membership systems”).

  • I have created a storage provider for ASPNET Identity using PostgresSQL & EntityFramework 6 a few weeks ago. It supports the new columns in ASPNET Identity 2.0.  A working demo and source is available on
    GitHub: PostgreSQL.AspNet.Identity.EntityFramework

  • In the process of creating PostgreSQL.AspNet.Identity.EntityFramework, I found that Npgsql now has better Entity Framework 6 support. It can now auto create the database. The GitHub repo has a demo that take advantage of the auto database creation.

  • There is a new MvcMusicStore project on GitHub that’s targeting ASPNET 5 (vNext). It uses the ASPNET Identity and Entity Framework 7. Note: EntityFramework 7 currently only has provider for  SQL Server, SQLite, and InMemory provider. Npgsql also has an EF7 experimental provider.