Authenticating to a REST API from c#

Foreword

This tutorial follows on directly from my previous post: Consuming a REST API from c#. You may need to start there if you want to pick up the thread of the coding examples below.

A full video of this tutorial can be found here.

User Story

As a developer
I want to call a REST API that requires authentication
So that I can consume its “protected” resources

Acceptance Criteria

By the end of this tutorial you should be able to:

• Authenticate to a REST API (using a c# Windows app), using Basic Authentication
• Authenticate to a REST API (using a c# Windows app), using NTLM, (Windows), Authentication

Ingredients

For this tutorial you will need the following, (or something similar):

• Windows PC (I’m running Windows 10)
• Visual Studio (I’m using the 2017 Community Edition – which is free)
• Web Browser (I’m using Firefox and Edge)
• Access to a REST API that uses “Basic” Authentication
  • Don’t worry if you don’t we spin one up as part of this tutorial
• Access to a REST API that uses “NTLM” Authentication
  • Don’t worry if you don’t we spin one up as part of this tutorial
• ~40 Minutes of your time

My Lab Set Up

Having listed the ingredients above, here’s a simple schematic of my Lab Set up for this tutorial:

You’ll notice that I’m using the Firefox plugin called “Live HTTP Headers” this isn’t mandatory for the tutorial but it’s a useful little tool that allows you to see the HTTP Headers sent and received by Firefox. Kind of helps demystify the whole thing.

You’ll also notice that I have a Linux VM running “Jira”. This is absolutely not necessary but I included it as: a) I’ll be using it in a later tutorial, and b) it’s good to try this stuff against a proper production API.

What’s “Jira”?

Jira is a web-based tool from a company called Atlassian, it’s actually quite difficult to describe what it is, but in essence it’s an “Issue Tracker”. Think of it like a big “To Do List” repository for organsations. It’s used a lot by Technical help desks, (to raise fault / support / defect tickets), and software development teams to capture requirements and plan / track the release, (as well as bugs).

I actually use it to plan the tutorials & Youtube videos I’m going to produce, I even knew a guy who used it to plan his wedding!

I’ve included it as is provides a actual REST API interface that I can test against. You can safely ignore it if you want to though!

User Interface Design

Again using Balsamiq I created the following wire frame to help clarify my thinking on what the app is going to do. You’ll notice I’ve provided for:

• username and
• password credentials

which will be passed to our REST Class on an as needed basis. I won’t be covering how to securely store credentials persistently as part of this tutorial. If that’s something of interest though drop me a line!

There’s also 2 radio button groups:

1. Authentication Type
2. Technique

Which I thought may be necessary at the start of this project to dynamically switch between, (surprise-surprise!), Authentication Type and Technique. As it transpires they’re probably a bit superfluous, but I’ve left them in for now.

Class Diagram

We’re basically just extending upon the Class from the previous post, the updated diagram is shown below:

I’ve added 2 new enumerations:

• authenticationType
• authenticationTechnique

Think they’re self explanatory, so we’ll move on…

Our constructor and makeRequest method have not changed, at least from a calling perspective, there are obviously some internal changes which is what we’re going to code up below.

We have some new attributes, again I think self-explanatory:

• authType
• authTech
• userName
• userPassword

Is that a good constructor?

You’ll notice, (when it comes to coding below), that the constructor doesn’t take any arguments and that we set all the class attributes, (e.g. authType etc.), following construction.

There is no real reason, (that I can think of), why this really needs to be the case, so obviously feel free to change that!

Coding – Part 1: Self Rolled “Basic” Authentication

Ok ok! We’re here finally at the coding – I’m guessing the bit you’re all interested in! Again to pick up this coding thread you should start here.

I’m going to assume that you’ve updated the UI with the following elements:

So basically your UI will look like the following:

vp-6-rest-auth-form-ui.png

Now we’re going to update our RestClient class as follows, (new code from the previous tutorial is in blue):

[code language="csharp"]
using System;
using System.IO;
using System.Net;


namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    public enum authenticationType
    {
        Basic,
        NTLM
    }

    public enum autheticationTechnique
    {
        RollYourOwn,
        NetworkCredential
    }

    class RestClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }
        public authenticationType authType { get; set; }
        public autheticationTechnique authTech { get; set; }
        public string userName { get; set; }
        public string userPassword { get; set; }


        public RestClient()
        {
            endPoint = string.Empty;
            httpMethod = httpVerb.GET;
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();

            String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
            request.Headers.Add("Authorization", authType.ToString() + " " + authHeaer);

            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
        }

    }

[code]

The really only interesting thing here is the addition of the String “authHeader“, and we attach it to our HttpWebRequest object – “request“.

What is this header string?

This “self-rolled” header string supports “Basic” Authentication – see the section below. We construct it so that it follows RFC2617 – The HTTP Basic Authentication scheme and pass it with our initial request so that we are authenticated through, (assuming the credentials are correct).

As per RFC2617, the string should be constructed using the following method:

1. Obtain the user id and password (this will be supplied from our text boxes)
2. Construct the “user-pass” be concatenating the user id, a single colon (“:”) character and the password. E.g. “john:abc123”
3. Encode the “user-pass” into an octet sequence
4. Encode the the octet sequence using Base64 into a sequence of US-ASCII characters

The spec goes into more detail, you can read it here.

All we need to do now is update out “Click” button event to pass the user name and password, (we can also set the Auth Type and Technique values here although they are not really used at this stage).

The updated click event handler will look something like this, (new code is in blue):

[code language="csharp"]
private void cmdGO_Click(object sender, EventArgs e)
        {
            RestClient rClient = new RestClient();
            rClient.endPoint = txtRequestURI.Text;
            rClient.authTech = autheticationTechnique.RollYourOwn;
            rClient.authType = authenticationType.Basic;
            rClient.userName = txtUserName.Text;
            rClient.userPassword = txtPassword.Text;

            debugOutput("Rest Client Created");

            string strResponse = string.Empty;

            strResponse = rClient.makeRequest();

            debugOutput(strResponse);
        }
[code]

If all is successful then this should allow you to use a REST API that uses Basic Authentication.

As an example I called my Jira REST API, (I wanted to retrieve the JSON for a single issue with id: wp-1), with the following request:

• http://192.168.0.16:8080/rest/api/2/issue/vp-1

Note: I did not pass the any credentials over, and so I received the following response:

You can see that Jira returned back a 401 response, (unauthorised). More about that later…

Trying again but this time supplying the correct credentials yields success and some JSON related to the issue:

How Basic Authentication Works

There are plenty of resources on the internet that detail how Basic Authentication works, (I have placed a few links in the references section below), but thought I’d give a brief overview here as its relevant to a point I want to make…

The following sequence diagram illustrates a typical request response scenario when the initial request does not have the necessary authentication header:

You’ll see that the 2nd request supplies and Authorisation Header as per RFC2617 and we are returned a successful 200 OK response.

This “challenge/response” scenario is important when we come to look at the NetworkCredential class as it relies on this to supply the credentials. I.e. it won’t initially supply the credentials until it receives the 1st 401 Unauthorized response along with the Authentication type, in this case “Basic”.

If it, (the NetworkCredential Class), does not receive this response, then it may not work correctly. This is actually the scenario I encountered with the Jira api, that being when I used the NetworkCredetial class with the Jira API I could not get it to work. Had I read the Jira API documentation I’d have realised that Atlassian, (the company that make Jira), did not implement their API that way, observe:

Why do you care?

Good question! The reason I have made a bit of a diversion is 2 fold:

1. When we come to the NetworkCredential Class below it is useful background
2. In researching this tutorial I have found many people having a similar issue with the NetworkCredential class, (when in fact it’s not really the NetworkCredential class to blame, but API vendors that haven’t implement the Basic Authentication standard correctly). Basically, (no pun intended), I don’t want you to waste the same amount of time I did.

TIP

As boring as it sounds, READ your API’s documentation. I may save you a lot of time!

Set Up Your own API?

The thought occurred to me that perhaps some people don’t have access to a test api? Or in my case I wanted to spin up a really stock-standard api so I could do more testing and play with different types of authentication.

I was also confused why the Jira API did not work with NetworkCredential class so wanted to test with another Basic Authentication API I set up myself – as mentioned above had I read the API documentation my confusion would have dissipated!

I therefore decided to insert this section into the tutorial so I:

• Could Demonstrate using the NetworkCredential Class
• Play with NTLM, (Windows), authenticaton

How Many Types of Authentication?

There are a plethora of different authentication mechanisms in use, so would have been difficult for me to cover them all. I chose Basic and NTLM in this case.

Set Up IIS

Ok since we’re running on Windows and as per my lab set up above, we’re going to run a REST API using the Microsoft ASP .NET MVC framework, (don’t worry if that doesn’t make sense), which will require that we have IIS installed.

What About IIS Express?

We could use the embedded version of IIS, (IIS Express), that starts when you run your ASP.NET project from within Visual Studio, but I’ve found that doesn’t give you the same power and flexibility when it comes to configuring the authentication options.

Step 1 – Open Control Panel

• Click “Programs”

Step 2 – Click: Turn Windows features on or off

Step 3 – Select the IIS Features You Want, (or need!)

You’ll notice I’ve placed arrows against the primary artifacts you’ll need to run a REST API with authentication…

Click OK and assuming all’s well the required components should be installed.

You can check that IIS is working ok by firing up a browser and navigating to “localhost”: http://localhost, you should see something like this:

Spin Up a Template API

OK cool so IIS is up an running, (hopefully), so all we need to do now is spin up a REST API… Here we go!

Step 1 : New Project

• Open Visual Studio and select New Project
• Expand the Visual c# node
• Select “Web”
• And Select “ASP .NET Web Application (.NET Framework)
• Give it a name and hit OK

Step 2: Select the “Web API” Template

• Select the “Web API” Template
• Ensure both “MVC” and “Web API” tick boxes are checked
• DON’T Click OK YET! (We need to set authentication type)

Step 3: Click “Change Authentication”

Step 4: Select Windows Authentication

• Select Windows Authentication (We can change this in IIS Manager later)
• Click OK

Click OK once more, (to close out the ASP. NET Template Selection Window)

Your API Project will be created!

Step 5 – Edit the “Index” Method of the “Values” Controller

Don’t worry if you don’t underand what this is, we can cover it in another tutorial.

• In the Solution Explorer, Expand the “Controllers” folder
• Double Click the “ValuesController” Class file – the file should open in the editor
• Optional: Change the “Value 1” and “Value 2” values in the 1st method to something else.
• Edit the “Authorize” Directive at the top of the class to include a user and specified account. This will take the form: domain\username.

If you are running outside of a windows Domain, (if you’re running a stand alone PC at home this will probably be the case), then the “domain” value is just your PC name.

Your Edited code should look like this:

Note my PC name is: DESKTOP-H9580B0

My Account is: lesja

Step 6 – Build

I usually just to a build at this stage to make sure there are no errors:

• Build Menu->Build Solution

Deploy and Configure Authentication

Ok now we have our test API build – we can now deploy it to IIS:

Step 1: Right Click Project and Publish

• Locate your “project” in solution explorer, (it should be the 2nd node under the main solution)
• Right Click and select “Publish…”

Step 2: Select IIS, FTP

• Select the “IIS, FTP, etc” Option and click “Publish”

Step 3: Select File System Method

• Select “File System” as the Publish Method in the resulting dialog box
• Click on the “…” ellipsis for the target location and locate the folder you want to publish to.

Step 4 – Select Publish Destination

• Create a New Folder if Required
• Note that using this method the location you select should be on the same server where IIS is installed, (as we’re doing all this on our PC it’s fine)
• Select the folder and Click OK

• Click Next

• Expand “File Publish Options”
• Select “Delete all existing files prior to publish”
• Click “Save”

• The Project should be published to your chosen destination if all is well.
• You should see something similar in your “Output” window:

Step 5: Configure Application in IIS Manager

• Type “IIS” into Cortana (if using windows 10) or just “Search Programs and Files” in older versions of Windows, (from memory you’ll have to click the start menu)

• Internet Information Services (IIS) Manager should be found. (if it’s not go back to the step where we installed IIS and ensure that you have the “Management Tools” box selected)

• Once IIS Manager has started, expand the tree in the left hand plane until you see the “Default Web Site” node, (it looks like a little globe).
• Right Click and Select “Add Application…”

• Give the application an “Alias”. I usually use the same name as the Project
• Click the “…” ellipsis for the “Physical Path” destination and locate the folder location where you previously published the app, (see steps above).
• Click OK

• Expand, (if you haven’t already), the “Default Web Site” node and you’ll see your configured IIS “Application”

OK we’re nearly there! All this hard work now means that we can configure the “Authentication” type for our app.

Step 6: Select The Applications Authentication Type

Note: You’ll repeat this step multiple times as you switch between the 2 authentication types covered in this tutorial:

• Baisc
• NTLM (windows)

In IIS Manager, double-click on the “Authentication” Icon:

1. Click on “Basic Authentication”
2. Click Enable

You should see that “Basic Authentication” is enabled for this application. You may also get a warning about SSL.

The point about SSL is extremely valid, if you’re not using SSL, (essentially “https”), then the credentials you’re sending over the wire are not secure…

Step 7: Test Site Manually in a Browser First

To just test that the basic site works, go to your favorite browser and navigate to the applications home page, this will be:

http://localhost/

E.g. In my case:

http://localhost/tutorialRest

We can now browse to our, (secured), API, which as you may remember formed part of the “Values Controller”.

The URL you’ll need is:

http://localhost//api/values

So in my case this would be:

• http://localhost/tutorialRest/api/values

Assuming all is correct you’ll get a 401 challenge from the server:

As mentioned right at the start, (seems like a long time ago now right!), I installed a plugin to Firefox called “Live HTTP Headers”, this allows us to look at, (surprise surprise), the http headers sent to and from IIS. Here’s the output for that first request / response as shown above:

You’ll notice that at this stage we hadn’t sent the credentials…

Now following entering, (the correct!), credentials we get the following http header traffic:

And finally we get the following output in our Browser:

Note: We can repeat step 6 – Select Application Authentication Type but this time for NTLM, (windows), authentication, (remembering to “disable” Basic Authentication first). If you do this and you interrogate the http header you’ll see something like the following in the first response leg:

Step 8 – Test with Our C# Client

Ok, before we begin ensure that the test API has been set to use “Basic Authentication” once again!

• Fire up our c# Rest Window client and make a first request to the test api, (don’t supply any credentials you should see:

You’ll see that we correctly get a 401 response from the server.

• Now let’s enter our correct credentials and try again:

And we have success!

We’ve basically called our test API, (that’s using Basic Authentication), from our c# client by constructing our own header.

What happens with we switch the authentication type to NTLM, (windows), authentication and try again with our client?

yeah that’s right we get a 401 response because we’re still using a self-rolled authentication header that’s used only by Basic Authentication.

What we need now is the “NetWorkCredential” Class!

Coding – Part 2: NetworkCredential

In our previous example we constructed our authentication header on our own, but we could have used a much simpler method – the NetworkCredential class, (assuming Basic Authentication is implemented as we expect).

To use the NetworkCredential class with both Basic and NTLM authentication it’s pretty easy… Let’s first update the code in our Form Click event so it sets the correct attributes on our RestClient object, (new code is in blue):

[code language="csharp"]
private void cmdGO_Click(object sender, EventArgs e)
        {
            RestClient rClient = new RestClient();
            rClient.endPoint = txtRequestURI.Text;
            
            if(rdoBasicAuth.Checked)
            {
                rClient.authType = authenticationType.Basic;
                debugOutput("authenticationType.Basic");
            }
            else
            {
                rClient.authType = authenticationType.NTLM;
                debugOutput("authenticationType.NTLM");
            }

            if(rdoRollOwn.Checked)
            {
                rClient.authTech = autheticationTechnique.RollYourOwn;
                debugOutput("autheticationTechnique.RollYourOwn;");
            }
            else
            {
                rClient.authTech = autheticationTechnique.NetworkCredential;
                debugOutput("autheticationTechnique.NetworkCredential");
            }


            rClient.userName = txtUserName.Text;
            rClient.userPassword = txtPassword.Text;

            debugOutput("REst Client Created");

            string strResponse = string.Empty;

            strResponse = rClient.makeRequest();

            debugOutput(strResponse);

[code]

Next we just add a simple if clause to our RestClient class:

• If we want to use the “Self-Rolled” technique, then we default to basic authentication
• Else if we want to use NetworkCredential Class then we let it take care of the Authentication Type (Basic or NTLM) – that’s the power of using it!

Note when we come on to the CredentialCache we will use the attribute “authType”.

[code language="csharp"]
using System;
using System.IO;
using System.Net;


namespace restClient
{
    public enum httpVerb
    {
        GET,
        POST,
        PUT,
        DELETE
    }

    public enum authenticationType
    {
        Basic,
        NTLM
    }

    public enum autheticationTechnique
    {
        RollYourOwn,
        NetworkCredential
    }

    class RestClient
    {
        public string endPoint { get; set; }
        public httpVerb httpMethod { get; set; }
        public authenticationType authType { get; set; }
        public autheticationTechnique authTech { get; set; }
        public string userName { get; set; }
        public string userPassword { get; set; }


        public RestClient()
        {
            endPoint = string.Empty;
            httpMethod = httpVerb.GET;
        }

        public string makeRequest()
        {
            string strResponseValue = string.Empty;

            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);

            request.Method = httpMethod.ToString();

            if (authTech == autheticationTechnique.RollYourOwn)
            {
                //We'll only do Basic Authentication if we roll our own
                String authHeaer = System.Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(userName + ":" + userPassword));
                request.Headers.Add("Authorization", "Basic " + authHeaer);
            }
            else
            {
                NetworkCredential netCred = new NetworkCredential(userName, userPassword);
                request.Credentials = netCred;
            }
            HttpWebResponse response = null;

            try 
            {
                response = (HttpWebResponse)request.GetResponse();
                

                //Proecess the resppnse stream... (could be JSON, XML or HTML etc..._

                using (Stream responseStream = response.GetResponseStream())
                {
                    if(responseStream != null)
                    {
                        using (StreamReader reader = new StreamReader(responseStream))
                        {
                            strResponseValue = reader.ReadToEnd();
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                strResponseValue = "{\"errorMessages\":[\"" + ex.Message.ToString() + "\"],\"errors\":{}}";
            }
            finally
            {
                if(response != null)
                {
                    ((IDisposable)response).Dispose();
                }
            }

            return strResponseValue;
        }

    }

[code]

The only real code of interest is the NetworkCredential class its self:

• We simply create a new instance of one, passing the username and password in the constructor
• We then set the “credentials” attribute of our request object to our newly created NetworkCredential object

Lets’ test with our client!

Test 1: API Set to Basic – Client Using Self-Rolled Header

Result: Pass!

Test 2: API Set to Basic – Client Using NetworkCredential Class

Result: Pass!

Test 3: API Set to Windows (NTLM) – Client Using Self Rolled Header

Result: Fail! – Technically this is not a fail as the software is behaving exactly as we expect, but just laboring the point that our self-rolled header does not allow us to authenticate when the api is using Windows authentication.

Test 4: API Set to Windows (NTLM) – Client Using NetworkCredential Class

Result: Pass! We have for the first time, authenticated to a Windows, (NTLM), protected API using the NetworkCredential class.

You can see the power and simplicity of this approach, but let’s try one more test…

Test 5: JIRA API Set to “Basic” – Client Using NetworkCredential Class

As mentioned previously when using our Jira API – the self rolled approach worked fine, let’s try with the NetworkCredential Class:

Result: Fail! – I actually would call this a fail this time. I think this should work. As mentioned above though, Atlassian did not implement their API according to the true nature of Basic Authentication.

Test 6: JIRA API Set to “Basic” – Client Using Self Rolled Header

Result: Pass! We knew this already though!

CredentialCache

I was going to write about the CredentialCache but the MSDN entry on it says everything I was going to say anyway! It can be found HERE.

In short:

• Used to store multiple credentials
• You assign it to HttpWebRequest Credentials attribute the same way we assigned the NetWorkCredential
• When Adding multiple credentials you can assign the “Type” of authenticaton, e.g. Basic, NTLM etc, see below:

References

The unique reference for this tutorial is: VP-6

• Balsamiq Mock Ups: Balsamiq.com

The Video