Azure Functions in C#

In this step by step tutorial, you’ll learn how to build an “information aggregator” by using Azure Functions to call the Stack Overflow API and send the results back to your Slack channel.

What Are Azure Functions?

Azure Functions are little “serverless” bits of code that react to various triggers to perform some resulting action – that’s it!

Serverless?

Yes that’s right. Azure Functions are serverless meaning that they are hosted in the minds of unicorns and are triggered via telepathy or in some cases smoke signals.

Trigger my function…

Of course that’s not quite technically correct, Azure Functions have to physically reside somewhere, but the beauty is that you don’t really have to worry about the provisioning of infrastructure to support them – it’s taken care of by Azure.

Triggers

As mentioned above, Azure Functions are activated by “triggers” so are therefore suited to performing integration type operations. Examples of the triggers Azure Functions can respond to are listed below:

• HTTPTrigger – Azure Function code executes when it receives a HTTP request
• TimerTrigger – Azure Function code executes on a schedule, ideal for batch or clean-up tasks.
• EventHubTrigger – Azure Function code executes in response to events delivered by an Azure Event Hub

There are more, so if you’re interested jump over the Azure Functions Overview for more details.

What Are We Building?

The User Story, (for those of you familiar with Agile Software Development), would be something like:

As Developer Support
I want to know when any new questions about our company are asked on Stack Overflow
So that I can proactively respond to the question and provide great customer service

So, we’re going to create an Azure Function that:

1. Is triggered by a timer (it will run once a day)
2. When triggered it will search Stack Overflow, (using its API), for questions on a certain topic
3. “Process” the results, (we’ll cache results in an Azure SQL DB)
4.  Send a message to a Slack Channel with the number of new questions it finds

The, (very!), high level schematic is shown below:

What are we waiting for let’s get started!

Set Up Slack

Slack is essentially group chat, that allows you to organise those chats in to channels, (think of a channel as a specific topic). It’s used a lot by remote, (and sometimes not so remote), teams to collaborate without the need for loads of emails… It’s pretty popular in the software development world so it’s why I’ve chosen it as our integration end point.

So go over to slack.com and sign up for a free account. Once logged in, click on “Create a channel”:

I’ve just created one called “azure_functions”:

Once created, highlight the channel then select the little “information” icon near the top right hand corner of the app, (it’s near the search box):

This should open the information window for your channel, then click “Add App”, (see above).

This will open the App Directory, look for the “Incoming WebHooks” app, or type: “incoming” in the search box. Once located, click “Install”:

This will give you some details on what functionality the Incoming WebHook app provides, click “Add” to continue:

On the set up page it will ask you which channel you want to post to when you call the WebHook, select whatever channel you want, and click “Add Incoming WebHooks integration”:

You should now be presented with the WebHook URL we’ll need to post messages to our Slack channel:

Copy this URL to a notepad somewhere as we’ll use it now to test the integration, once you have it copied, click “close”, (see above).

Then scroll down to the bottom of the page and click: “Save Settings”:

Test Our Slack WebHook

Before we go on to setting up our Azure Function, we’ll just test that the WebHook works, (nothing worse than thinking your code’s not working when the problem lies elsewhere).

I usually use Postman for this type of stuff, (https://www.getpostman.com/), and to be honest I would still recommend it if you do quite a bit of “web programming”.

At the same time I appreciate that you may not want to install that, so for a light-weight alternative navigate to: https://apitester.com/ and enter the details as shown below, (use your link from Slack though!):

• Set method to POST
• Paste your WebHook URL in
• Enter the following POST Data:

{"text":"testing 1, 2, 3..."}

• Click Test

All being well you’ll get a success message in API Tester and more importantly, you should see a message in your Slack channel:

Awesome – We’re ready to set up our Azure Function!

Create Azure Function Resource

Obviously you’ll need an account on Azure to continue, so check out https://azure.microsoft.com/en-us/free/ for the free pricing tier.

Note: You will still need to provide a Credit Card even for the free account, in much the same way Hotels and Hire Car companies require one…

However you can be quite comfortable that you’re not going to ring up any charges using Azure Functions as you get 1,000,000 Free Azure Function requests per month:

If you’re still worried about charges on Azure, I’ve written a previous post on how you can estimate and control your spend, I think we should be OK here though…

So login to the Azure Portal: https://portal.azure.com/ and click on “Create a Resource”:

In the search marketplace box type: “function app”:

Hit Enter, and you should get results similar to the following:

For some reason it appears twice!?

Click “Function App”, then on the following details screen, click “Create”:

This is where you enter the specific details for your Function App, see example below, (don’t worry I take you through what everything means):

OK here’s what it all means…

1. App Name

Well pretty obvious, but needs to be unique.

2. Subscription

Again, quite obvious, just select your subscription, (you may have more than one I guess).

3. Resource Group

Every “Resource” in Azure needs to get place in a “Resource Group”. Personally I’d rather use existing ones rather than creating new ones all the time, especially if it’s just for testing.

4.OS

An interesting one for me… For serverless functions, I think it’s weird that they ask for an OS, however this selection does determine some other options later, e.g. language…

5. Hosting PLan

Just keep this at Consumption, (pay as you go), which is fine for our test purposes.

6. Location

Another obvious one, but interestingly the list of locations varies depending on your OS selection in step 4!

7. RUntime Stack (Language)

Again options determined by OS Selection, we’ll be using good old C# so keep this as .NET Core.

8. Storage

Again, similar to Resource Groups, most Resources require storage space, and again I just prefer to reuse existing locations for testing. If you’re new to Azure though, as with Resource Groups, you may need to create some new storage.

When you’re done, click “Create” and Azure will go off an provision you Azure Function.

Note: You’ll get notified when it’s created:

Finally click on “Go to resource” and we’re ready to start coding!

Create a New Function

So technically we’ve not actually created a function, just the “function app” which will host the function and the code, so we’ll add one now.

In the Function App “tree” click on the cross beside the “Functions” node:

This will present you, (in the frame to the right), a quick start wizard where you get to choose your development environment:

To keep things simple, we’re going to write our code “In-portal”, i.e. in the web browser session. This option has the advantage of zero set up, but does lack some of the nice authoring features, (e.g. Intellisense), that VS Code or Visual Studio would have. It is fine for our demands though, so go ahead and click In-Portal, then “Continue”:

Next we get to choose which template we want for our function, or if you prefer which “trigger type” our function will respond to. The quick start wizard displays:

• Webhook + API (Basically a HTTPTrigger)
• Timer (Well a TimerTrigger)

You also have the option to look at more templates, but as we’re going to use the Timer, just go ahead and click that:

Then just click “create”:

This should create our TimerTrigger and take us into our development environment. So just before we start coding, let’s take a quick tour:

1. Our TimerTrigger function is created
2. We’re taken into the code for this function which is hosted in a file called run.csx
3. You may need to click to expand this, but this allows us to view all the files in out Function App
4. Our run.csx file, this is where we will do all our coding
5. function.json, this is our config file for our app, we can configure things like the timing interval here – more on that later
6. Again, you may need to click here to expand, but this portion of the environment gives us access to a console and the log output.

Ok now we’re ready to code!

Code Our Slack Integration

Looking back out our application flow, we’re going to start with our Slack integration:

So we’re going to:

• Create a new asynchronous function MakeSlackRequest
• MakeSlackRequest will take 1 string parameter: the message we want to send to Slack
• Use the “HttpClient” object to make a POST request to our Slack endpoint

The code looks like this:

public static async Task<string> MakeSlackRequest(string message)
{
  using (var client = new HttpClient())
  {
    var requestData = new StringContent("{'text':'" + message + "'}", Encoding.UTF8, "application/json");

    var response = await client.PostAsync($"https://hooks.slack.com/services/TCGUHDLTY/BP3JKEU03/<REDACTED>", requestData);

    var result = await response.Content.ReadAsStringAsync();

    return result;
  }
}

In terms of where this code is placed in your function, it should look something like this:

You’ll note in addition to our new method, we also made the following changes to the Run Method:

• Added an addition using directive

using System.Text;

• Changed the Run method signature so it’s asynchronous

public static async Task Run(TimerInfo myTimer, ILogger log)

• Made a call to the MakeSlackRequest method

await MakeSlackRequest("Test From Inside Azure Function!");

Click Save at the top of the screen and you should get a compilation successful output in the console.

Now Click Run at the top of the screen, you should get a “Status: 202 Accepted” message in the test explorer:

More importantly, if you move over Slack you should see our message appear in the channel!

Setting Out Schedule

While we can run this function manually, remember it’s also a Timer function, which means that it gets triggered on a schedule that we can specify.

To do this click on View files again to view or “file tree”, then click on function.json:

This json file allows us to configure the timing schedule using something called CRON expressions, which derive form the world of Unix Scheduling…. If you’re really interested you can read more about their configuration here.

Currently ours is set to run every five minutes, as denoted by this line in the file:

"schedule": "0 */5 * * * *"

Let’s up it to run at 9:30 AM every day

"schedule": "0 30 9 * * *"

Don’t forget to click “Save”

Note: You can “stop” your Azure Function from running by Clicking on the name of your Function App, (in my case stackslack), to bring up its properties, then just click the “stop” button to cease the function.

Consuming The Stackoverflow API

Ok so we move on to the next part of the Function and that’s consuming, (in this instance), the Stack Overflow API:

The documentation for the Stack Overflow, (well technically it’s the “Stack Exchange”), API is really comprehensive so if you want to delve into the details you’ll find what you’re looking for there. We’re just going to be leveraging from the simple search functionality, again detailed here.

Near the bottom of the page you can try it out, and it even formats the url you need to make a request to – what could be easier:

Click Run to get the example JSON back, just try to make sure the search criteria only brings back a few results, (I used the search term “Telstra”). I then coped the results into JSON Editor Online:

If you haven’t used json before and this looks confusing, then I’d recommend my video on deserialising json. Basically we have:

• An array called “items” that contains all out result objects
• Each object then has a number of key / value pairs, the only one we’re interested in is the “question_id”. We’ll store these in a db, so we can determine if we’ve had that result before, more on that later…

So with the following API request url in hand:

https://api.stackexchange.com/2.2/search?order=desc&sort=activity&intitle=rcs&site=stackoverflow

We can move to coding our function, first we need to add two references, (‘#r’), right at the top of our function code.

#r "Newtonsoft.Json"
#r "System.Data.SqlClient"

These reference tags work in a similar way to adding package references to a .csproj file, they basically reference packages that our code needs to run, but are not included by default.

In this case we are adding:

• Json.Net (from Newtonsoft), this allows us to deserialise our JSON payload from Stack Overflow
• System.Data.SqlClient: this gives us access to things this SqlConnection, SqlCommand etc. that allow us to connect in to our database, (see later section).

Next we need to add 3 new “using directives”:

using System.Net;
using Newtonsoft.Json;
using System.Data.SqlClient;

Note the difference between “using directives” and the “require” tags:

• #r: As mentioned these include our external packages for use
• using: Just allow us to directly reference object name spaces within a package. If we don’t have the necessary references, (#r), before we try to “use” them, we’ll get an error.

So the top of your function code should look like this:

#r "Newtonsoft.Json"
#r "System.Data.SqlClient"

using System;
using System.Net;
using System.Text;
using Newtonsoft.Json;
using System.Data.SqlClient;

Now we need to add our new method – add it below our MakeSlackRequest method:

public static async Task<string> MakeStackExchangeRequest()
{
    HttpClientHandler handler = new HttpClientHandler()
    {
        AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
    };
            
    using (var client = new HttpClient(handler))
    {
        var response = await client.GetAsync($"http://api.stackexchange.com/2.2/search?order=desc&sort=activity&intitle=rcs&site=stackoverflow");

        var result = await response.Content.ReadAsStringAsync();

        return result;
    }
}

The code is quite self explanatory, indeed it’s not that different from the MakeSlackRequest method, there are a couple of novelties though:

• This time we use “GetAsync” on the HttpClient object, (as opposed to PostAsync)
• We create a HttpClientHandler object that we use when we set our our HttpClient object. We need this as we have to specify that we want to decompress the results from our API call. Stack Exchange returns its json payload compressed in zip format – this caught me out when I first used it. The documentation explains this further.

Finally we need to make a call to our new method back up in our “Run” method:

var result = await MakeStackExchangeRequest();

So your code should now look like this:

Click Save and Run, and all being well you should get the same exact result as before… Mmm so we’re not quite there yet, we need to “process” the Stack Overflow response in the following way:

• Deserialise the json string response (so we have an object)
• Determine if we’ve had the same question returned before:
  • If not, we add them to our Database and increment out “new questions counter”
  • If so, we ignore

So before we move onto our 3rd and final method, we need to set up a SQL Database on Azure.

Creating an Azure SQL Server

Warning: Running a SQL Server will cost money. With the settings I’m going to show you however, you should only be charged around $5USD per month. You can of course delete it after you’ve played with it so may cost you even less…

So over in Azure click: Create Resource:

Type “SQL Database” into the search box:

Hit Enter, and you should see something like this, (click the SQL Database option):

You should get the details screen, click create:

You now have to supply some details to create your SQL Server + Database:

1. Select your subscription
2. Select your resource group, (I’d put it in the same one as the function)
3. Give the Database a name, I’ve called mine “Aggregator”
4. Select the Server where the Database will reside, I don’t have one so click “Create New”

Fill out the necessary details, just make sure you keep a record of the Server Admin login and password. You’ll also want to make sure you select the most appropriate region, (it should be the same as the Function).

When you’re happy click OK, this will bring you back to the Create SQL Database options screen:

It’s really important that you click “Configure Database” as you’ll want to change the default options they give you.

You’ll see something like this, click on “Looking for basic, standard, premium?”

Ensure “Basic” is selected on the next screen:

1. Basic Option Selected
2. Estimated Monthly cost (here it’s in Australian dollars – convert as required)
3. Click Apply when done

This will bring you back to the Create SQL Database options screen, you’ll see the “Compute + storage” setting for your server has been updated. When you’re happy click “Next: Networking >”

On the Networking Options Tab, select the following options:

This will allow us to access the SQL Server locally from our desktop using SQL Enterprise Manager, it’ll also allow our Function App to connect and use the DB.

When you’re happy click: “Next: Additional settings >”

Leave the defaults here, then click: “Next: Tags >”

Leave the defaults here, then click: “Next Review + Create >”

Ensure you’re happy with everything, if so, click “Create”

Azure will go off and deploy the SQL Server…

When done, Azure will notify you:

You can click Go to resource, and when there, locate the “Connection strings” settings, and click:

Ensure the ADO.NET Tab is selected and take a note, (or better still copy off somewhere), the connection string for our database.

Adding a Table to our Database

Now the final bit of set up on our SQL Server is to create a simple table where we’ll store a list of the Question IDs that we’ve previously received.

For this you’ll need SQL Enterprise Manager which is free and available from Microsoft here. Unfortunately it’s only available on Windows…

Once installed start Management Studio and enter:

• Server Name (you can get this from the connection string you just copied)
• Login: this is the admin account you set up when you created the server
• Password: ditto

As an example my connection string was:

I’ve highlighted the server name you’ll need in Management studio.

Note: if for some reason the port is not 1433, you should also include that part of the string up to but not including the semi-colon. Port 1433 is the default SQL Server port so you don’t need to supply it if that’s what’s being used.

So your Management Studio Login should look like:

Click Connect and you’ll be taken into Management studio. Expand the Databases folder, then the “Aggregator” database, (or whatever you called yours), and you should see:

Select the Database and click “New Query” to open a SQL Query window:

1. Selected Database
2. New Query Button
3. Query Window
4. When run queries, check that the database you want is named here

To create our database, use copy and paste the SQL below into your query window:

USE [Aggregator]
GO

/****** Object:  Table [dbo].[questions]    Script Date: 20/10/2019 7:40:07 PM ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [dbo].[questions](
	[id] [int] IDENTITY(1,1) NOT NULL,
	[QuestionID] [nvarchar](20) NOT NULL,
	[CreatedDate] [date] NOT NULL,
 CONSTRAINT [PK_questions] PRIMARY KEY CLUSTERED 
(
	[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

You’ll notice the first command is:

USE [Aggregator]
GO

This just ensures that we execute the code against the correct DB, (if your database has a different name, replace that here).

Otherwise the SQL just creates a table called “questions” with the following columns:

• id – integer – primary key
• QuestionID – nvarchar(20)
• CreateDate – Date

These should all be relatively self explanatory. To execute the SQL, hit F5 or click the “! Execute” button in Management Studio, either way you should see:

Expand the “Tables” folder under your Database and you should see your newly created table, if not right click the Tables folder and select “Refresh” and try again:

Ok so we’re ready to move onto the last part of our app.

Create Our Processing Method

So just before we  write any code I’m going to create an Environment Variable to hold the password of our database user account. In fact we could create environment variables for the account name, the server name etc, but for brevity I’m just doing the password.

So back over in our Azure Function, click on the same of your app, and you should see the Configure Features screen:

Click on “Configuration” (#2).

We are going to add an environment variable to hold our SQL User password – see #1. You’ll notice that there is a section dedicated to Connection Strings, but as I’m not going to use Entity Framework we can’t make use of this section…

Add Application Setting

Click on  “New application setting”

On the resulting screen, give your setting, (Environment Variable), a name and a value. As I’m storing the password of my Admin account I’ve redacted the value:

Click ok to add the Setting:

You’ll see the setting has been “added”, and that the value is hidden, (which is good as it our password!). Make sure to click “Save”, (#1), at the top of the screen to make these changes permanent. I’ve been caught out before by not doing this…

Our Final Method

So just a word of warning on this last method – there is WAY too much going on inside it, and it really should be split into separate concerns. For brevity though I’ve just included it as-is.

So the code for our final method, (add it under the other 2 we have created is):

public static async Task<int> ProcessQuestions(string jsonString, ILogger log)
{
    int NewQuestionCount  = 0;
    var jsonOb = JsonConvert.DeserializeObject<dynamic>(jsonString);

    log.LogInformation($"-> ProcessQuestions at {DateTime.Now} - {jsonOb.items.Count} items to process.");
    var SqlUserPassword = Environment.GetEnvironmentVariable("SQLUserPassword");
    
    var connectionString = $"Server=tcp:binarythistlesqlserver.database.windows.net,1433;Initial Catalog=Aggregator;Persist Security Info=False;User ID=questiondbuser;Password={SqlUserPassword};MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;";
    
    using (SqlConnection conn = new SqlConnection(connectionString))
    {
        await conn.OpenAsync();
        log.LogInformation($"--> SQL Connection Opened {DateTime.Now}");
        foreach (var question in jsonOb.items)
        {
            var CheckQuestionIsThere = $"select * from questions where QuestionID = '{question.question_id}';"; 
            using (SqlCommand cmd = new SqlCommand(CheckQuestionIsThere, conn))
            {
                var reader = await cmd.ExecuteReaderAsync();
                if(!reader.HasRows)
                {
                    reader.Close();
                    log.LogInformation($"Question: {question.question_id} does not exit - INSERT");
                    var InsertCommand = $"insert into questions (QuestionID, CreatedDate) values ('{question.question_id}', getdate());";
                    using(SqlCommand cmdInsert = new SqlCommand(InsertCommand, conn))
                    {
                        await cmdInsert.ExecuteNonQueryAsync();
                    }
                    NewQuestionCount++;
                }
                reader.Close();
            }  
        }
        conn.Close(); 
    }
    return NewQuestionCount;
}

We’ll step through the code below, but just to round off, update your Run method so it calls our new processing method: ProcessQuestions,  as well as updating the call to MakeSlackRequest so we pass a more meaningful message.

public static async Task Run(TimerInfo myTimer, ILogger log)
{
    var result = await MakeStackExchangeRequest();

    var NewQuestions = await ProcessQuestions(result, log);

    await MakeSlackRequest($"You have {NewQuestions} new questions on Slackoverflow");

    log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
}

So just to step through our ProcessQuestions method:

• Line 51: Deserialise the JSON string passed in from the Stack Overflow api
• Line 54: Retrieve the SQL User Password Environment Variable value we just set
• Lines 56-60: Set our connection string, (remember we copied this previously). Note that we inject the password environment variable in here
• Line 62: Set up a SqlConnection to our DB
• Line 64: Open the connection
• Line 66: Loop round all the questions in the the JSON Object we deserialised
• Line 68: Create the SQL to query our DB to see if the current question has been stored
• Line 69: Create a SqlCommand and execute our SQL
• Line 71: Using a SqlDataReader we get our result set
• Line 72: If we have no results then this is a new question
• Line 76: Create the SQL to insert the question into our DB
• Line 78: Create a SqlCommand to execute the Insert SQL
• Line 80: Execute our command
• Line 82: Increment our NewQuestionCount
• Line 87: Close our connection
• Line 89: Return our new question count – this will be sent to Slack

Running the function, (or waiting for the schedule to execute), you’ll get something like:

Subsequent runs will yield the additional new questions, (or Zero), if there are none:

Wrapping Up

That’s it folks! We covered a lot in the lesson, but there are a few changes I’d make going forward:

• Split out the processing method
• Find a more elegant way to process the questions, it’s kind of brute-force at the moment and would get slower as time went on
• Use Entity Framework to access the DB. This is the first time in a long while that I’ve written code to directly access the DB, and I realised how much I dislike it
• If you’re going to be developing Azure Functions a lot, I’d use either VS Code or Visual Studio as your development environment. The In-Portal environment is fine for testing but understandably its a bit limited/

If you’ve any questions, please leave them below.