Diving into Serverless: Azure Durable Functions in Node JS (+Troubleshooting)

Serverless Functions

Serverless might sound like your code runs without a server but it is not like that 😂. In Azure, Functions are tiny pieces of code that run based on a trigger or input and have an output. It is called serverless because you do not have full control over its underlying OS, network, storage or infrastructure.
You might compare it with other PaaS offerings like Azure App Service but there's a difference. Azure App Service will always be running and readily available. Also, you will be charged for the entire time the server is running irrespective of whether people are using it or not.
Azure Functions on the other hand are not always active. They only run when it is triggered based on some event and you are charged only for the time your code runs and that's it. It is great for doing atomic work e.g. renaming, moving or creating and filling files.
The problem with Azure Functions is that it is stateless, meaning it does not remember the inputs, outputs, variables, other function calls etc.

Stateful Functions

This is where the durable functions come in. These functions can store state in a storage account (by default in a file share) and can reuse them. There are several patterns in which you can build durable functions.

Generally speaking, they have 3 components:

1. Orchestrator Client: It is the primary function that is called. This activates the orchestrator function.

2. Orchestrator: The function that acts like the opera head and decides what are things to be done and in which order they should be done (kind of like Azure logic apps but logic apps are all about drag and drop whereas this is just code). This orchestrator function calls any number of activity functions asynchronously.

3. Activity: This is the atomic level function that actually performs some task.

Durable Function Patterns

There are multiple patterns in which you can develop the durable functions workflow.

- Function chaining: One function calls another function and so on.

- Fan-Out/Fan-In: A function calls multiple functions in an async fashion and waits for all of them to be completed and then it moves and narrows down to one function.

- Asynchronous API pattern: A function starts some other function and another function checks if the work is completed at regular intervals in an async fashion.

- Monitor pattern: A function calls and waits for an event to happen and when it occurs it triggers another function.

- Human interaction pattern: A function waits for a human to perform an action and in case there is a timeout it triggers some other function.

Demonstration

Go ahead and create a function app from the marketplace, the plan should be consumption-based.

Select a subscription, and a resource group (or create a new one) and set the function app name. Btw, the function app name has to be unique globally as this can be accessed from the internet. Select the stack as Node js, version as 20 LTS and OS as Windows and your preferred region.

Keep clicking next to keep the default values for the rest of the fields and finally create the function app.

Troubleshooting #1

Once you are done creating the function app, you might get a warning that "dynamic scaling is not enabled".

Spoilers! If you go ahead and create a durable function from the portal you will complicate things and will get more errors like this one below

What's happening? 🕵
Logically speaking, the functions that you will be creating under the function app are basically code, that is stored in files right? And it has to be stored somewhere right? This "somewhere" is a storage account that is not yet linked with the Azure function app and thus nothing will work because it cannot store the files. Also as I stated earlier, you need a storage account where the state of the durable functions will be stored for later use.

Solution ✅
What you need to do is create an Azure Storage Account in the same region as your function app and in the networking section make sure it is publicly accessible.

Go to data storage > file shares and create a file share

Copy and paste the file share name in Notepad or any other text editor as you will be needing this later.

Once you are done with this, go to Security + Networking > Access Keys and copy one of the connection strings

paste it in Notepad or any other text editor as you will be needing this later.

Now come back to your function app and go to settings > environment variables -- App Settings create new.

The first setting name would be "AzureWebJobsStorage" and the value would be the connection string of your storage account.

The next setting would be "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING" and the value would again be the connection string of your storage account.

The third and final setting would be "WEBSITE_CONTENTSHARE" and the value would be the file share name inside your storage account.

Now click on apply and confirm.

Congratulations! You have just set up a storage account that will allow dynamic scaling and let your durable functions store state. You can visit the overview page of your function app and the error is now gone

You can go to the default domain of your function app and see that the host is up and running!

After this I went ahead and created all the required durable functions and this was a mistake 😭 Please don't repeat the mistake and wait till you reach Troubleshooting #3 after that follow the steps to create durable functions

Troubleshooting #2

On various tutorials, you will be told to open the console and install "durable-functions" which is a required package and then simply go ahead with durable functions. Unfortunately, it did not work for me. Instead, I got a 500 internal server error.

Upon searching the logs from the invocations tab I got this

Pasting the error so that you can view this too.

Result: Failure Exception: Cannot read properties of undefined (reading 'extraInputs') Stack: 
TypeError: Cannot read properties of undefined (reading 'extraInputs') at
Object.getClient (C:\home\site\wwwroot\node_modules\durable-functions\lib\src\durableClient\getClient.js:11:40) at module.exports 
(C:\home\site\wwwroot\DurableFunctionsHttpStart\index.js:4:23) at t.InvocationModel.<anonymous> 
(C:\Program Files (x86)\SiteExtensions\Functions\4.36.0\workers\node\dist\src\worker-bundle.js:2:63453) at Generator.next (<anonymous>) 
at C:\Program Files (x86)\SiteExtensions\Functions\4.36.0\workers\node\dist\src\worker-bundle.js:2:61778 at new Promise (<anonymous>) at p 
(C:\Program Files (x86)\SiteExtensions\Functions\4.36.0\workers\node\dist\src\worker-bundle.js:2:61523) at t.InvocationModel.invokeFunction
(C:\Program Files (x86)\SiteExtensions\Functions\4.36.0\workers\node\dist\src\worker-bundle.js:2:63260) at y.<anonymous> 
(C:\Program Files (x86)\SiteExtensions\Functions\4.36.0\workers\node\dist\src\worker-bundle.js:2:39015) at Generator.next (<anonymous>)

What's happening? 🕵
Upon searching the internet I got to know that if I just do npm install durable-functions it will simply install the latest package which at the time of writing is 3.1.0. This version is however not compatible with Azure functions v4. So I had to downgrade it to 2.0.0 and that error was gone.

Solution ✅
Go to Development tools > console

_(if you have already installed the latest package just uninstall it using npm uninstall durable-functions)
_

now install the correct version of the package npm install durable-functions@2.0.0 and check the contents of package.json using cat package.json

Troubleshoot #3

Even after doing all these, I was still getting a 500 internal server error and I checked the logs again under the invocations

Pasting the error

Result: Failure Exception: Worker was unable to load function DurableFunctionsHttpStart: 
'Cannot find module 'moment' Require stack: - 
C:\home\site\wwwroot\node_modules\durable-functions\lib\src\task.js - 
C:\home\site\wwwroot\node_modules\durable-
functions\lib\src\durableorchestrationcontext.js - 
C:\home\site\wwwroot\node_modules\durable-functions\lib\src\orchestrator.js - 
C:\home\site\wwwroot\node_modules\durable-functions\lib\src\classes.js - 
C:\home\site\wwwroot\node_modules\durable-
functions\lib\src\index.js - 
C:\home\site\wwwroot\DurableFunctionsHttpStart\index.js - C:\Program Files 
(x86)\SiteExtensions\Functions\4.36.0\workers\node\dist\src\worker-bundle.js - C:\Program Files

What's happening? 🕵
As you can tell from the error log, a package 'moment' is missing and it is a required package.

Solution ✅
Simply open the console again and install 'moment' using npm install moment and verify the contents of package.json

And this error is gone too!

Congratulations! We are done with all the errors and now we are ready to create the durable functions

Creating Durable Functions

Start by creating the first component, the Orchestrator Client.

Type 'durable' in the search bar and select 'durable functions HTTP starter'

Rename the function as per your wish, the authorization level should be set to function and create it

Now we will create the orchestrator. Go back and in a similar way select "durable functions orchestrator" and rename it as per your wish.

Finally, we will create an activity. Go back and in a similar way select "durable functions activity" and rename it as per your wish

We are done!

Now let's understand the code in a bit more detail. See the code for every function by clicking on it and going to the Code + Test tab

1. Orchestrator Client (myClient)
This functions will trigger the Orchestrator upon getting a HTTP request. In line 5 client.startNew() will call the Orchestrator function and req.params.functionName will contain the name of the Orchestrator function which in my case is 'myOrch'. We will see how we will pass in the function name to the orchestrator client.

const df = require("durable-functions");

module.exports = async function (context, req) {
    const client = df.getClient(context);
    const instanceId = await client.startNew(req.params.functionName, undefined, req.body);

    context.log(`Started orchestration with ID = '${instanceId}'.`);

    return client.createCheckStatusResponse(context.bindingData.req, instanceId);
};

2. Orchestrator (myOrch)
This is the orchestrator function which will take care of calling activity functions. In line 7,8 and 9 "Hello" is the name of the activity function that will be called. (If your activity function has some different name then replace Hello with it) "Tokyo" in line 7 is the argument that will be passed on to 'Hello'. Similarly "Seattle" and "London" would be passed on to 'Hello' in lines 8 and 9. It will wait for all the three responses from 'Hello' and will push each response into the outputs[]. This is why we call this a 'Fan-out' (expanding) as 3 functions are called from 1 orchestrator function and then all the responses are collected by the orchestrator function and returned as a response thus 'Fan-in' (shrinking). Finally outputs[] would have ["Hello Tokyo!","Hello Seattle!","Hello London!"]

const df = require("durable-functions");

module.exports = df.orchestrator(function* (context) {
    const outputs = [];

    // Replace "Hello" with the name of your Durable Activity Function.
    outputs.push(yield context.df.callActivity("Hello", "Tokyo"));
    outputs.push(yield context.df.callActivity("Hello", "Seattle"));
    outputs.push(yield context.df.callActivity("Hello", "London"));

    // returns ["Hello Tokyo!", "Hello Seattle!", "Hello London!"]
    return outputs;
});

3. Activity (Hello)
This is the activity function. It will be called by the orchestrator function. It is very simple. It will just take the argument passed from the orchestrator and will format it and return. For example in line 7 of the orchestrator we were calling activity using outputs.push(yield context.df.callActivity("Hello", "Tokyo")); so we will get Hello Tokyo!

module.exports = async function (context) {
    return `Hello ${context.bindings.name}!`;
};

Let's see this in action!
First, open the orchestrator client (myClient) and go to Code + Test tab. Click on 'Get function URL' and copy the default(function key) URL.

https://test746234.azurewebsites.net/api/orchestrators/{functionName}?code=L9cALt2iP5V39M023c1I9lALDlpumzXurTUNzut4S9cTAzFujCPDOA%3D%3D

Now replace the {functionName} with your orchestrator function name which in this case is myOrch. So your URL should look like this

https://test746234.azurewebsites.net/api/orchestrators/myOrch?code=L9cALt2iP5V39M023c1I9lALDlpumzXurTUNzut4S9cTAzFujCPDOA%3D%3D

Now paste this in a browser and open it. You will see a bunch of URLs.

Now copy the URL of statusQueryGetUri and paste it in the browser and open it.

Voila! Congratulations! We got our expected results back! The results might or might not be in the same order as they are called as these called asynchronously.

Thank you for reading till the end. Hope this was useful for you :)