Programming the Cloud with Pulumi and Typescript

This article was originally posted on my blog.

Server side development and the managing of the infrastructure has gone a long way in the recent years. From scalable virtual machines easy to manage to containerisation with Docker and even serverless functions with Amazon Lambda. But all of this advancements still require you to change your mindset, from programming your application to configuring the infrastructure.

Even if it's easier than ever before it's still not frictionless, and when you just want to try a quick idea or work on a weekend project the last thing that you want is friction. That's why I've been so curious about Pulumi. I've been following Joe Duffy for a while because of his interesting work in programming languages and his most recent work on Programming the Cloud was more than promising.

Pulumi provides a cloud native programming model to create containers, serverless functions, and cloud infrastructure, with all the benefits of immutable infrastructure, and real programming languages.

This is great. I'm a huge believer that some configuration should be done with real programming languages, Pulumi brings this idea to the cloud infrastructure.

So this weekend I decided to give a try to an idea that I've been wanting to implement for a while, and using Pulumi seemed the quickest way to work on it.

Serverless

For this idea I just need a couple of endpoints and a small database. I've recently worked with Amazon Lambda and DynamoDB in my work and I knew that Amazon provided a free tier of those services, so it seemed ideal for a weekend project.

Even if serverless has many advantages you still need to setup quite a bit of stuff. Lambda functions, dynamo tables, user roles, policies, API gateways... it's not something you can just learn in 5 minutes. And luckly for me I had a couple of great mentors that helped me quickstart it at work, but still I'm not confident enough on all that setup to do it myself.

Pulumi

Here's where Pulumi comes in. After installing the CLI and setting up your AWS credentials (this is the only part where you need to deal with AWS directly, and you will get tired of it just creating an account 😂 ) you can create a project and start working on your application.

Pulumi programming model treats infrastructure resources as just objects in your code. So if you want an API endpoint you just create one in your code!

const endpoint = new cloud.API("langs");

That's it! That's everything you have to do to have a working API Gateway with Lambda. The best thing is that you can treat that object as the router of your webservice, like in any other web framework.

endpoint.get("/test", async (req, res) => {
    ...
}

With this you are configuring a "/test" endpoint in the API Gateway that you run the code that you add in the closure from a lambda function.

And this is one other advantage of Pulumi. Because this code creates the infrastructure and has your application logic, you can easily communicate between different parts of your infrastructure just by accessing their variables and even capturing them in closures!

This means that you can easily access DynamoDB tables from Lambdas just by calling a method on the table objects from inside the endpoint closure!

const endpoint = new cloud.API("langs");
const table = new cloud.Table("projectsTable", "project");
endpoint.get("/test", (req, res) => {
    table.get({ 'project': "1" }).then(value => {
        res.status(200).json(value)
    })
})

Pulumi pulls thus off by serialising the objects captured in the lambda closure and passing them transparently for you at runtime.

It is great that you don't have to worry about setting up the services you need or even how to pass data between them. You just focus on the product you want to create.

Powerful infrastructure configuration

By now you have seen how cool being able to access infrastructure just by creating object is, specially for somebody like me that is not an expert in devops. But for the experts in setting up infrastructure Pulumi is also really awesome.

First of all, you have all the power of a real programming language at your disposal. Conditionals, loops, functions... anything.

Having all this power brings the reusing of infrastructure to the next level. You can easily create and distribute components, at the end of the day is just a library!

To see how amazing this is I have a real world example. One of the things I'm still not convinced about Lambda is the cold starts. Lambda functions instances can be shut down and reused after some minutes of inactivity. This can add quite a bit of latency specially if you're hitting those endpoints in a mobile network. Usually it's not something that you need to worry about but sometimes is useful to have a system that keeps pinging the lambda to keep it warm.

This setup can be quite convoluted but check out this post to see how you can build a reusable component to keep an AWS Lambda warm.

Just by using the component Pulumi will setup all the infra needed to keep it warm:

const lambda = new mylibrary.WarmLambda("my-warm-function", { /* options */ }, handler);

Typescript

The other thing that I didn't want to deal with was, you guessed it, Javascript. I'm well aware that the language is not that bad if you use all the modern tooling, but I'm still a Swift lover and I need my compiler 😘

Luckily Pulumi supports Typescript which is one of the best languages out there in my opinion. The fact that you can write a mix of safe checked language with crazy javascript dynamism is a pretty cool accomplishment.

And even if I love my types I don't miss them that much when dealing with a JSON API that I just need to grab a couple of values from, specially in a side project. In a more serious programming I would probably add typing and more safety to it but it's great that I don't have to do it just now.

Conclusion

Pulumi is great. I'm not an expert in infrastructure so this opinion goes in both ways. The fact that it allowed me to deploy all the services without having to check how to configure them is awesome. But because I'm not an expert about it there are probably some things that you could say are not ideal. Maybe. Probably.

In any case the project is evolving quite quickly. They support different languages like Javascript/Typescript, Python and Go (I would love to see Swift included someday) and different cloud providers like AWS, Google Cloud, Azure, etc. And even if I focused in Serverless because that's what I'm working with, you can use Containers or any other type of infrastructure.

If you're curious about any of this stuff don't doubt on checking it out, with a Free AWS account you can learn all of it for free ^^