Pulumi Has Wowed Me

As a programmer, there's always this urge to turn everything into code, hence the variety of xxx as code solutions. Even when it comes to diagramming, I prefer to use code, which led to the concept of diagram as code. I've discussed some related content in my previous article, "Quickly Overviewing the Domain Model of Code with tplant - Jeff Tian's Article - Zhihu."

But that's not all. We even want to store our secrets in code repositories! Secret as code. Of course, we can't store them in plain text. Hence, SOPS is a convenient tool that allows us to track changes to secrets using git, as detailed in "Encrypting Sensitive Information in Kubernetes Clusters - Jeff Tian's Article - Zhihu."

Is that enough? Not quite! We also hope to allocate and track changes to our infrastructure through code, so-called GitOps, or Infrastructure as code (IaC).

How do you do IaC? Feel free to share in the comments.

Let me start with how we do it.

Initially, we took a rather basic approach, using shell or PowerShell scripts, which is purely imperative. It was a hassle to tweak the scripts and verify the changes simultaneously. It was quite a headache.

Later, we moved on to Terraform, and since we primarily used AWS cloud, we also employed CloudFormation. This declarative approach significantly improved, though it fell short in the automation testing department. Moreover, writing the declarative files was somewhat dull; it wasn't programming but felt like writing HTML a decade ago. Strictly speaking, HTML is a markup language, not a programming language.

After comparing various options, our team recently decided to use Pulumi for IaC. Although I'm still getting the hang of it, the experience has been quite impressive, particularly in the following aspects:

Secret as Code is even better than SOPS.

As mentioned in "Encrypting Sensitive Information in Kubernetes Clusters - Jeff Tian's Article - Zhihu," after learning about SOPS, I've used it in several real-world projects. However, a significant issue I encountered was often forgetting to encrypt secrets before committing, leading to plain text ending up in the code repository. While there are ways to avoid this, it still leaves room for error. With Pulumi, to my surprise, there's no need for SOPS. It has the same functionality as SOPS, encrypting secrets within the code instead of saving them in plain text. But it's simpler and leaves no room for error because it provides alternative ways to retrieve plain text secrets rather than displaying them in the files.

Automated Testing

I never thought IaC could be like writing business logic with automated testing! That means you can now do TDD in your IaC projects! For example, you define the following secret information in your secrets.json file:

{
  "secrets": [
    {
      "pulumiSecretName": "rds-credentials",
      "awsSecretName": "/rds/{env}-{region}",
      "createAwsSecret": true,
      "kmsKeyIdOrAlias": null,
      "keys": [
        { "name": "username", "value": "master", "configKey": null },
        { "name": "password", "value": null, "configKey": "rds-password" }
      ],
      "description": "Secret storing credentials to RDS used by human users to access the database",
      "tags": {
        "from": "passwordstate",
      }
    },
    ...
  ]
}

In the above secret definition, we added a tag. You can use automated tests to verify that the final deployed resources contain the expected tags before actually deploying to production:

[Theory]
[ClassData(typeof(StackTestData))]
public async Task Should_CreateSecretsWithProperTags(string stackName)
{
    // act
    var resources = await RunDeployment(stackName);
    var names = GetService<ManagedSecretsResourceNames>();

    // assert
    var secrets = resources.OfType<Secret>().ToList();
    var expectedSecretsCount = Config.SecretsConfig.Secrets.Count * Config.DeploymentEnvironment.Environments.Count;
    secrets.Count.Should().Be(expectedSecretsCount);

    foreach (var secret in secrets)
    {
        var secretName = await secret.Name.GetValue();
        // ignore created secrets for different environments
        if (!secretName!.Contains(Config.DeploymentEnvironment.CurrentEnvironment.Name))
        {
            continue;
        }

        var secretConfig = Config.SecretsConfig.Secrets.Single(x => names.SecretName(x.AwsSecretName) == secretName);

        var secretTags = await secret.Tags.GetValue();
        secretTags.Should().NotBeNull();
        foreach (var tag in Config.DefaultResourceTags)
        {
            secretTags.Should().Contain(secretTag => secretTag.Key == tag.Key && secretTag.Value == tag.Value);
        }

        foreach (var tag in secretConfig.Tags)
        {
            secretTags.Should().Contain(secretTag => secretTag.Key == tag.Key && secretTag.Value == tag.Value);
        }

        secretTags.Should().Contain(secretTag => secretTag.Key == "pulumi-secret-name" && secretTag.Value == secretConfig.PulumiSecretName);
    }
}

Easter Egg: The Unexpected AI Feature

The team decided to use the C# version of Pulumi, which I found a bit daunting. However, I discovered that Pulumi offers an AI feature that answers any question I have. From what I've seen of the AI's guidance, it's focused on coding and does an incredibly reliable job of preventing any mishaps in the team (I'm still not fired at 35+).

Even better, although it seems only to write code on the surface, this AI can do anything. For instance, when I come across English words I don't recognize, I ask it to translate:

See? It can translate and even provides more thoughtful explanations than other AIs I've tried, including pinyin annotations!

Of course, you can use it directly in Chinese as well because it understands Chinese:

The most crucial point is that unlike OpenAI's ChatGPT or many other foreign AI assistants inaccessible to users in China, it is available directly from within China!

Another outrageous easter egg is that you can use it without logging in! Can you believe that?

Lastly, there are no limits whatsoever! It's incredibly generous; I've been using it for almost a month now, and it has never refused to answer my questions! There is no need to log in, let alone pay.

In one word: Phenomenal!