Visualización de costos de Azure en Astro: una solución práctica

Durante el desarrollo del proyecto Goliat - Dashboard, una herramienta de código abierto para gestionar y optimizar despliegues de Terraform , surgió una necesidad específica: identificar los costos asociados a recursos en Azure que estén etiquetados con información específica, como el nombre de una organización o un identificador de proyecto.

Las etiquetas en Azure permiten clasificar recursos, lo que facilita la administración de costos por equipo, proyecto o entorno. Para aprovechar esta capacidad, desarrollé una solución que permite:

1. Buscar resource groups (RG) que contienen una etiqueta específica.
2. Obtener los costos de esos resource groups.
3. Visualizar esos costos segmentados por grupo de recursos, ubicación y nivel de servicio.

Esta solución se compone de:

1. Un endpoint en Astro para consultar Azure y devolver datos de costos filtrados por etiquetas.
2. Un componente React para visualizar los datos de manera clara y segmentada.

¿Qué ofrece esta solución?

• Filtrado preciso : identifica y analiza solo los resource groups que contienen una etiqueta específica.
• Monitoreo en tiempo real : consulta actualizada directamente desde Azure.
• Visualización clara : muestra gráficos segmentados por:
  • Resource group.
  • Ubicación.
  • Nivel de servicio (tier).
• Cacheo inteligente : guarda resultados en MongoDB para evitar consultas repetidas y mejorar el rendimiento.

1. Crear el endpoint en Astro

¿Qué hace el endpoint?

El endpoint se conecta a las APIs de Azure y realiza lo siguiente:

1. Busca todos los resource groups dentro de las suscripciones disponibles.
2. Filtra aquellos resource groups que contengan una etiqueta específica , como organization, workspace o projectId.
3. Obtiene los costos asociados a esos resource groups utilizando el servicio de cost management de Azure.
4. Devuelve los datos en formato JSON para que puedan ser consumidos por el componente React.

¿Dónde crear el endpoint?

1. Crea un archivo en la siguiente ruta de tu proyecto Astro:

import { methodValidator } from "../utils/methodValidator";
import { log } from "../utils/logging";
import { ClientSecretCredential } from "@azure/identity";
import { SubscriptionClient } from "@azure/arm-subscriptions";
import { ResourceManagementClient } from "@azure/arm-resources";
import { CostManagementClient } from "@azure/arm-costmanagement";
import getDatabase from "../utils/mongoClient";

const TENANT_ID = import.meta.env.AZURE_TENANT_ID as string;
const CLIENT_ID = import.meta.env.AZURE_CLIENT_ID as string;
const CLIENT_SECRET = import.meta.env.AZURE_CLIENT_SECRET as string;

const credential = new ClientSecretCredential(
    TENANT_ID,
    CLIENT_ID,
    CLIENT_SECRET,
);

const getSubscriptions = async () => {
    log("Fetching subscriptions...", "DEBUG");
    const client = new SubscriptionClient(credential);
    const subscriptions = [];
    for await (const subscription of client.subscriptions.list()) {
        subscriptions.push(subscription);
    }
    log(`Fetched ${subscriptions.length} subscriptions.`, "DEBUG");
    return subscriptions;
};

const getResourceGroups = async (subscriptionId: string) => {
    log(`Fetching resource groups for subscription: ${subscriptionId}`, "DEBUG");
    const client = new ResourceManagementClient(credential, subscriptionId);
    const resourceGroups = [];
    for await (const rg of client.resourceGroups.list()) {
        resourceGroups.push(rg);
    }
    log(
        `Fetched ${resourceGroups.length} resource groups for subscription ${subscriptionId}.`,
        "DEBUG",
    );
    return resourceGroups;
};

const getCostData = async (
    subscriptionId: string,
    resourceGroupName: string,
) => {
    log(
        `Fetching cost data for resource group: ${resourceGroupName} in subscription: ${subscriptionId}`,
        "DEBUG",
    );
    const costClient = new CostManagementClient(credential);
    const result = await costClient.query.usage(
        `/subscriptions/${subscriptionId}/resourceGroups/${resourceGroupName}`,
        {
            type: "ActualCost",
            timeframe: "BillingMonth",
            dataset: {
                granularity: "None",
                aggregation: {
                    totalCost: {
                        name: "PreTaxCost",
                        function: "Sum",
                    },
                },
                grouping: [
                    {
                        type: "Dimension",
                        name: "ServiceName",
                    },
                    {
                        type: "Dimension",
                        name: "ResourceLocation",
                    },
                    {
                        type: "Dimension",
                        name: "MeterSubCategory",
                    },
                ],
            },
        },
    );
    log(
        `Fetched cost data for resource group ${resourceGroupName}: ${JSON.stringify(result)}`,
        "DEBUG",
    );
    return result;
};

export const GET = async ({ request }: { request: Request }) => {
    const url = new URL(request.url);
    const organization = url.searchParams.get("organization");
    const workspace = url.searchParams.get("workspace");
    const projectId = url.searchParams.get("projectId");

    log("Received GET request.", "DEBUG");

    if (!methodValidator(request, ["GET"])) {
        log("Invalid method used. Only GET is allowed.", "ERROR");
        return new Response(JSON.stringify({ error: "Method Not Allowed" }), {
            status: 405,
            headers: { "Content-Type": "application/json" },
        });
    }

    if (!organization) {
        log("Organization parameter is missing.", "ERROR");
        return new Response(
            JSON.stringify({ error: "Organization is required." }),
            {
                status: 400,
                headers: { "Content-Type": "application/json" },
            },
        );
    }

    if ((workspace && projectId) || (!workspace && !projectId)) {
        log(
            "Invalid parameters: Provide either workspace or projectId, but not both.",
            "ERROR",
        );
        return new Response(
            JSON.stringify({
                error: "Provide either workspace or projectId, but not both.",
            }),
            {
                status: 400,
                headers: { "Content-Type": "application/json" },
            },
        );
    }

    try {
        log("Checking database connection", "DEBUG");
        const db = await getDatabase();
        const identifier = workspace || projectId || "";

        if (db) {
            log(
                `Checking cached data for organization: ${organization}, identifier: ${identifier}`,
                "DEBUG",
            );
            const collection = db.collection("azure_cost_data");
            const cachedData = await collection.findOne({
                organization,
                identifier,
            });
            if (cachedData) {
                log(
                    `Returning cached data for organization: ${organization}, identifier: ${identifier}`,
                    "DEBUG",
                );
                return new Response(JSON.stringify(cachedData), {
                    status: 200,
                    headers: { "Content-Type": "application/json" },
                });
            }
        } else {
            log("Database not available, proceeding without caching.", "DEBUG");
        }

        log("Fetching subscriptions...", "DEBUG");
        const subscriptions = await getSubscriptions();

        const results: Record<
            string,
            {
                organization: string;
                identifier: string;
                totalAllRGs?: string;
                totalAllRGsByLocation?: Record<string, string>;
                totalAllRGsByTier?: Record<string, string>;
                resourceGroups: Array<{
                    resourceGroupName: string;
                    totalCost: string;
                    costByCategory: Record<string, string>;
                    costByLocation: Record<string, string>;
                    costByTier: Record<string, string>;
                }>;
            }
        > = {};

        results[identifier] = { organization, identifier, resourceGroups: [] };

        let totalCostSum = 0;
        let costCurrency: string | null = null;
        const totalCostByLocation: Record<string, number> = {};
        const totalCostByTier: Record<string, number> = {};

        for (const subscription of subscriptions) {
            const subscriptionId = subscription.subscriptionId;
            if (!subscriptionId) {
                log("Subscription ID is undefined.", "ERROR");
                continue;
            }
            log(`Processing subscription: ${subscriptionId}`, "DEBUG");
            const resourceGroups = await getResourceGroups(subscriptionId);

            const matchingResourceGroups = resourceGroups.filter((rg: any) => {
                const tags = rg.tags || {};
                return (
                    tags.organization === organization &&
                    ((workspace && tags.workspace === workspace) ||
                        (projectId && tags.projectId === projectId))
                );
            });
            log(
                `Found ${matchingResourceGroups.length} matching resource groups.`,
                "DEBUG",
            );

            for (const rg of matchingResourceGroups) {
                try {
                    const resourceGroupName = rg.name;
                    if (!resourceGroupName) {
                        log(
                            `Resource group name is undefined for subscription ${subscriptionId}`,
                            "ERROR",
                        );
                        continue;
                    }
                    const costData = await getCostData(subscriptionId, resourceGroupName);
                    log(
                        `Cost data response for ${resourceGroupName}: ${JSON.stringify(costData)}`,
                        "DEBUG",
                    );
                    if (!costData || !costData.columns || !costData.rows) {
                        log("No columns or rows in cost data result.", "ERROR");
                        continue;
                    }
                    const costIndex = costData.columns.findIndex(
                        (col: any) => col.name === "PreTaxCost",
                    );
                    const serviceNameIndex = costData.columns.findIndex(
                        (col: any) => col.name === "ServiceName",
                    );
                    const locationIndex = costData.columns.findIndex(
                        (col: any) => col.name === "ResourceLocation",
                    );
                    const tierIndex = costData.columns.findIndex(
                        (col: any) => col.name === "MeterSubCategory",
                    );
                    const currencyIndex = costData.columns.findIndex(
                        (col: any) => col.name === "Currency",
                    );

                    let rgTotalCost = 0;
                    let currency = "";
                    const costByCategory: Record<string, number> = {};
                    const costByLocation: Record<string, number> = {};
                    const costByTier: Record<string, number> = {};

                    for (const row of costData.rows) {
                        const cost = row[costIndex];
                        const serviceName = row[serviceNameIndex];
                        const resourceLocation = row[locationIndex];
                        const tier = row[tierIndex] || "No Tier Info";
                        currency = row[currencyIndex];
                        rgTotalCost += cost;
                        costByCategory[serviceName] =
                            (costByCategory[serviceName] || 0) + cost;
                        costByLocation[resourceLocation] =
                            (costByLocation[resourceLocation] || 0) + cost;
                        costByTier[tier] = (costByTier[tier] || 0) + cost;
                    }

                    if (!costCurrency) {
                        costCurrency = currency;
                    }

                    totalCostSum += rgTotalCost;

                    for (const [location, cost] of Object.entries(costByLocation)) {
                        totalCostByLocation[location] =
                            (totalCostByLocation[location] || 0) + cost;
                    }

                    for (const [tier, cost] of Object.entries(costByTier)) {
                        totalCostByTier[tier] = (totalCostByTier[tier] || 0) + cost;
                    }

                    const formattedTotalCost = new Intl.NumberFormat("es-ES", {
                        style: "currency",
                        currency: currency,
                    }).format(rgTotalCost);

                    const formattedCostByCategory: Record<string, string> = {};
                    for (const [serviceName, cost] of Object.entries(costByCategory)) {
                        formattedCostByCategory[serviceName] = new Intl.NumberFormat(
                            "es-ES",
                            {
                                style: "currency",
                                currency: currency,
                            },
                        ).format(cost);
                    }

                    const formattedCostByLocation: Record<string, string> = {};
                    for (const [location, cost] of Object.entries(costByLocation)) {
                        formattedCostByLocation[location] = new Intl.NumberFormat("es-ES", {
                            style: "currency",
                            currency: currency,
                        }).format(cost);
                    }

                    const formattedCostByTier: Record<string, string> = {};
                    for (const [tier, cost] of Object.entries(costByTier)) {
                        formattedCostByTier[tier] = new Intl.NumberFormat("es-ES", {
                            style: "currency",
                            currency: currency,
                        }).format(cost);
                    }

                    results[identifier].resourceGroups.push({
                        resourceGroupName,
                        totalCost: formattedTotalCost,
                        costByCategory: formattedCostByCategory,
                        costByLocation: formattedCostByLocation,
                        costByTier: formattedCostByTier,
                    });
                } catch (error) {
                    log(`Error processing resource group ${rg.name}: ${error}`, "ERROR");
                }
            }
        }

        if (results[identifier].resourceGroups.length === 0) {
            log("No matching resource groups found.", "ERROR");
            return new Response(
                JSON.stringify({ error: "No matching resource groups found." }),
                {
                    status: 404,
                    headers: { "Content-Type": "application/json" },
                },
            );
        }

        if (costCurrency) {
            results[identifier].totalAllRGs = new Intl.NumberFormat("es-ES", {
                style: "currency",
                currency: costCurrency,
            }).format(totalCostSum);

            const formattedTotalAllRGsByLocation: Record<string, string> = {};
            for (const [location, cost] of Object.entries(totalCostByLocation)) {
                formattedTotalAllRGsByLocation[location] = new Intl.NumberFormat(
                    "es-ES",
                    {
                        style: "currency",
                        currency: costCurrency,
                    },
                ).format(cost);
            }
            results[identifier].totalAllRGsByLocation =
                formattedTotalAllRGsByLocation;

            const formattedTotalAllRGsByTier: Record<string, string> = {};
            for (const [tier, cost] of Object.entries(totalCostByTier)) {
                formattedTotalAllRGsByTier[tier] = new Intl.NumberFormat("es-ES", {
                    style: "currency",
                    currency: costCurrency,
                }).format(cost);
            }
            results[identifier].totalAllRGsByTier = formattedTotalAllRGsByTier;
        }

        if (db) {
            log(
                `Caching data for organization: ${organization}, identifier: ${identifier}`,
                "DEBUG",
            );
            const collection = db.collection("azure_cost_data");
            await collection.updateOne(
                { organization, identifier },
                { $set: results[identifier] },
                { upsert: true },
            );
        }

        return new Response(JSON.stringify(results[identifier]), {
            status: 200,
            headers: { "Content-Type": "application/json" },
        });
    } catch (error: unknown) {
        log(
            `Internal server error: ${error instanceof Error ? error.stack : String(error)}`,
            "ERROR",
        );
        return new Response(JSON.stringify({ error: "Internal server error" }), {
            status: 500,
            headers: { "Content-Type": "application/json" },
        });
    }
};

1. Copia el código del endpoint en ese archivo. Este código manejará la búsqueda de resource groups etiquetados y la consulta de costos a Azure.

2. Configura las variables de entorno en el archivo .env:

3. Instala las dependencias necesarias para conectar con Azure y MongoDB:

2. Crear el componente en React

¿Qué hace el componente?

El componente React se encarga de:

1. Consultar el endpoint para obtener los costos de los resource groups filtrados por etiquetas.
2. Mostrar el estado de carga y los posibles errores.
3. Visualizar los datos en gráficos tipo doughnut usando Chart.js :
   • Costos por resource group.
   • Costos por ubicación.
   • Costos por nivel de servicio (tier).

¿Dónde crear el componente?

1. Crea el archivo del componente en:

import React, { useEffect, useState } from "react";
import { Doughnut } from "react-chartjs-2";
import { FaSpinner } from "react-icons/fa";
import { Chart as ChartJS, ArcElement, Tooltip, Legend } from "chart.js";

ChartJS.register(ArcElement, Tooltip, Legend);

export default function WorkspaceCostsController({ organization, workspace }) {
    const [data, setData] = useState(null);
    const [loading, setLoading] = useState(true);
    const [error, setError] = useState(null);

    const fetchData = async () => {
        setLoading(true);
        setError(null);
        try {
            const url = `/api/private/azureCost?organization=${organization}&workspace=${workspace}`;
            const response = await fetch(url);
            if (!response.ok) {
                const errorText = await response.text();
                throw new Error(`Error fetching data: ${errorText}`);
            }
            const jsonData = await response.json();
            setData(jsonData);
        } catch (err) {
            setError(err.message);
        } finally {
            setLoading(false);
        }
    };

    useEffect(() => {
        fetchData();
    }, [organization, workspace]);

    if (loading) {
        return (
            <div className="flex justify-center items-center h-64">
                <FaSpinner className="animate-spin text-2xl text-gray-900 dark:text-white" />
            </div>
        );
    }

    if (error) {
        return <p className="text-red-500">Error: {error}</p>;
    }

    if (!data) {
        return (
            <p className="text-gray-500 dark:text-gray-400">No data available</p>
        );
    }

    const {
        totalAllRGs,
        totalAllRGsByLocation,
        totalAllRGsByTier,
        resourceGroups,
    } = data;

    const isDarkMode = document.documentElement.classList.contains("dark");
    const textColor = isDarkMode ? "#F9FAFB" : "#1F2937";
    const borderColor = isDarkMode ? "#4B5563" : "#D1D5DB";

    const parseEuroValue = (val) => {
        if (!val) return 0;
        let numericString = val.replace(/[^\d.,-]/g, "");
        numericString = numericString.replace(",", ".");
        const numberValue = parseFloat(numericString);
        return isNaN(numberValue) ? 0 : numberValue;
    };

    const formatEuro = (value) => {
        return new Intl.NumberFormat("es-ES", {
            style: "currency",
            currency: "EUR",
        }).format(value);
    };

    const generateRandomPastelColor = () => {
        const hue = Math.floor(Math.random() * 360);
        const base = `hsl(${hue}, 70%, 80%)`;
        const hover = `hsl(${hue}, 70%, 60%)`;
        return { base, hover };
    };

    const generateDoughnutData = (obj) => {
        const entries = Object.entries(obj || {});
        const filteredEntries = entries.filter(
            ([_, val]) => parseEuroValue(val) !== 0,
        );

        if (filteredEntries.length === 0) {
            return null;
        }

        const labels = filteredEntries.map(([key]) => key);
        const values = filteredEntries.map(([_, val]) => parseEuroValue(val));

        const backgroundColors = [];
        const hoverBackgroundColors = [];

        labels.forEach(() => {
            const { base, hover } = generateRandomPastelColor();
            backgroundColors.push(base);
            hoverBackgroundColors.push(hover);
        });

        return {
            labels,
            datasets: [
                {
                    data: values,
                    backgroundColor: backgroundColors,
                    hoverBackgroundColor: hoverBackgroundColors,
                    borderColor: borderColor,
                    borderWidth: 2,
                },
            ],
        };
    };

    const locationData = totalAllRGsByLocation
        ? generateDoughnutData(totalAllRGsByLocation)
        : null;
    const tierData = totalAllRGsByTier
        ? generateDoughnutData(totalAllRGsByTier)
        : null;

    let rgData = null;
    if (resourceGroups && resourceGroups.length > 0) {
        const rgObj = {};
        resourceGroups.forEach((rg) => {
            if (rg.totalCost) {
                rgObj[rg.resourceGroupName] = rg.totalCost;
            }
        });
        rgData = generateDoughnutData(rgObj);
    }

    const doughnutOptions = {
        responsive: true,
        maintainAspectRatio: false,
        layout: {
            padding: {
                top: 10,
                bottom: 10,
            },
        },
        plugins: {
            legend: {
                position: "bottom",
                labels: {
                    color: textColor,
                },
            },
            tooltip: {
                bodyColor: textColor,
                titleColor: textColor,
                backgroundColor: isDarkMode ? "#374151" : "#ffffff",
                callbacks: {
                    label: function (context) {
                        const label = context.label || "";
                        const value = context.parsed;
                        return `${label}: ${formatEuro(value)}`;
                    },
                },
            },
        },
    };

    return (
        <div className="mt-6">
            <div className="grid w-full grid-cols-1 gap-4 xl:grid-cols-3">
                <div className="relative p-4 bg-white border border-gray-200 rounded-lg shadow dark:border-gray-700 dark:bg-gray-800 flex flex-col">
                    <h3 className="text-lg font-medium text-gray-900 dark:text-white mb-2">
                        Cost by Resource Group
                    </h3>
                    <p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
                        Cost breakdown by each resource group.
                    </p>
                    <div className="h-64">
                        {rgData ? (
                            <Doughnut data={rgData} options={doughnutOptions} />
                        ) : (
                            <p className="text-gray-500 dark:text-gray-400">No data</p>
                        )}
                    </div>
                </div>

                <div className="relative p-4 bg-white border border-gray-200 rounded-lg shadow dark:border-gray-700 dark:bg-gray-800 flex flex-col">
                    <h3 className="text-lg font-medium text-gray-900 dark:text-white mb-2">
                        Cost by Location
                    </h3>
                    <p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
                        Cost breakdown by resource region.
                    </p>
                    <div className="h-64">
                        {locationData ? (
                            <Doughnut data={locationData} options={doughnutOptions} />
                        ) : (
                            <p className="text-gray-500 dark:text-gray-400">No data</p>
                        )}
                    </div>
                </div>

                <div className="relative p-4 bg-white border border-gray-200 rounded-lg shadow dark:border-gray-700 dark:bg-gray-800 flex flex-col">
                    <h3 className="text-lg font-medium text-gray-900 dark:text-white mb-2">
                        Cost by Tier
                    </h3>
                    <p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
                        Cost breakdown by service tier.
                    </p>
                    <div className="h-64">
                        {tierData ? (
                            <Doughnut data={tierData} options={doughnutOptions} />
                        ) : (
                            <p className="text-gray-500 dark:text-gray-400">No data</p>
                        )}
                    </div>
                </div>
            </div>
        </div>
    );
}

1. Copia el código del componente en ese archivo.

2. Instala las dependencias necesarias para los gráficos y los íconos de carga:

3. Integrar el componente en una página de Astro

Para mostrar el dashboard de costos en tu aplicación Astro:

1. Crea una página Astro en:

2. Importa el componente y pásale los parámetros necesarios, como organization y workspace:

4. Probar la implementación

1. Inicia el servidor de desarrollo con el siguiente comando:

2. Accede a la página en tu navegador, por ejemplo:

3. Deberías ver los gráficos que muestran los costos de los resource groups filtrados por la etiqueta especificada.

Conclusión

Esta solución permite monitorear y visualizar los costos de Azure de una manera eficiente, enfocándose en aquellos resource groups que cumplen con criterios específicos a través de etiquetas. Esto facilita la gestión de recursos y la optimización de costos en proyectos complejos.

Si te interesa una herramienta integral para gestionar tus despliegues de Terraform y visualizar costos en Azure, te invito a explorar el proyecto Goliat - Dashboard, una plataforma de código abierto diseñada para optimizar tus operaciones de infraestructura.