PART 1: How to Set Up Grafana and Prometheus Using Docker

Ready to supercharge your NestJS applications? In this blog, we'll walk through the easy steps of setting up Grafana and Prometheus with Docker. Say hello to seamless monitoring and visualization, making your development journey smoother and more insightful. Let's dive into the world of containerized deployment together and add a visual edge to your NestJS projects, bringing your data to life in a way that goes beyond performance metrics.

This will be a comprehensive guide :

1. Create a Docker Compose file named 'docker-compose.monitoring.yml' in your main project

2. Create a folder named "config" and inside it config/prometheus.yaml create file named promtheus.yaml

• We need four essential components to establish a monitoring setup: Prometheus, Grafana, cAdvisor, and Node Exporter :

Prometheus: An open-source monitoring and alerting toolkit designed for reliability and scalability. Prometheus collects and stores time-series data, allowing for querying and analysis.

Grafana: A popular open-source platform for monitoring and observability. Grafana provides a customizable dashboard for visualizing and analyzing metrics from various data sources, including Prometheus.

cAdvisor: Short for Container Advisor, cAdvisor is an open-source container monitoring tool. It analyzes resource usage and performance characteristics of running containers, providing valuable insights into containerized environments.

Node Exporter: A Prometheus exporter that collects hardware and OS-level metrics from Linux and other Unix-like systems. Node Exporter enables monitoring of host-level resources and performance metrics.

Let's dive into practice again 🔥

1. in our "docker-compose.monitoring.yml" file we need to add the code below :

   --First One :

version: '3.8'
networks:
  monitoring:
    driver: bridge

In this step, we set the Docker Compose version to 3.8 and define a custom network named "monitoring" with the bridge driver.

-- Second One:

volumes:
  prometheus-data:
    driver: local
  grafana-data:
    driver: local

This step creates local volumes named "prometheus-data" and "grafana-data" for persistent storage.

-- Third One: Service 1: prometheus :

prometheus:
  image: prom/prometheus:v2.37.9
  container_name: prometheus
  ports:
    - 9090:9090
  command:
    - '--config.file=/etc/prometheus/prometheus.yaml'
  volumes:
    - ./config/prometheus.yaml:/etc/prometheus/prometheus.yaml:ro
    - ./data:/prometheus
  restart: unless-stopped

Image: Specifies the Docker image for Prometheus and its version.
Container Name: Assigns a specific name to the Prometheus container.
Ports: Maps the host machine's port 9090 to the container's port 9090 for accessing Prometheus.
Command: Sets the configuration file path for Prometheus.
Volumes: Binds local directories for Prometheus configuration and data persistence.
Restart: Ensures that the Prometheus container restarts unless explicitly stopped.

-- Service 2: Grafana :

grafana:
  image: grafana/grafana-oss:latest
  container_name: grafana
  ports:
    - '3000:3000'
  volumes:
    - grafana-data:/var/lib/grafana
  restart: unless-stopped

-- Service 3 : node-exporter:

node_exporter:
  image: quay.io/prometheus/node-exporter:v1.5.0
  container_name: node_exporter
  command: '--path.rootfs=/host'
  pid: host
  restart: unless-stopped
  volumes:
    - /:/host:ro,rslave

-- Service 4 : cAdvisor:

cadvisor:
  image: gcr.io/cadvisor/cadvisor:v0.47.0
  container_name: cadvisor
  command:
    - '-port=8098'
  volumes:
    - /:/rootfs:ro
    - /var/run:/var/run:ro
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
    - /dev/disk/:/dev/disk:ro
  devices:
    - /dev/kmsg
  privileged: true
  restart: unless-stopped

So the result will be like this 🚨:

version: '3.8'
networks:
  monitoring:
    driver: bridge
volumes:
  prometheus-data:
    driver: local
  grafana-data:
    driver: local
services:
  prometheus:
    image: prom/prometheus:v2.37.9
    container_name: prometheus
    ports:
      - 9090:9090
    command:
      - '--config.file=/etc/prometheus/prometheus.yaml'
    volumes:
      - ./config/prometheus.yaml:/etc/prometheus/prometheus.yaml:ro
      - ./data:/prometheus
    restart: unless-stopped
  grafana:
    image: grafana/grafana-oss:latest
    container_name: grafana
    ports:
      - '3000:3000'
    volumes:
      - grafana-data:/var/lib/grafana
    restart: unless-stopped
    #password: root123
  node_exporter:
    image: quay.io/prometheus/node-exporter:v1.5.0
    container_name: node_exporter
    command: '--path.rootfs=/host'
    pid: host
    restart: unless-stopped
    volumes:
      - /:/host:ro,rslave
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.47.0
    container_name: cadvisor
    command:
      - '-port=8098'
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    devices:
      - /dev/kmsg
    privileged: true
    restart: unless-stopped

Now, let's dive into the Prometheus configuration file:
in our config/prometheus.yaml file we need to define the scarpe configurations :

global:
  scrape_interval: 15s

Sets a global scrape interval of 15 seconds. This means Prometheus will collect metrics from configured targets at this interval

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['myapi:6003']
    metrics_path: '/tam-metrics'

Defines a scrape configuration for a job named 'prometheus'.
scrape_interval: 5s: Overrides the global scrape interval for this specific job to 5 seconds.
static_configs: Specifies a list of statically configured targets.
targets: ['myapi:6003']: Indicates that Prometheus should scrape metrics from the target 'myapi' on port 6003.
metrics_path: '/tam-metrics': Specifies the path where Prometheus should scrape metrics for this job.

3.

  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node_exporter:9100']

Adds another job configuration named 'node-exporter'.
targets: ['node_exporter:9100']: Indicates that Prometheus should scrape metrics from the target 'node_exporter' on port 9100.

4.

  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8098']

targets: ['cadvisor:8098']: Specifies that Prometheus should scrape metrics from the target 'cadvisor' on port 8098.

the full result will be like this :

global:
  scrape_interval: 15s
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['myapi:6003']
    metrics_path: '/tam-metrics'
  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node_exporter:9100']

  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8098']

to run all this just type in terminal :

docker-compose -f docker-compose.monitoring.yml up -d

and BOOM 🤯:

Prometheus

Grafana

first time login : admin/admin

So, here we are, reaching the conclusion of this section. We've successfully configured our containers, everything is running smoothly, and we've achieved some impressive results. But hold on, this is just the beginning! The real magic happens when we dive into configuring our dashboards and exporting metrics from our Nest app. Get ready for some visual delights! Stay tuned for Part 2, where we'll explore these exciting aspects in detail. See you there for an adventure in data visualization.

👋 You can also Contact me and follow my posts in Linkedin

Comments

[Ladalo]

Wooow, find it usefull

[Phương Nam]

hello, Why declare networks monitoring and not use it in each service?
static_configs:
- targets: ['myapi:6003']
Why is it 6003 and not 9090 for prometheus?

[Chafroud Tarek]

You can, both methods work well.

[JF]

I had to add
user: "1000:1000" (or your UID) to make it work.

[JF]

... to the prometheus service.