Suprim Devkota

Posted on Mar 10 • Edited on Mar 11

Setting Up a Windows VM for Log Collection Using the ELK Stack - A Step-by-Step Guide

#elk #cybersecurity #learning #security

Introduction

Log collection is vital for cybersecurity as it provides visibility into system activities, allowing organizations to monitor and track events happening within their infrastructure. They capture detailed information about user actions, network connections, application errors, and system performance, helping to detect suspicious activity such as unauthorized logins, privilege escalation, or malware execution. In the event of a cyberattack, logs become essential for forensic investigation, enabling cybersecurity professionals to trace the attacker's movements, understand the methods used, and identify the vulnerabilities exploited, leading to faster resolution and containment.

The ELK Stack (now often called the Elastic Stack) is a powerful, open-source suite of tools used for log management, data analysis, and visualization, comprising Elasticsearch, Logstash, and Kibana. Let us understand each tool in detail.

Elasticsearch: It is a distributed JSON-based search and analytics engine that stores and indexes data.
Think of it like a giant library where books (data) are stored in an organized way. When you need to find a specific piece of information (search for data), Elasticsearch helps you find it quickly by searching through the library's catalog (indexes).
Logstash: A server-side data processing pipeline that ingests, transforms and sends data to a desired destination. Think of it like a conveyor belt that takes raw materials (data from different sources), processes them (filters and formats the data) and moves them to an appropriate destination (Elasticsearch).
Kibana: A visualization layer that works on top of Elasticsearch, providing search and data visualization capabilities for data indexed in Elasticsearch. Think of it like the control panel in a car, providing you with graphs and information that give you insight to the performance and status of your system. Instead of just showing you the raw data, it helps you visualize and make sense of all the data you have.

In addition to these, Logs are typically collected using Beats. Beats are a collection of lightweight, open-source data shippers that collect and forward data (like logs, metrics and events) from various sources to Elasticsearch or Logstash for processing and analysis.

In this project, I'll be using Winlogbeat which focuses exclusively on Windows Event Logs. Think of Winlogbeat as a specialized messenger who only delivers important notes or messages (event logs) from your Windows system to a destination like Elasticsearch or Logstash.

So to wrap it all up, Winlogbeat collects logs from the system we want to monitor and sends it to Logstash. Logstash filters and formats the raw data and sends it to Elasticsearch which indexes the data for faster searching. Finally Kibana, lets you explore and interact with the indexed data in Elasticsearch in a user-friendly way.

In this guide, I'll walk you through setting up a Windows Virtual Machine (VM) on a host Windows system, installing the Windows OS, and configuring log collection from the VM to the host machine using the ELK Stack (Elasticsearch, Logstash, and Kibana).

By the end of this tutorial, you'll have a working ELK stack capturing Windows event logs from your VM and displaying them in Kibana for analysis.

Setting up the Windows Virtual Machine

Using VMware Workstation or VirtualBox on the host machine, attach the Windows ISO file and install the Operating system. Allocate at least 2 CPUs, 4 GB RAM and 32 GB disk space for the Virtual Machine that will be used to collect logs. To ensure network connectivity configure the network settings to Bridged Adapter or NAT

Setting Up Winlogbeat on the Virtual Machine

Once the Virtual Machine has been setup, the following things must be done to ensure that logs are properly collected and sent to the Logstash server running on the host.

Verify important logs are enabled in Event Viewer

Open Event Viewer (eventvwr.msc).
Expand Windows Logs > Security, System, Application.
Right-click each log category > Properties.
- Ensure Enable Logging is checked.
- Set log size to at least 10 MB.

Enable Audit Policies

Open Local Group Policy Editor (gpedit.msc).
Navigate to: Computer Configuration > Windows Settings > Security Settings > Advanced Audit Policy Configuration.
Enable relevant audit policies (e.g., Logon Events, Account Management, Object Access).

In this project I enabled auditing the following policies: Audit Security Group Management, Audit Audit Policy Change, Audit Process Creation, Audit Process Termination, Audit Sensitive Privilege Use, Audit File System, Audit Registry, Audit Handle Manipulation, Audit User Account Management, Audit Logon, Audit Logoff, Audit Special Logon and Audit Security State Change.

Install & Configure Winlogbeat

To ensure that firewall rules don't interfere with Winlogbeat we can run the following command to add an outbound rule to Windows Firewall.

  New-NetFirewallRule -DisplayName "Allow Winlogbeat to Logstash" -Direction Outbound -RemotePort 5044 -Protocol TCP -Action Allow

Download Winlogbeat from Winlogbeat downloads page.
Extract to C:\Program Files\Winlogbeat. (You might need to rename the downloaded directory to just Winlogbeat in order to avoid PATH TOO LONG errors.).
Configure winlogbeat.yml by uncommenting the following lines and adding your Host machine's IP as:

  output.logstash:
  hosts: ["<HOST_MACHINE_IP>:5044"]

winlogbeat.event.logs field specifies what feed to send to the Logstash server. The default configuration covers a large variety of relevant logs so I left that unchanged.

Install and Start Winlogbeat by running PowerShell as Administrator and entering the following commands:

  cd "C:\Program Files\Winlogbeat"
  Set-Execution-Policy Unrestricted -Scope CurrentUser
  .\install-service-winlogbeat.ps1
  Start-Service winlogbeat
  Set-Execution-Policy Restricted -Scope CurrentUser

Ensure that the winlogbeat service is running by executing the following command in Powershell: Get-Service winlogbeat.

Setting Up ELK Stack on the Host Machine

Now that logs are sent from the virtual machine to the host machine, we have to make sure that ElasticSearch, Logstash and Kibana are installed and configured to receive, index and display the logs in the host machine.

Install Java JDK

Download Java JDK and install it in the host computer.
Set the JAVA_HOME environment variable to point to the jdk directory.

Download & Install Elasticsearch

Download Elasticsearch from Elasticsearch downloads page.
Extract to C:\Program Files\Elasticsearch (You might need to rename the downloaded directory to just Elasticsearch in order to avoid PATH TOO LONG errors.)
Configure elasticsearch.yml as:

  network.host: 0.0.0.0
  http.port: 9200
  discovery.type: single-node

Make sure to set security features to false in order to use http instead of https which requires an SSL certificate.
Start Elasticsearch as:

  cd C:\ELK\elasticsearch\bin
  .\elasticsearch.bat

Verify that Elasticsearch has started by browsing to: http://localhost:9200.

If asked to signin, run the following command in Powershell to generate a new password from the /bin directory: elasticsearch-reset-password -u elastic

Download & Install Logstash

To ensure that firewall rules don't interfere with Logstash we can run the following command to add an inbound rule to Windows Firewall.

  New-NetFirewallRule -DisplayName "Allow Logstash" -Direction Inbound -LocalPort 5044 -Protocol TCP -Action Allow

Download Logstash from Logstash downloads page.
Extract to C:\Program Files\Logstash (You might need to rename the downloaded directory to just Logstash in order to avoid PATH TOO LONG errors.)
Configure C:\Program Files\Logstash\config\logstash.conf as:

  input {
    beats {
      port => 5044
      ssl => false
    }
  }
  output {
    elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" 
      user => "..."
      password => "..."
    }
  }

Start Logstash as: C:\ELK\logstash\bin\logstash.bat -f C:\ELK\logstash\config\logstash.conf

Download & Install Kibana

Download Kibana from Kibana downloads page.
Extract to C:\Program Files\Kibana (You might need to rename the downloaded directory to just Kibana in order to avoid PATH TOO LONG errors.)
Configure kibana.yml as:

  server.port: 5601
  elasticsearch.hosts: ["http://localhost:9200"]

Start Kibana as: C:\ELK\kibana\bin\kibana.bat
Verify that Kibana has started by browsing to: http://localhost:5601.

Create Index Pattern and validate logs in Kibana

Open Kibana at: http://localhost:5601.
Navigate to Management > Stack Management > Kibana > Data views.
Create an data view: Enter a name ,winlogbeat-* as the index pattern and select @timestamp as the time filter.
Go to Discover and select the name of the data view to view logs from the index pattern.

Conclusion

In this guide, we have covered the essential steps to set up a Windows VM for log collection using the ELK Stack:

Successfully created and configured a Windows Virtual Machine.
Enabled Windows Event Logging and configured auditing policies.
Installed and configured Winlogbeat to forward logs to Logstash.
Installed and set up the ELK Stack (Elasticsearch, Logstash, and Kibana) on the host machine.
Verified and analyzed logs in Kibana.

Next Steps and Enhancements:

Implement alerting: Use Kibana’s Watcher feature to create real-time alerts based on specific log patterns or anomalies.
Strengthen security: Secure your ELK stack using TLS encryption and authentication to protect sensitive log data.
Create custom dashboards: Develop tailored visualizations and dashboards in Kibana to monitor key performance indicators and security events.
Expand log collection: Integrate additional log sources such as Sysmon, PowerShell, and network traffic for comprehensive visibility.

DEV Community