Writing logs to files is a crucial aspect of application monitoring and debugging in production environments. In this article, we'll explore how to effectively manage file-based logging in Node.js applications using LogLayer and the Log File Rotation Transport, a powerful and flexible logging library.
What is LogLayer?
LogLayer is a modern, TypeScript-first logging library that provides a clean and intuitive API for structured logging, and it acts as a layer on top of your favorite logging library like winston or pino.
Why LogLayer for File Logging?
LogLayer provides a robust solution for file-based logging with several advantages:
- Automatic log file rotation based on size or time
- Support for log compression
- Batching capabilities for improved performance
- Flexible log formatting
- Automatic cleanup of old log files
Getting Started
First, install the necessary packages:
npm install loglayer @loglayer/transport-log-file-rotation serialize-error
Basic File Logging Setup
Here's a basic example of setting up file logging:
import { LogLayer } from "loglayer";
import { LogFileRotationTransport } from "@loglayer/transport-log-file-rotation";
import { serializeError } from "serialize-error";
const logger = new LogLayer({
errorSerializer: serializeError,
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log"
}),
],
});
// Start logging!
logger.withMetadata({
port: 3000,
}).info("Application started");
logger
.withError(new Error("Something went wrong"))
.error("An error occurred");
Adding Context and Metadata
LogLayer provides several ways to enrich your logs with additional information:
Using Context
Context is persistent data that applies to all subsequent log entries:
import { hostname } from "node:os";
const logger = new LogLayer({
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log"
}),
],
});
// Add context that will be included in all subsequent logs
const contextLogger = logger.withContext({
hostname: hostname(),
environment: process.env.NODE_ENV,
version: process.env.npm_package_version
});
contextLogger.info("Server starting"); // Will include context
contextLogger.error("Connection failed"); // Will include same context
Using Metadata
Metadata is one-time data that only applies to the current log entry:
// Add metadata for a single log entry
logger.withMetadata({
requestId: "123",
duration: 45,
statusCode: 200
}).info("Request processed");
// Different metadata for different log entries
logger.withMetadata({
userId: "user-456",
action: "login"
}).info("User logged in");
// Combine context and metadata
const userLogger = logger.withContext({ service: "user-service" });
userLogger.withMetadata({
duration: 123,
status: "success"
}).info("Operation completed");
Static Data in Transport
You can also configure static data at the transport level:
const logger = new LogLayer({
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log",
staticData: {
hostname: hostname(),
environment: process.env.NODE_ENV,
version: process.env.npm_package_version
}
}),
],
});
This will produce logs like:
{
"level": "info",
"message": "Request processed",
"timestamp": "2024-01-17T12:34:56.789Z",
"hostname": "prod-server-1",
"environment": "production",
"version": "1.0.0",
"requestId": "123",
"duration": 45,
"statusCode": 200
}
Advanced Configuration
Daily Log Rotation
For applications that need to rotate logs daily:
const logger = new LogLayer({
transport: [
new LogFileRotationTransport({
filename: "./logs/app-%DATE%.log",
frequency: "daily",
dateFormat: "YMD",
compressOnRotate: true,
maxLogs: "7d" // Keep logs for 7 days
}),
],
});
Size-based Rotation
For high-volume applications, rotating based on file size:
const logger = new LogLayer({
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log",
size: "10M", // Rotate when file reaches 10 MB
maxLogs: 5, // Keep last 5 log files
compressOnRotate: true
}),
],
});
Performance Optimization with Batching
For high-throughput applications, enable batching to reduce disk I/O:
const logger = new LogLayer({
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log",
batch: {
size: 1000, // Write after 1000 logs are queued
timeout: 5000 // Or after 5 seconds, whichever comes first
}
}),
],
});
Best Practices
- Use Rotation: Always configure log rotation to prevent disk space issues:
new LogFileRotationTransport({
filename: "./logs/app-%DATE%.log",
frequency: "daily",
maxLogs: "30d",
compressOnRotate: true
})
- Enable Batching for High Volume: For applications with high log volume:
new LogFileRotationTransport({
filename: "./logs/app.log",
batch: {
size: 1000,
timeout: 5000
}
})
- Separate Logs by Concern: Use different transports for different types of logs:
const logger = new LogLayer({
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log"
}),
new LogFileRotationTransport({
filename: "./logs/errors.log",
levelMap: {
error: "ERROR",
fatal: "FATAL"
}
})
],
});
- Use Context and Metadata Appropriately:
// Use context for values that remain constant
const serviceLogger = logger.withContext({
service: "user-service",
version: "1.0.0"
});
// Use metadata for request-specific information
serviceLogger.withMetadata({
requestId: "req-123",
duration: 45,
statusCode: 200
}).info("Request completed");
Error Handling
Proper error serialization is crucial for debugging:
import { serializeError } from "serialize-error";
const logger = new LogLayer({
errorSerializer: serializeError,
transport: [
new LogFileRotationTransport({
filename: "./logs/app.log"
}),
],
});
try {
throw new Error("Something went wrong");
} catch (error) {
logger.withError(error).error("Operation failed");
}
Conclusion
LogLayer provides a powerful and flexible solution for file-based logging in Node.js applications. With features like automatic rotation, compression, and batching, it helps you maintain clean and efficient logs while ensuring you don't miss important information for debugging and monitoring.
For more detailed information about LogLayer's file rotation transport, check out:
Top comments (0)