Using Structured Logging

Structured logging is a method of logging where log entries are formatted in a way that machines can easily parse them. Using structured logging enables better log analysis and easier querying of logs. In Go, there are several libraries that facilitate structured logging, with logrus being one of the most popular.

Basic Structured Logging with Logrus

Below is an example demonstrating how to use the logrus package for structured logging.

package main

import (
	"github.com/sirupsen/logrus"
)

func main() {
	// Initialize logrus.
	logger := logrus.New()

	// Set log level.
	logger.SetLevel(logrus.InfoLevel)

	// Structured logging.
	logger.WithFields(logrus.Fields{
		"event": "user_login",
		"user":  "johndoe",
	}).Info("User logged in")

	// Another structured log entry.
	logger.WithFields(logrus.Fields{
		"event": "file_upload",
		"user":  "johndoe",
		"size":  3456,
	}).Warn("File upload completed with warnings")
}

Configuring Logrus for JSON Output

Typically, structured logs are formatted as JSON for easy parsing in log management systems. Here's how you can output logs in JSON format.

package main

import (
	"github.com/sirupsen/logrus"
	"os"
)

func main() {
	// Initialize logrus.
	logger := logrus.New()

	// Set formatter to JSON.
	logger.SetFormatter(&logrus.JSONFormatter{})

	// Output to stdout instead of the default stderr.
	logger.SetOutput(os.Stdout)

	logger.WithFields(logrus.Fields{
		"event": "user_registration",
		"user":  "janedoe",
		"status": "success",
	}).Info("New user registration")
}

Best Practices

• Use a structured logging library, like logrus, to avoid manually formatting JSON.
• Consistently use fields to capture important context about log events (e.g., user IDs, operation types).
• Set log levels appropriately to avoid unnecessary verbosity in production environments.

Common Pitfalls

• Forgetting to set the output of logs, resulting in logs being directed to stderr where they might be missed or ignored.
• Using plain log messages without any structured fields can make searching and filtering difficult.
• Ignoring performance considerations when logging extensively in high-throughput applications.

Performance Tips

• Buffer log output to minimize impact on application performance.
• Avoid excessive logging in hot paths to prevent bottlenecks.
• Consider asynchronous logging to reduce latency.