Python Logging Formatter Example with Ready-to-Use Code

Ever wonder why your Python logs look like cryptic one-liners when you’re debugging a production issue at 2 AM? Default logging gives you bare-bones output that’s hard to parse when things go wrong. Python’s logging.Formatter class lets you structure log messages with timestamps, severity levels, file locations, and custom fields, so you can actually find what broke and when. This guide shows you ready-to-use formatter examples for console output, file logging, custom formatters, and multi-handler setups that’ll save you hours of log-digging.

Basic Formatter Implementation with Code Examples

The logging.Formatter class controls how your log messages appear. It defines the structure and content of each logged line, transforming raw LogRecord objects into formatted strings that include timestamps, severity levels, module names, and your actual log messages.

You’ve got two common approaches for implementing formatters: attach them to StreamHandler for console output or to FileHandler for writing logs to disk. Both use the same logging.Formatter class but route the formatted messages to different destinations.

Here’s a basic console logging example with StreamHandler:

import logging

# Create logger
logger = logging.getLogger('my_app')
logger.setLevel(logging.DEBUG)

# Create console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)

# Create formatter
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(message)s')

# Add formatter to handler
console_handler.setFormatter(formatter)

# Add handler to logger
logger.addHandler(console_handler)

# Test logging
logger.debug('Debug message')
logger.info('Application started')
logger.warning('Low disk space')
logger.error('Failed to connect to database')

Output:

2024-01-15 10:30:45,123 - DEBUG - my_app - Debug message
2024-01-15 10:30:45,124 - INFO - my_app - Application started
2024-01-15 10:30:45,125 - WARNING - my_app - Low disk space
2024-01-15 10:30:45,126 - ERROR - my_app - Failed to connect to database

Here’s a file logging example with FileHandler using a different format:

import logging

# Create logger
logger = logging.getLogger('file_logger')
logger.setLevel(logging.INFO)

# Create file handler
file_handler = logging.FileHandler('application.log')
file_handler.setLevel(logging.INFO)

# Create formatter with additional details
formatter = logging.Formatter(
    '%(asctime)s | %(levelname)-8s | %(filename)s:%(lineno)d | %(message)s'
)

# Add formatter to handler
file_handler.setFormatter(formatter)

# Add handler to logger
logger.addHandler(file_handler)

# Test logging
logger.info('User logged in successfully')
logger.warning('API rate limit approaching')
logger.error('Payment processing failed')

Output (in application.log):

2024-01-15 10:30:45,123 | INFO     | app.py:15 | User logged in successfully
2024-01-15 10:30:45,124 | WARNING  | app.py:16 | API rate limit approaching
2024-01-15 10:30:45,125 | ERROR    | app.py:17 | Payment processing failed

Understanding Format String Attributes and Placeholders

Format strings access LogRecord attributes using Python’s % formatting style with attribute names wrapped in parentheses and followed by format specifiers. Each time you log a message, Python creates a LogRecord object containing metadata about that log event (timestamp, severity, location, message), and the formatter extracts specific attributes using placeholders like %(asctime)s or %(levelname)s.

You can choose from three format string styles: “%” (printf-style, the default), “$” (Template-style), and “{” (f-string style). The % style is most common because it’s the logging module’s default and offers the widest compatibility. Use Template style when you need simpler syntax without type specifiers, and use f-string style when working with Python 3.2+ and prefer modern string formatting syntax. For most production applications, stick with % style unless you have a specific reason to switch.

Timestamp Formatting with datefmt Parameter

The datefmt parameter lets you customize how timestamps appear using strftime format codes. This gives you control over date and time representation, from ISO 8601 standard format to custom layouts with milliseconds.

Here are three common timestamp formats:

import logging

# ISO 8601 format (recommended for production)
formatter_iso = logging.Formatter(
    '%(asctime)s - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%dT%H:%M:%S'
)

# US-style format with 12-hour clock
formatter_us = logging.Formatter(
    '%(asctime)s - %(levelname)s - %(message)s',
    datefmt='%m/%d/%Y %I:%M:%S %p'
)

# Custom format with milliseconds
formatter_ms = logging.Formatter(
    '%(asctime)s.%(msecs)03d - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S'
)

# Example outputs:
# ISO 8601: 2024-01-15T10:30:45 - INFO - Server started
# US-style: 01/15/2024 10:30:45 AM - INFO - Server started
# Milliseconds: 2024-01-15 10:30:45.123 - INFO - Server started

Custom Formatter Class Implementation

Custom formatter classes come in handy when you need to add contextual information that’s not available in standard LogRecord attributes, transform data before formatting, or apply conditional formatting based on log content or severity.

import logging
import threading

class ContextFormatter(logging.Formatter):
    """Custom formatter that adds thread name and custom fields"""

    def format(self, record):
        # Add thread name to the record
        record.thread_name = threading.current_thread().name

        # Add custom field if it exists
        if not hasattr(record, 'user_id'):
            record.user_id = 'N/A'

        # Call parent format method
        return super().format(record)

    def formatException(self, exc_info):
        """Custom exception formatting"""
        result = super().formatException(exc_info)
        return f"EXCEPTION DETAILS:\n{result}"

# Set up logger with custom formatter
logger = logging.getLogger('custom_app')
logger.setLevel(logging.DEBUG)

handler = logging.StreamHandler()
formatter = ContextFormatter(
    '%(asctime)s | %(thread_name)s | %(levelname)s | User: %(user_id)s | %(message)s'
)
handler.setFormatter(formatter)
logger.addHandler(handler)

# Test with custom field
logger.info('Regular log message')
logger.info('User action', extra={'user_id': 'user_12345'})

# Test exception formatting
try:
    result = 10 / 0
except ZeroDivisionError:
    logger.exception('Division error occurred')

The key methods you can override in custom formatter classes are format() for modifying the entire log record before formatting, formatTime() for custom timestamp representations, formatException() for controlling how exception tracebacks appear, and formatStack() for customizing stack trace output. Override format() when you need to add or modify record attributes, formatTime() when the datefmt parameter isn’t flexible enough, formatException() when you want to filter or enhance stack traces, and formatStack() when debugging requires custom stack information.

Handler Configuration with Multiple Formatters

Python’s logging system provides five severity levels that control which messages get logged: DEBUG (10) for detailed diagnostic information, INFO (20) for general informational messages, WARNING (30) for warning messages about potential issues, ERROR (40) for error messages when something fails, and CRITICAL (50) for critical failures that might stop the application. Each handler can have its own log level, and messages below that threshold won’t be processed by that handler.

Multiple Handlers with Different Format Configurations

import logging

# Create logger
logger = logging.getLogger('multi_handler_app')
logger.setLevel(logging.DEBUG)  # Logger accepts all levels

# Console handler with simplified format
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)  # Only INFO and above to console
console_formatter = logging.Formatter('%(levelname)s - %(message)s')
console_handler.setFormatter(console_formatter)

# File handler with detailed format
file_handler = logging.FileHandler('detailed.log')
file_handler.setLevel(logging.DEBUG)  # All messages to file
file_formatter = logging.Formatter(
    '%(asctime)s | %(levelname)-8s | %(filename)s:%(lineno)d:%(funcName)s | %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S'
)
file_handler.setFormatter(file_formatter)

# Add both handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Test logging
logger.debug('Detailed debug information')  # Only in file
logger.info('Application started')  # Both console and file
logger.warning('Configuration file missing')  # Both console and file
logger.error('Database connection failed')  # Both console and file

Each handler operates independently with its own formatter and log level. The logger’s setLevel() acts as a gatekeeper (if the logger level is INFO, DEBUG messages never reach any handler), while each handler’s setLevel() provides a secondary filter. In this example, DEBUG messages reach the file handler but not the console handler, even though the logger accepts them.

Level-Based Formatting Strategies

import logging

logger = logging.getLogger('level_based')
logger.setLevel(logging.DEBUG)

# Verbose handler for DEBUG and ERROR levels
verbose_handler = logging.FileHandler('verbose.log')
verbose_handler.setLevel(logging.DEBUG)
verbose_formatter = logging.Formatter(
    '%(asctime)s | %(levelname)-8s | %(pathname)s:%(lineno)d | '
    '%(funcName)s | %(process)d:%(thread)d | %(message)s'
)
verbose_handler.setFormatter(verbose_formatter)

# Minimal handler for INFO and WARNING
minimal_handler = logging.FileHandler('minimal.log')
minimal_handler.setLevel(logging.INFO)
minimal_formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
minimal_handler.setFormatter(minimal_formatter)

# Add custom filter to verbose handler to only accept DEBUG and ERROR
class DebugErrorFilter(logging.Filter):
    def filter(self, record):
        return record.levelno in (logging.DEBUG, logging.ERROR)

verbose_handler.addFilter(DebugErrorFilter())

logger.addHandler(verbose_handler)
logger.addHandler(minimal_handler)

DEBUG level: Use verbose formatting with full paths, line numbers, function names, process IDs, and thread IDs to pinpoint issues during development and troubleshooting.

INFO level: Use minimal formatting with just timestamp, level, and message since these are routine operational messages that don’t need deep context.

WARNING level: Use minimal to moderate formatting, adding filename or module name to quickly identify which component raised the warning without cluttering logs.

ERROR level: Use verbose formatting similar to DEBUG, including full context like stack location and thread information to investigate failures.

CRITICAL level: Use maximum verbosity with all available context, since these represent severe failures requiring immediate investigation.

Using basicConfig() for Simple Setups

The basicConfig() function provides the fastest way to configure logging for small scripts and prototypes. It sets up a single handler with a format string in one function call.

import logging

# Configure logging with basicConfig()
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
    filename='app.log',
    filemode='a'  # append mode
)

# Use root logger
logging.debug('Debug message')
logging.info('Info message')
logging.warning('Warning message')
logging.error('Error message')

Output (in app.log):

2024-01-15 10:30:45 - root - DEBUG - Debug message
2024-01-15 10:30:45 - root - INFO - Info message
2024-01-15 10:30:45 - root - WARNING - Warning message
2024-01-15 10:30:45 - root - ERROR - Error message

The basicConfig() function only works once and must be called before any logging functions (debug(), info(), warning(), etc.). If you call logging.info() before basicConfig(), Python automatically configures a default handler, and your basicConfig() call won’t change anything. For anything beyond quick scripts, use explicit logger, handler, and formatter configuration instead.

Dictionary-Based Configuration with dictConfig()

import logging
import logging.config

# Define logging configuration as a dictionary
LOGGING_CONFIG = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'detailed': {
            'format': '%(asctime)s | %(name)-12s | %(levelname)-8s | %(filename)s:%(lineno)d | %(message)s',
            'datefmt': '%Y-%m-%d %H:%M:%S'
        },
        'simple': {
            'format': '%(levelname)s - %(message)s'
        },
        'json_style': {
            'format': '%(asctime)s | %(levelname)s | %(name)s | %(message)s'
        }
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'level': 'INFO',
            'formatter': 'simple',
            'stream': 'ext://sys.stdout'
        },
        'file': {
            'class': 'logging.FileHandler',
            'level': 'DEBUG',
            'formatter': 'detailed',
            'filename': 'application.log',
            'mode': 'a'
        },
        'error_file': {
            'class': 'logging.FileHandler',
            'level': 'ERROR',
            'formatter': 'detailed',
            'filename': 'errors.log',
            'mode': 'a'
        }
    },
    'loggers': {
        'app': {
            'level': 'DEBUG',
            'handlers': ['console', 'file', 'error_file'],
            'propagate': False
        },
        'app.database': {
            'level': 'WARNING',
            'handlers': ['file'],
            'propagate': False
        }
    },
    'root': {
        'level': 'INFO',
        'handlers': ['console']
    }
}

# Apply configuration
logging.config.dictConfig(LOGGING_CONFIG)

# Get loggers and test
app_logger = logging.getLogger('app')
db_logger = logging.getLogger('app.database')

app_logger.debug('Application starting')
app_logger.info('Configuration loaded')
app_logger.error('Failed to initialize module')
db_logger.warning('Slow query detected')

Dictionary configuration is preferred for enterprise applications and complex projects because it provides a complete, declarative view of your entire logging setup in one structure. You can store this configuration in a separate file, load it from JSON or YAML, manage different configurations for development versus production, and modify the entire logging system without changing code. It’s especially useful when working with frameworks like Django or Flask that expect dictionary-based logging configuration. When you need to manage API and web tools applications with multiple log levels for debugging different endpoints and services, dictionary configuration lets you define environment-specific handlers and formatters cleanly.

Structured JSON Logging Formatter Examples

JSON formatting is preferred in production environments because it produces machine-readable output that log aggregation tools can parse without custom regex patterns or parsing rules. Each log entry becomes a structured object with named fields, making it simple to filter, search, and analyze logs across distributed systems and microservices.

Install the python-json-logger package:

pip install python-json-logger

Here’s a complete implementation with custom fields:

import logging
from pythonjsonlogger import jsonlogger

# Create logger
logger = logging.getLogger('json_app')
logger.setLevel(logging.DEBUG)

# Create handler
handler = logging.StreamHandler()

# Create JSON formatter with custom fields
formatter = jsonlogger.JsonFormatter(
    '%(asctime)s %(name)s %(levelname)s %(filename)s %(lineno)d %(message)s',
    timestamp=True
)

handler.setFormatter(formatter)
logger.addHandler(handler)

# Test logging with extra fields
logger.info('User login successful', extra={
    'user_id': 'usr_12345',
    'ip_address': '192.168.1.100',
    'session_id': 'sess_abcdef'
})

logger.error('Payment processing failed', extra={
    'user_id': 'usr_67890',
    'transaction_id': 'txn_xyz123',
    'amount': 99.99,
    'currency': 'USD'
})

Output:

{"asctime": "2024-01-15 10:30:45,123", "name": "json_app", "levelname": "INFO", "filename": "app.py", "lineno": 15, "message": "User login successful", "user_id": "usr_12345", "ip_address": "192.168.1.100", "session_id": "sess_abcdef"}
{"asctime": "2024-01-15 10:30:45,456", "name": "json_app", "levelname": "ERROR", "filename": "app.py", "lineno": 21, "message": "Payment processing failed", "user_id": "usr_67890", "transaction_id": "txn_xyz123", "amount": 99.99, "currency": "USD"}

Structured logging shines in microservices architectures where each service generates logs that need to be aggregated into a central system like Elasticsearch, Splunk, or Datadog. Instead of writing complex parsing rules to extract fields from text logs, these tools can directly index JSON fields, enabling powerful queries like “show all ERROR logs where transactionid exists and amount > 100″ or “count failed logins by ipaddress in the last hour.” The consistent structure also makes it easier to correlate events across multiple services using shared fields like requestid or userid.

Advanced Formatter Techniques for Production Applications

Including contextual information in your log formatters makes debugging production issues exponentially faster. Instead of logging “Payment failed,” log “Payment failed for userid=usr12345, transactionid=txnabc123, amount=99.99, gateway=stripe.” You can add this context using the extra parameter in logging calls or by creating custom formatters that automatically include request IDs from web frameworks, correlation IDs for distributed tracing, or environment identifiers (prod, staging, dev) to distinguish logs from different deployments.

Never log sensitive data like passwords, authentication tokens, API keys, credit card numbers, social security numbers, or other personally identifiable information (PII). Even if your logs are stored securely, compliance regulations like GDPR and HIPAA require you to minimize PII collection and implement data retention policies. If you must log user-related information for debugging and troubleshooting tools, use pseudonymized identifiers like hashed user IDs instead of raw email addresses or usernames. Configure formatters to explicitly exclude certain record attributes, and train your team to use extra={‘safeuserid’: hashed_id} instead of including raw user data in messages.

Exception logging and stack traces require special handling to remain readable across multiple lines. Use the exc_info parameter or logging.exception() to automatically capture and format tracebacks:

import logging

logger = logging.getLogger('exception_app')
handler = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(logging.ERROR)

try:
    result = 10 / 0
except ZeroDivisionError:
    # Method 1: Use logging.exception() (automatically includes traceback at ERROR level)
    logger.exception('Division by zero occurred')

    # Method 2: Use exc_info=True with any logging level
    logger.error('Calculation failed', exc_info=True)

The formatter automatically handles multiline traceback formatting, preserving the entire stack trace as part of the log record. For JSON logging, some formatters can serialize exceptions into structured fields instead of embedding the full traceback text, making it easier to analyze error patterns programmatically.

Formatter Performance Optimization and Best Practices

Complex format strings with many placeholders have minimal performance impact because the formatting only happens for messages that pass the log level threshold. The real performance cost comes from expensive operations inside custom formatter classes like database queries, API calls, or complex string manipulation. If your formatter needs to enrich logs with external data, cache that data or fetch it asynchronously to avoid blocking the logging call.

Always use lazy evaluation by passing variables as arguments to logging functions instead of formatting strings beforehand. The logging module only formats the message if it will actually be logged:

# Bad - formats string even if DEBUG is disabled
logger.debug(f'Processing {len(items)} items: {items}')

# Good - only formats if DEBUG level is active
logger.debug('Processing %d items: %s', len(items), items)

Avoid string formatting in log calls: Use ‘%s’, ‘%d’, ‘%r’ placeholders and pass values as arguments so Python skips formatting when the log level is disabled.

Keep custom formatters fast: Don’t perform I/O operations, database queries, or expensive computations inside format(), formatTime(), or other formatter methods.

Cache static formatter data: If your formatter adds constant fields like hostname or application version, compute them once during initialization instead of on every log call.

Use filters for expensive decisions: Instead of checking conditions in a custom formatter, attach a filter to the handler that rejects records early before formatting happens.

Filters provide an efficient way to prevent unnecessary formatting by rejecting log records before they reach the formatter. If you only want ERROR logs from a specific module or only want to log messages that contain certain keywords, create a filter function that returns False for unwanted records. This saves the CPU cycles that would otherwise go into formatting a message that ultimately gets discarded. Attach filters with handler.addFilter() and remember they receive the full LogRecord object, giving you access to all attributes for precise filtering logic.

Final Words

The python logging formatter example patterns shown here cover everything from basic StreamHandler setups to custom formatter classes and production-ready JSON logging.

Start with simple format strings and logging.Formatter() for quick debugging. When your application grows, add multiple handlers with different formats for console and file output. Production systems benefit from structured JSON formatters paired with contextual information and proper security filtering.

Pick the approach that matches your current needs. You can always refactor to more sophisticated configurations as your logging requirements evolve.

Most bugs get fixed faster when your format strings include timestamps, line numbers, and function names from the start.

FAQ

Q: What does a basic Python logging formatter implementation look like?

A: A basic Python logging formatter implementation creates a logging.Formatter object with a format string containing placeholders like %(asctime)s, %(levelname)s, and %(message)s, then attaches it to a handler using handler.setFormatter(). This produces structured log output with timestamps and severity levels.

Q: What are the most common LogRecord attributes used in format strings?

A: The most common LogRecord attributes in format strings are %(asctime)s for timestamps, %(levelname)s for severity level, %(message)s for the actual log message, %(name)s for logger name, %(filename)s for source file, %(lineno)d for line number, and %(funcName)s for function name.

Q: How do I customize timestamp format in Python logging?

A: Customize timestamp format in Python logging by passing the datefmt parameter to logging.Formatter() with strftime format codes. For example, datefmt=’%Y-%m-%d %H:%M:%S’ produces ISO 8601 format, while datefmt=’%I:%M:%S %p’ creates 12-hour clock format with AM/PM.

Q: When should I create a custom formatter class instead of using logging.Formatter?

A: Create a custom formatter class when you need specialized formatting beyond standard placeholders, such as adding contextual thread information, custom fields, specialized exception handling, or conditional formatting based on log record attributes. Extend logging.Formatter and override the format() method.

Q: Can I use different formatters for console and file handlers?

A: Yes, you can use different formatters for console and file handlers by creating separate formatter objects and attaching each to its respective handler using setFormatter(). Console handlers often use simplified formats while file handlers include detailed information like timestamps, line numbers, and function names.

Q: What is basicConfig() and when should I use it?

A: basicConfig() is the simplest logging configuration method that sets up format, level, and output destination in one call. Use it for quick setups and small projects, but note it only works once before any logging functions are called and doesn’t support complex multi-handler configurations.

Q: How do I configure handlers with different log levels?

A: Configure handlers with different log levels by calling setLevel() on each handler after creation. The logger’s level acts as a minimum threshold, and handlers filter messages independently. For example, set a console handler to WARNING while a file handler captures DEBUG messages.

Q: What are the advantages of JSON formatters for production logging?

A: JSON formatters produce machine-readable structured logs that are easily parsed by log aggregation tools and monitoring systems. They’re ideal for microservices architectures and production environments where logs need automated analysis, searching, and correlation across distributed systems.

Q: How do I install and use python-json-logger?

A: Install python-json-logger using pip install python-json-logger, then import JsonFormatter and attach it to a handler. The formatter automatically converts log records into JSON objects with fields like timestamp, level, message, and any custom fields you add for structured logging.

Q: What contextual information should I include in production log formatters?

A: Include contextual information like request IDs, user identifiers, correlation IDs, module names, line numbers, and function names in production log formatters. This helps track requests across services and quickly identify error locations during debugging without compromising security.

Q: What data should never be included in log messages?

A: Never include sensitive data like passwords, API keys, authentication tokens, credit card numbers, or personally identifiable information (PII) in log messages. This violates security best practices and regulatory requirements like GDPR and HIPAA, and creates security vulnerabilities.

Q: How do I handle multiline logs and exception formatting?

A: Handle multiline logs and exception formatting by using logging.exception() for automatic stack trace capture at ERROR level, or add exc_info=True to any logging call. The formatter automatically includes traceback information without requiring manual exception string formatting.

Q: What is lazy evaluation in logging and why does it matter?

A: Lazy evaluation in logging means using logger.debug(“Value: %s”, value) instead of f-string or string concatenation like logger.debug(f”Value: {value}”). This prevents expensive string formatting operations when the log level filters out the message, improving performance.

Q: How can I optimize formatter performance in high-volume logging scenarios?

A: Optimize formatter performance by using lazy evaluation syntax, avoiding expensive operations in custom format methods, keeping format strings simple, and using filters to prevent unnecessary formatting of messages that won’t be output based on log level thresholds.

Q: Can I use filters to control which messages get formatted?

A: Yes, filters receive the LogRecord object and return Boolean values to control which messages proceed to formatting and output. Attach filters using handler.addFilter() to prevent expensive formatting operations on messages that don’t meet specific criteria before they reach the formatter.

Q: What are the three format string styles available in Python logging?

A: The three format string styles in Python logging are “%” (printf-style, the default), “$” (Template style), and “{” (f-string style). Specify the style parameter when creating the formatter, though most developers use the default % style for compatibility.