Towards Effortless Python Configuration Files Version Ø

Introduction

We write many Python applications that require customization via external properties or applications for which we wish to customize or affect behavior with non-hardcoded properties and/or runtime configuration properties. Various Google searches for solutions yield tutorials that point us to code samples that while practically work, do not scale appropriately for real world applications.

These set of articles document my journey through various implementations as I repeatedly refactored and reimplemented on my journey to a simple, maintainable, and easily extensible mechanism to manage application configuration properties.

Version Ø

The tutorial versions I found were simply expansions of the code snippets that the library developers provide to prove that their implementations work. While this is sufficient to provide a proof of concept these snippets do not scale in the application real world.

An example of this is the following code snippet.

import configparser

def create_config():
    config = configparser.ConfigParser()

    # Add sections and key-value pairs
    config['General'] = {
      'debug': 'True', 
      'log_level': 'info'
    }
    config['Database'] = {
      'db_name': 'example_db',
      'db_host': 'localhost', 
      'db_port': '5432'
    }

    with open('config.ini', 'w') as configfile:
        config.write(configfile)


if __name__ == "__main__":
    create_config()

While this code snippet certainly allows us to persist our configuration values it leaves us with the problem of reading these persisted values. Again, the implementation developer snippets provide us with sample code on how to retrieve these values as depicted in the following code snippets.

import configparser

def read_config():
    config = configparser.ConfigParser()

    config.read('config.ini')

    # Access values from the configuration file
    debug_mode = config.getboolean('General', 'debug')
    log_level = config.get('General', 'log_level')
    db_name = config.get('Database', 'db_name')
    db_host = config.get('Database', 'db_host')
    db_port = config.get('Database', 'db_port')

    # Return a dictionary with the retrieved values
    config_values = {
        'debug_mode': debug_mode,
        'log_level': log_level,
        'db_name': db_name,
        'db_host': db_host,
        'db_port': db_port
    }

    return config_values

if __name__ == "__main__":
    # Call the function to read the configuration file
    config_data = read_config()

    # Print the retrieved values
    print('Debug Mode',       config_data['debug_mode'])
    print('Log Level',         config_data['log_level'])
    print('Database Name', config_data['db_name'])
    print('Database Host', config_data['db_host'])
    print('Database Port', config_data['db_port'])

Conclusion

I see many issues in the above code. While it is perfectly acceptable for small scripts, the code suffers from the usage of string values, related to actual Python variable names and their potential proliferation across a large code base. While this is potentially mitigated by the use of global constants, my opinion is that this is not scaleable because it does not follow a basic software design principle espoused by Andrew Hunt and David Thomas, in their seminal book, The Pragmatic Programmer and fails the DRY principle, aka Do not Repeat Yourself.

The source code for this article is here.

See my next post that documents an initial implementation to solve some of the issues I outlined.