Implementing Unit Testing in ReadmeGenie

In this post, I’ll walk through the journey of implementing unit testing, handling complex configuration challenges, and introducing robust code coverage in ReadmeGenie. From initial test design to setting up pre-commit hooks, this process involved a range of improvements in code quality, reliability, and developer workflow.

1. Setting Up the Testing Environment

To start, I chose unittest as the primary framework for writing and executing tests. Python’s built-in unittest provides a structured approach for defining test cases, and its integration with mock makes it ideal for testing complex configurations and API calls.

I created a dedicated test runner (tests/test_runner.py) for automatic discovery and execution of all test files in the tests/ directory:

# tests/test_runner.py
import unittest

if __name__ == "__main__":
    loader = unittest.TestLoader()
    suite = loader.discover(start_dir="tests", pattern="test_*.py")
    runner = unittest.TextTestRunner(verbosity=2)
    runner.run(suite)

This setup ensures that running python tests/test_runner.py will automatically load and run all test files, making it easy to validate the project’s overall functionality.

2. Structuring the Unit Tests

The ReadmeGenie project required comprehensive testing for several components:

• Argument Parsing: Verifying correct parsing of command-line arguments and handling of default values.
• Configuration and Environment Handling: Testing for proper retrieval of API keys and handling errors when they’re missing.
• API Calls: Using mocks to simulate API requests to avoid real API calls in tests.
• Helper Functions: Testing utility functions, such as file reading and README processing.

Each test file is named according to the module it tests (e.g., test_parse_arg.py for argument parsing and test_model.py for model functions), ensuring a clear, maintainable structure.

3. Biggest Challenge: Configuring test_loadConfig.py

Setting up test_loadConfig.py turned out to be the most challenging part of this project. Initially, I encountered persistent issues related to environment variables and file path checks. Since load_config() is intended to handle various configuration sources (e.g., environment variables, .env files, JSON, and TOML files), the tests required extensive mocking to simulate these environments accurately.

Errors and Solutions in test_loadConfig.py

The primary issues involved:

• Environment Variable Conflicts: Existing environment variables sometimes interfered with mocked values. Using @patch.dict("os.environ", {}, clear=True), I cleared the environment variables within the test scope to ensure consistent results.

• File Path Checks: Since load_config() checks for file existence, I used os.path.exists to simulate scenarios where configuration files were present or absent.

• Mocking open and toml.load: These required precise mocking to handle cases of missing, empty, or populated configuration files. Using mock_open with patch on toml.load, I effectively simulated each situation.

After resolving these issues, test_loadConfig.py now covers three main scenarios:

1. Empty Configuration: Tests that an empty configuration is returned when no environment variables or files are found.
2. Valid Configuration Data: Tests that the api_key is correctly retrieved from the configuration file.
3. File Not Found: Simulates a missing file, expecting an empty configuration to be returned.

Here’s the final version of test_loadConfig.py:

import unittest
from unittest.mock import mock_open, patch
from loadConfig import load_config

class TestLoadConfig(unittest.TestCase):
    @patch.dict("os.environ", {}, clear=True)
    @patch("loadConfig.os.getenv", side_effect=lambda key, default=None: default)
    @patch("loadConfig.os.path.exists", return_value=False)
    @patch("builtins.open", new_callable=mock_open, read_data="{}")
    @patch("loadConfig.toml.load", return_value={})
    def test_load_config_empty_file(self, mock_toml_load, mock_open_file, mock_exists, mock_getenv):
        config = load_config()
        self.assertEqual(config, {})

    @patch.dict("os.environ", {}, clear=True)
    @patch("loadConfig.os.getenv", side_effect=lambda key, default=None: default)
    @patch("loadConfig.os.path.exists", return_value=True)
    @patch("builtins.open", new_callable=mock_open, read_data='{"api_key": "test_key"}')
    @patch("loadConfig.toml.load", return_value={"api_key": "test_key"})
    def test_load_config_with_valid_data(self, mock_toml_load, mock_open_file, mock_exists, mock_getenv):
        config = load_config()
        self.assertEqual(config.get("api_key"), "test_key")

    @patch.dict("os.environ", {}, clear=True)
    @patch("loadConfig.os.getenv", side_effect=lambda key, default=None: default)
    @patch("loadConfig.os.path.exists", return_value=False)
    @patch("builtins.open", side_effect=FileNotFoundError)
    @patch("loadConfig.toml.load", return_value={})
    def test_load_config_file_not_found(self, mock_toml_load, mock_open_file, mock_exists, mock_getenv):
        config = load_config()
        self.assertEqual(config, {})

4. Code Coverage Analysis

With our tests in place, we focused on measuring and improving coverage using coverage.py. By setting an 80% threshold, we aimed to ensure all critical parts of the code are tested.

Tool Configuration for Coverage

I configured coverage.py with the following settings in pyproject.toml:

[tool.coverage.run]
source = [""]
branch = true
omit = ["tests/*"]

[tool.coverage.report]
show_missing = true
fail_under = 75

This configuration includes branch coverage, highlights missing lines, and enforces a minimum 75% coverage threshold.

Pre-Commit Coverage Check

To integrate this into the development workflow, I added a pre-commit hook to ensure code coverage is checked on each commit. If coverage falls below 75%, the commit is blocked, prompting developers to improve coverage before proceeding:

- repo: local
  hooks:
    - id: check-coverage
      name: Check Coverage
      entry: bash -c "coverage run --source=. -m unittest discover -s tests && coverage report -m --fail-under=75"
      language: system

5. Current Coverage and Opportunities for Improvement

Our recent coverage report shows:

Name                   Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------
loadConfig.py              8      0      2      0   100%
logging_config.py         19      0      2      0   100%
models/__init__.py         0      0      0      0   100%
models/cohere_api.py      12      0      0      0   100%
models/groq_api.py        12      0      0      0   100%
models/model.py           64     13     16      3    78%   25-26, 44-46, 70-74, 84-87
readme_genie.py           37     16      4      1    54%   20-24, 66-86, 90
------------------------------------------------------------------
TOTAL                    152     29     24      4    79%

While coverage is strong in some areas (e.g., loadConfig.py at 100%), there are still opportunities for improvement in models/model.py and readme_genie.py. Focusing on untested branches and edge cases will be crucial to reaching our goal of 85% or higher overall coverage.

Final Thoughts

This project has taught me a lot about unit testing, mocking, and code coverage. Setting up test_loadConfig.py was a particularly valuable experience, pushing me to explore deeper levels of configuration mocking. The pre-commit hook for coverage has added a layer of quality assurance, enforcing consistent test standards.

Moving forward, I aim to refine these tests further by adding edge cases and improving branch coverage. This will not only make ReadmeGenie more robust but also lay a solid foundation for future development.