Skip to content

Narrow Overly Broad Exception Handlers #1225

@kunal-10-cloud

Description

@kunal-10-cloud

Problem Statement

The codebase contains four locations that catch the overly broad except Exception clause. This anti-pattern masks real bugs by silently swallowing unexpected errors, making debugging significantly more difficult.

Affected Locations:

File Line Context Current Behavior
malariagen_data/anoph/base.py 137 Filesystem initialization Catches all exceptions from _init_filesystem()
malariagen_data/anoph/base.py 146 Config file loading Catches all exceptions from json.load()
malariagen_data/anoph/sample_metadata.py 1867 Pandas query evaluation Catches all exceptions from data.eval()
malariagen_data/anoph/map_params.py 45 Basemap provider initialization Catches all exceptions from provider functions

Impact

Debuggability Issues

When unexpected errors occur:

  • Real bugs are silently caught and re-raised generically, losing critical stack trace context
  • Hours spent debugging because the true root cause is masked
  • Difficult to distinguish between expected and unexpected failures

Example Scenario

If a typo or logic error occurs inside _init_filesystem() (e.g., a NameError or AttributeError):

  • Current behavior: silently caught, wrapped in generic IOError
  • Correct behavior: should propagate, revealing the actual bug

Current Anti-Pattern

try:
    self._fs, self._base_path = _init_filesystem(self._url, **storage_options)
except Exception as exc:  # ← Catches EVERYTHING
    raise IOError(...) from exc

Solution Approach

Replace each bare except Exception with a specific tuple of exception types that the wrapped code is actually designed to raise.

Expected Exceptions by Location:

  1. base.py:137(OSError, ImportError)

    • OSError: filesystem/connection/auth failures
    • ImportError: missing auth modules
  2. base.py:146(OSError, json.JSONDecodeError)

    • OSError: file read failures
    • JSONDecodeError: malformed JSON in config
  3. sample_metadata.py:1867(KeyError, NameError, SyntaxError, TypeError, AttributeError)

    • All expected errors from pandas DataFrame.eval()
  4. map_params.py:45(ImportError, ModuleNotFoundError, AttributeError)

    • Import failures and missing provider attributes

Benefits

  • Prevents Silent Failures — Unexpected errors bubble up immediately with full context
  • Improves Debuggability — Stack traces show real root causes, not generic wrappers
  • Explicit Intent — Exception types clearly document what errors are expected at each location
  • Zero Behavior Change — Expected errors handled identically, only unintended exceptions propagate
  • Minimal Code Diff — 4 lines changed, no logic alterations

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions