Skip to content

Seed random in test suite to eliminate CI flakiness #1223

@Yashsingh045

Description

@Yashsingh045

Problem

The test suite utilizes random.choice(), random.sample(), and random.randint() approximately 400 times across various test files, yet random.seed() is never called globally. This omission leads to intermittent CI failures due to non-deterministic random draws.

Evidence

  • grep -rn "random\.choice\|random\.sample\|random\.randint" tests/ | wc -l: 399
  • grep -rn "random\.seed" tests/ | wc -l: 1 (limited scope)

Proposed Fix

Implement a global seeding mechanism in a top-level tests/conftest.py to seed both the random and numpy.random modules with a fixed value (e.g., 42). This will ensure that all test runs are reproducible and eliminate false CI failures.

Impact

  • Eliminates intermittent CI failures across all Pull Requests.
  • Improves developer experience by providing consistent test results.
  • Facilitates easier debugging of test failures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions