Merge branch 'master' into feat-allow-region-input-snp-allele-frequencies

Author: joshitha1808

CONTRIBUTING.md

@@ -12,9 +12,10 @@ This package provides Python tools for accessing and analyzing genomic data from
 
 You'll need:
 
-- Python 3.10.x (CI-tested version)
-- [Poetry](https://python-poetry.org/) for dependency management
-- [Git](https://git-scm.com/) for version control
+- [pipx](https://python-poetry.org/) for installing Python tools
+- [git](https://git-scm.com/) for version control
+
+Both of these can be installed using your distribution's package manager or [Homebrew](https://brew.sh/) on Mac.
 
 ### Initial setup
 
@@ -33,16 +34,25 @@ You'll need:
    git remote add upstream https://github.com/malariagen/malariagen-data-python.git
    ```
 
-3. **Install Poetry** (if not already installed)
+3. **Install Poetry**
 
    ```bash
    pipx install poetry
    ```
 
-4. **Install the project and its dependencies**
+4. **Install Python 3.12**
+
+   Python 3.12 is tested in the CI-system and is the recommended version to use.
+
+   ```bash
+   poetry python install 3.12
+   ```
+
+5. **Install the project and its dependencies**
 
    ```bash
-   poetry install
+   poetry env use 3.12
+   poetry install --extras dev
    ```
 
    **Recommended**: Use `poetry run` to run commands inside the virtual environment:
@@ -71,7 +81,7 @@ You'll need:
    python script.py
    ```
 
-5. **Install pre-commit hooks**
+6. **Install pre-commit hooks**
 
    ```bash
    pipx install pre-commit
@@ -107,16 +117,40 @@ You'll need:
 
 4. **Run tests locally**
 
-   Fast unit tests (no external data access):
+   Fast unit tests using simulated data (no external data access):
 
    ```bash
-   poetry run pytest -v tests/anoph
+   poetry run pytest -v tests --ignore tests/integration
    ```
 
-   All unit tests (requires setting up credentials for legacy tests):
+   To run integration tests which read data from GCS, you'll need to [request access to MalariaGEN data on GCS](https://malariagen.github.io/vector-data/vobs/vobs-data-access.html).
+
+   Once access has been granted, [install the Google Cloud CLI](https://cloud.google.com/sdk/docs/install). E.g., if on Linux:
 
    ```bash
-   poetry run pytest -v tests --ignore tests/integration
+   ./install_gcloud.sh
+   ```
+
+   You'll then need to obtain application-default credentials, e.g.:
+
+   ```bash
+   ./google-cloud-sdk/bin/gcloud auth application-default login
+   ```
+
+   Once this is done, you can run integration tests:
+
+   ```bash
+   poetry run pytest -v tests/integration
+   ```
+
+   Tests will run slowly the first time, as data required for testing will be read from GCS. Subsequent runs will be faster as data will be cached locally in the "gcs_cache" folder.
+
+6. **Run typechecking**
+
+   Run static typechecking with mypy:
+
+   ```bash
+   poetry run mypy malariagen_data tests --ignore-missing-imports
    ```
 
 5. **Check code quality**
@@ -150,7 +184,7 @@ ruff format .
 - **Fast tests**: Unit tests should use simulated data when possible (see `tests/anoph/`)
 - **Integration tests**: Tests requiring GCS data access are slower and run separately
 
-Run type checking with:
+Run dynamic type checking with:
 
 ```bash
 poetry run pytest -v tests --typeguard-packages=malariagen_data,malariagen_data.anoph

LINUX_SETUP.md

@@ -0,0 +1,67 @@
+# Developer setup (Linux)
+
+To get setup for development, see [this video if you prefer VS Code](https://youtu.be/zddl3n1DCFM), or [this older video if you prefer PyCharm](https://youtu.be/QniQi-Hoo9A), and the instructions below.
+
+## 1. Fork and clone this repo
+```bash
+git clone git@github.com:[username]/malariagen-data-python.git
+cd malariagen-data-python
+```
+
+## 2. Install Python
+```bash
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt install python3.10 python3.10-venv
+```
+
+## 3. Install pipx and poetry
+```bash
+python3.10 -m pip install --user pipx
+python3.10 -m pipx ensurepath
+pipx install poetry
+```
+
+## 4. Create and activate development environment
+```bash
+poetry install
+poetry shell
+```
+
+## 5. Install pre-commit hooks
+```bash
+pipx install pre-commit
+pre-commit install
+```
+
+Run pre-commit checks manually:
+```bash
+pre-commit run --all-files
+```
+
+## 6. Run tests
+
+Run fast unit tests using simulated data:
+```bash
+poetry run pytest -v tests/anoph
+```
+
+## 7. Google Cloud authentication (for legacy tests)
+
+To run legacy tests which read data from GCS, you'll need to [request access to MalariaGEN data on GCS](https://malariagen.github.io/vector-data/vobs/vobs-data-access.html).
+
+Once access has been granted, [install the Google Cloud CLI](https://cloud.google.com/sdk/docs/install):
+```bash
+./install_gcloud.sh
+```
+
+Then obtain application-default credentials:
+```bash
+./google-cloud-sdk/bin/gcloud auth application-default login
+```
+
+Once authenticated, run legacy tests:
+```bash
+poetry run pytest --ignore=tests/anoph -v tests
+```
+
+Tests will run slowly the first time, as data will be read from GCS and cached locally in the `gcs_cache` folder.

MACOS_SETUP.md

@@ -0,0 +1,77 @@
+# Developer setup (macOS)
+
+The Linux setup guide is available in [LINUX_SETUP.md](LINUX_SETUP.md). If you are on macOS, follow these steps instead.
+
+## 1. Install Miniconda
+
+Download and install Miniconda for macOS from https://docs.conda.io/en/latest/miniconda.html.
+Choose the Apple Silicon installer if you have an Apple Silicon Mac, or the Intel installer otherwise. You can check with:
+```bash
+uname -m
+# arm64 = Apple Silicon, x86_64 = Intel
+```
+
+After installation, close and reopen your terminal for conda to be available.
+
+## 2. Create a conda environment
+
+The package requires Python `>=3.10, <3.13`. Python 3.13+ is not currently supported.
+```bash
+conda create -n malariagen python=3.11
+conda activate malariagen
+```
+
+## 3. Fork and clone this repo
+
+Fork the repository on GitHub, then clone your fork:
+```bash
+git clone git@github.com:[username]/malariagen-data-python.git
+cd malariagen-data-python
+pip install -e ".[dev]"
+```
+
+## 4. Install pre-commit hooks
+```bash
+pre-commit install
+```
+
+Run pre-commit checks manually:
+```bash
+pre-commit run --all-files
+```
+
+## 5. Run tests
+
+Run fast unit tests using simulated data:
+```bash
+pytest -v tests/anoph
+```
+
+## 6. Google Cloud authentication (for legacy tests)
+
+To run legacy tests which read data from GCS, you'll need to [request access to MalariaGEN data on GCS](https://malariagen.github.io/vector-data/vobs/vobs-data-access.html).
+
+Once access has been granted, install the Google Cloud CLI:
+```bash
+brew install google-cloud-sdk
+```
+
+Then authenticate:
+```bash
+gcloud auth application-default login
+```
+
+This opens a browser — log in with any Google account.
+
+Once authenticated, run legacy tests:
+```bash
+pytest --ignore=tests/anoph -v tests
+```
+
+Tests will run slowly the first time, as data will be read from GCS and cached locally in the `gcs_cache` folder.
+
+## 7. VS Code terminal integration
+
+To use the `code` command from the terminal:
+
+Open VS Code → `Cmd + Shift + P` → type `Shell Command: Install 'code' command in PATH` → press Enter.

README.md

@@ -44,8 +44,11 @@ for release notes.
 
 ## Developer setup
 
-To get setup for development, see [this video if you prefer VS Code](https://youtu.be/zddl3n1DCFM), or [this older video if you prefer PyCharm](https://youtu.be/QniQi-Hoo9A), and the instructions below.
+To get setup for development, see [this video if you prefer VS Code](https://youtu.be/zddl3n1DCFM), or [this older video if you prefer PyCharm](https://youtu.be/QniQi-Hoo9A).
 
+For detailed setup instructions, see:
+- [Linux setup guide](LINUX_SETUP.md)
+- [macOS setup guide](MACOS_SETUP.md)
 Detailed instructions can be found in the [Contributors guide](https://github.com/malariagen/malariagen-data-python/blob/master/CONTRIBUTING.md).
 
 ## AI use policy and guidelines

docs/source/index.rst

@@ -92,7 +92,7 @@ Some data from MalariaGEN are subject to **terms of use** which may include an e
 public communication of any analysis results without permission from data owners. If you
 have any questions about terms of use please email support@malariagen.net.
 
-By default, this sofware package accesses data directly from the **MalariaGEN cloud data repository**
+By default, this software package accesses data directly from the **MalariaGEN cloud data repository**
 hosted in Google Cloud Storage in the US. Note that data access will be more efficient if your
 computations are also run within the same region. Google Colab provides a convenient and free
 service which you can use to explore data and run computations. If you have any questions about

malariagen_data/anoph/distance.py

@@ -365,24 +365,43 @@ def _njt(
         from scipy.spatial.distance import squareform  # type: ignore
 
         # Compute pairwise distances.
-        dist, samples, n_snps = self.biallelic_diplotype_pairwise_distances(
-            region=region,
-            n_snps=n_snps,
-            metric=metric,
-            sample_sets=sample_sets,
-            sample_indices=sample_indices,
-            site_mask=site_mask,
-            site_class=site_class,
-            inline_array=inline_array,
-            chunks=chunks,
-            cohort_size=cohort_size,
-            min_cohort_size=min_cohort_size,
-            max_cohort_size=max_cohort_size,
-            random_seed=random_seed,
-            max_missing_an=max_missing_an,
-            min_minor_ac=min_minor_ac,
-            thin_offset=thin_offset,
-        )
+        try:
+            dist, samples, n_snps_used = self.biallelic_diplotype_pairwise_distances(
+                region=region,
+                n_snps=n_snps,
+                metric=metric,
+                sample_sets=sample_sets,
+                sample_indices=sample_indices,
+                site_mask=site_mask,
+                site_class=site_class,
+                inline_array=inline_array,
+                chunks=chunks,
+                cohort_size=cohort_size,
+                min_cohort_size=min_cohort_size,
+                max_cohort_size=max_cohort_size,
+                random_seed=random_seed,
+                max_missing_an=max_missing_an,
+                min_minor_ac=min_minor_ac,
+                thin_offset=thin_offset,
+            )
+
+        except ValueError as e:
+            raise ValueError(
+                f"Unable to construct neighbour-joining tree. {e} "
+                f"This could be because the selected region does not "
+                f"contain enough polymorphic SNPs for the given sample "
+                f"sets and query parameters."
+            ) from e
+
+        # Validate enough samples for a tree.
+        n_samples = len(samples)
+        if n_samples < 3:
+            raise ValueError(
+                f"Not enough samples to construct a neighbour-joining tree. "
+                f"A minimum of 3 samples is required, but only {n_samples} "
+                f"were found for the given region and sample sets."
+            )
+
         D = squareform(dist)
 
         # anjl supports passing in a progress bar function to get progress on the