Skip to content

optimization='auto' cannot select metal or cuda_kernel unless torch is installed; availability probes never check for a device #304

Description

@Ramdam17

1. Platform detection depends entirely on torch.

MPS_AVAILABLE and CUDA_AVAILABLE are derived only from torch (hypyp/sync/base.py:53-61):

try:
    import torch
    TORCH_AVAILABLE = True
    MPS_AVAILABLE = torch.backends.mps.is_available()
    CUDA_AVAILABLE = torch.cuda.is_available()
except ImportError:
    TORCH_AVAILABLE = False
    MPS_AVAILABLE = False
    CUDA_AVAILABLE = False

_resolve_auto picks its platform from those two flags (base.py:414-418). So on a machine with CuPy and an NVIDIA GPU but no torch, optimization='auto' warns "No GPU available" and falls back to numba — even though cuda_kernel is present and working. Same for a Mac with pyobjc-framework-Metal but no torch: 'auto' can never reach the Metal kernels.

The two custom-kernel backends are therefore unreachable via 'auto' without installing torch, which is not a dependency of either. pip install "hypyp[cupy]" alone gets you a CPU fallback.

2. Availability probes are import-only and never touch the device.

hypyp/sync/kernels/__init__.py:10-21 sets both flags purely on import success:

  • METAL_AVAILABLE = True iff import Metal succeeds — MTLCreateSystemDefaultDevice() is never called. If it returns None (headless/VM/unsupported GPU), the first compute() dies on AttributeError: 'NoneType' object has no attribute 'newLibraryWithSource_options_error_'.
  • CUPY_AVAILABLE = True iff import cupy succeeds — cupy.cuda.runtime.getDeviceCount() is never called. On a cluster login node with CuPy installed but no GPU allocated (e.g. Narval), optimization='cuda_kernel' promises a GPU it does not have, and the 9 CUDA tests run and error at kernel launch instead of skipping.

Note the asymmetry: CUDA_AVAILABLE does a real runtime check via torch.cuda.is_available(), but CUPY_AVAILABLE does not.

Suggested fix: derive platform availability independently of torch (probe Metal device creation and cupy.cuda.runtime.getDeviceCount() directly), and make both flags reflect an actually usable device rather than a successful import.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions