.. Copyright (c) 2024 Lawrence Livermore National Security, LLC and other
   HYPRE Project Developers. See the top-level COPYRIGHT file for details.

   SPDX-License-Identifier: (MIT)

.. _PythonInterface:

Python Interface
================

The Python interface provides thin bindings to the hypredrive C library. It accepts
solver options as Python dictionaries, YAML strings, or YAML files; assembles sparse
matrices from CSR data or SciPy CSR matrices; runs the solve lifecycle; and returns the
local solution as a NumPy array.

The heavy lifting still happens in libHYPREDRV and HYPRE. The Python layer mainly handles
input normalization, dtype checks, YAML conversion, and result extraction.

Prerequisites
-------------

- Python 3.9 or newer.
- NumPy.
- Cython 3.0 or newer when building from source.
- An MPI implementation with compiler wrappers available at build time.
- ``mpi4py`` for distributed solves from Python.
- SciPy when passing ``scipy.sparse.csr_matrix`` objects directly.

Installation
------------

There are three install modes:

- GitHub Actions wheel artifacts for quick host-only installs with a compatible MPI runtime;
- source installs against an installed or build-tree libHYPREDRV;
- in-tree CMake builds for developers and CI.

Source builds keep Python packaging independent from ordinary C builds while still
linking against libHYPREDRV.

Source install:

.. code-block:: bash

   python -m pip install ./interfaces/python

When run from a full hypredrive checkout, this automatically builds and bundles
the in-tree HYPREDRV/HYPRE libraries. When run from a standalone source
distribution, CMake falls back to finding an installed MPI-enabled
HYPREDRV/HYPRE stack. Binary MPI wheels are currently GitHub Actions artifacts,
not PyPI or TestPyPI packages.

Against an installed hypredrive:

.. code-block:: bash

   cmake --install build --prefix $HOME/opt/hypredrive
   pip install ./interfaces/python \
     --config-settings=cmake.define.HYPREDRV_PYTHON_BUNDLE_CORE=OFF \
     --config-settings=cmake.define.CMAKE_PREFIX_PATH=$HOME/opt/hypredrive

Python source distributions are intended for downstream packagers and developer
environments where ``HYPREDRV`` and ``HYPRE`` are discoverable by CMake. They are not
self-contained PyPI-style source packages for systems without the C library stack.

Against an in-tree development build:

.. code-block:: bash

   cmake -S . -B build -DBUILD_SHARED_LIBS=ON -DHYPREDRV_ENABLE_TESTING=OFF
   cmake --build build --parallel
   pip install -e ./interfaces/python \
     --config-settings=cmake.define.HYPREDRV_DIR=$PWD/build

The top-level CMake build can also build the Python extension directly:

.. code-block:: bash

   cmake -S . -B build -DHYPREDRV_ENABLE_PYTHON=ON
   cmake --build build --target _core --parallel

This mode is intended for developer and CI builds. It requires Python, NumPy, and Cython
at configure/build time, so ``HYPREDRV_ENABLE_PYTHON`` is disabled by default.

Experimental wheel artifacts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The project CI can build experimental Python wheel artifacts for Linux and macOS.
These wheels bundle host-only ``libHYPREDRV`` and ``libHYPRE`` inside the Python
package, but they do not bundle MPI.

On pull requests, the wheel workflow runs only when the PR has the
``Run Python Wheels`` label. It can also be started manually with
``workflow_dispatch``.

Download a wheel artifact from the GitHub Actions ``Python Wheels`` workflow
run first. GitHub stores artifacts as zip files, so unzip the artifact before
installing the wheel into a virtual environment on a machine with a compatible
MPI runtime:

.. code-block:: bash

   python -m venv .venv
   source .venv/bin/activate

   unzip hypredrive-wheels-*.zip -d wheelhouse
   python -m pip install wheelhouse/hypredrive-*.whl

Wheel artifacts are built by MPI flavor. Use an ``mpich`` wheel with an
MPICH-compatible runtime and an ``openmpi`` wheel with an OpenMPI-compatible
runtime.

Use a source install instead for custom HYPRE builds, GPU support,
BIGINT/MIXEDINT, vendor MPI stacks, or downstream-packager control over shared
libraries.

At runtime, the package records how it was built:

.. code-block:: python

   import hypredrive as hd
   print(hd.BUILD_INFO)

Optional dependencies can be installed through package extras:

.. code-block:: bash

   pip install ./interfaces/python[mpi]
   pip install ./interfaces/python[scipy]
   pip install -e ./interfaces/python[test]

``mpi4py`` is optional for package import and serial solves, but it is the
supported MPI boundary for distributed Python use. HypreDrive accepts
``mpi4py.MPI.Comm`` objects and forwards the underlying communicator to the C
library instead of exposing its own Python MPI wrapper.

Quick start
-----------

One-shot solve
~~~~~~~~~~~~~~

.. code-block:: python

   import numpy as np
   import scipy.sparse as sp
   import hypredrive as hd

   n = 64
   diag_main = 2.0 * np.ones(n)
   diag_off = -np.ones(n - 1)
   A = sp.diags([diag_off, diag_main, diag_off], [-1, 0, 1], format="csr")
   b = np.ones(n)

   options = hd.configure(
       solver="pcg",
       preconditioner="amg",
       pcg={"max_iter": 100, "relative_tol": 1.0e-8},
       amg={"print_level": 0},
   )

   result = hd.solve(A, b, options=options)

   print("solution norm:", result.solution_norm)
   print("first entries:", result.x[:5])

Reusable driver
~~~~~~~~~~~~~~~

Use ``HypreDrive`` when one process needs to solve multiple related systems and reuse the
driver lifecycle.

.. code-block:: python

   import hypredrive as hd

   with hd.HypreDrive(options="my_config.yaml") as drv:
       for step in range(num_steps):
           drv.set_matrix_from_csr(build_matrix(step))
           drv.set_rhs(build_rhs(step))
           drv.solve()
           x = drv.get_solution()

Distributed solve with MPI
~~~~~~~~~~~~~~~~~~~~~~~~~~

When using ``mpi4py``, each rank provides its local CSR slab. Row bounds are global and
inclusive; column indices are global.

.. code-block:: python

   from mpi4py import MPI
   import hypredrive as hd

   comm = MPI.COMM_WORLD
   rank = comm.Get_rank()
   size = comm.Get_size()

   indptr, col_indices, data, rhs, row_start, row_end = build_local_slab(rank, size)

   with hd.HypreDrive(options=opts, comm=comm) as drv:
       drv.set_matrix_from_csr(
           indptr,
           col_indices,
           data,
           row_start=row_start,
           row_end=row_end,
       )
       drv.set_rhs(rhs)
       drv.solve()
       x_local = drv.get_solution()

CSR input details
-----------------

``set_matrix_from_csr`` accepts either a SciPy CSR matrix or the raw
``(indptr, col_indices, data)`` arrays. With raw arrays, ``row_start`` and
``row_end`` are required and describe the inclusive global row range owned by the
rank. ``indptr`` has length ``nrows + 1`` where
``nrows = row_end - row_start + 1``; ``col_indices`` contains global column
indices; and all input buffers may be released after the call returns because HYPRE
copies the data during IJ assembly.

Passing a SciPy sparse matrix with no explicit row range means single-rank/full-local
assembly. CSR inputs are used directly; other SciPy sparse formats are converted to
CSR first. Passing a SciPy sparse matrix with ``row_start`` and ``row_end`` means the
matrix is this rank's local slab and must have
``shape[0] == row_end - row_start + 1``.

Empty local row ranges are not currently supported by the CSR/array API; every
participating MPI rank must own at least one row.

Configuration input
-------------------

Anywhere ``options`` is accepted, pass one of:

.. list-table::
   :header-rows: 1

   * - Input
     - Behavior
   * - ``dict``
     - Converted to YAML in memory and parsed by hypredrive.
   * - ``str`` containing a newline
     - Treated as a YAML document.
   * - ``str`` or ``pathlib.Path`` naming an existing file
     - File contents are loaded and parsed.
   * - ``None``
     - Uses the Python binding's minimal default YAML.

The accepted YAML keys and solver/preconditioner options are the same as the CLI; see
:ref:`InputFileStructure`.

Testing
-------

After installing the package in editable mode with test dependencies:

.. code-block:: bash

   pip install -e ./interfaces/python[test]
   python -m pytest interfaces/python/tests/test_solve_serial.py -v

MPI tests must be launched under an MPI process manager:

.. code-block:: bash

   mpirun -np 2 python -m pytest \
     interfaces/python/tests/test_solve_mpi.py \
     interfaces/python/tests/laplacian/test_example_mpi.py -v

Tests that require the native extension or ``mpi4py`` skip when those optional runtime
components are unavailable.

Current limitations
-------------------

- The Python interface currently targets real-valued solves.
- Solution data is copied back to host NumPy arrays.
- GPU/device execution is not exposed as a Python-native data path.
- Result metadata is intentionally small; the one-shot API currently exposes the solution
  array and solution norm.
- Distributed Python solves require ``mpi4py`` and use ``Comm.py2f()`` plus the
  C-side ``MPI_Comm_f2c`` bridge.
- Python examples live under ``interfaces/python/examples``. ``laplacian/laplacian.py``
  is the MPI-capable 3D example; ``laplacian/laplacian2d_seq.py`` is the serial
  2D example; ``darcy/darcy_mixed.py`` is the mixed-form Darcy example.