Modernizing HPX: Architectural stdexec Migration and Subsystem Integration

Project: Modernizing HPX: Architectural stdexec Migration and Subsystem Integration
Contributor: Pratyksh Gupta (@guptapratykshh)
Organization: STE||AR Group
Mentors: Hartmut Kaiser (@hkaiser) and Isidoros Tsaousis-Seiras (@isidorostsa)

---

Project Goals

HPX is transitioning its asynchronous execution model toward the C++ Sender/Receiver (P2300) design standardized for C++26. While NVIDIA's stdexec was already integrated as a required dependency, several critical HPX boundaries still relied on transitional scaffolding: compatibility traits, legacy tag_invoke customizations, future-to-sender adapters, and subsystem-specific execution paths for MPI and CUDA.

This project focused on the architectural phase of that migration. The core goals were to:

• Remove remaining stdexec-era compatibility and dead fallback code.
• Modernize and stabilize the make_future and keep_future boundaries.
• Integrate HPX run-loop and synchronization behavior via standard Sender/Receiver environments and completion domains.
• Enable MPI and CUDA adapters to participate natively in sender pipelines.
• Measure the performance overhead of the future/sender boundary and optimize hot execution paths.
• Extend scheduler-backed Sender/Receiver support across parallel algorithms.

A primary guiding principle was maintaining backward compatibility: keeping public HPX APIs stable while making the underlying runtime native to Sender/Receiver semantics.

---

Key Contributions & Implementation

1. Cleaning Up the Execution Foundation

The first phase eradicated obsolete execution branches and compatibility layers remaining from the mandatory-stdexec transition:

• PR #7123: Cleaned up stdexec-only paths and modernized affected unit test suites.
• PR #7244: Stripped dead traits and test guards while migrating the asynchronous MPI receiver to native P2300 participation.
• PR #7246: Modernized HPX's stdexec dependency build configuration to leverage standard FetchContent_MakeAvailable flows.
• PR #7292: Modernized operation-state customization by replacing legacy tag_invoke(connect_t) friend definitions with member connect() methods, aligning HPX sender types with modern stdexec practices.

Key Insight: Modernizing templates is not a purely mechanical conversion from SFINAE to C++20 requires clauses. Constraints must accurately model runtime value categories and function invocations; preserving safe participation semantics remains far more critical than syntax replacement alone.

2. Stabilizing make_future and Run-Loop Integration

The bridge between classic HPX futures and Sender/Receiver execution (make_future) was historically fragile due to internal couplings with stdexec run-loop internals (such as loop pointer state):

• PR #7247: Encapsulated private run-loop access within isolated detail helpers instead of scattering it throughout execution headers.
• PR #7248: Introduced guards against invalid sender compositions that previously resulted in silent thread hangs.
• PR #7282: Refined SFINAE guards and resolved compiler diagnostics across toolchains.
• PR #7315: Consolidated sender-to-future boundary conversions and cleaned up lifetime handling for nested execution contexts.

---

Future Work & Acknowledgements

With the core architectural migration established, the foundation is set to complete P2300 integration across remaining distributed components and fine-tune compiler throughput across heterogeneous hardware targets.

I would like to express my sincere gratitude to my mentors Hartmut Kaiser and Isidoros Tsaousis-Seiras, as well as the entire STE||AR Group community, for their continuous guidance, architectural insights, and thorough reviews throughout Google Summer of Code.