Nested parallelism: NESL and ADL
These languages exploit arbitrary data parallelism
- sparse linear algebra, for example
apply-to-all or map describes the parallelism
Segmented scans and reductions are also available
Both need hardware or software multithreading
- to schedule the heterogeneous work in each node
The data distribution approach varies
- NESL linearizes the data to one vector and blocks it
- this strategy sometimes violates the owner-computes rule
- ADL uses programmer-supplied partition functions
- these can vary as the computation progresses