XSEDE15 has ended
All dates, times and locations of tech program events or other scheduled plans are subject to change. Please check back regularly to ensure you view the most up-to-date version of the schedule.
Back To Schedule
Thursday, July 30 • 10:30am - 11:00am
Integrating Apache Spark Into PBS-Based HPC Environments

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

This paper describes an effort at the University of Tennessee’s National Institute for Computational Sciences (NICS) to integrate Apache Spark into the widely used TORQUE HPC batch environment. The similarities and differences between the execution of a Spark program and that of an MPI program on a cluster are used to motivate how to implement Spark/TORQUE integration. An implementation of this integration, pbs-spark-submit, is described, including demonstrations of functionality on two HPC clusters and a large shared-memory system.

Thursday July 30, 2015 10:30am - 11:00am CDT
Majestic F

Attendees (0)