Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
All dates, times and locations of tech program events or other scheduled plans are subject to change. Please check back regularly to ensure you view the most up-to-date version of the schedule.
View analytic
Thursday, July 30 • 10:30am - 11:00am
Integrating Apache Spark Into PBS-Based HPC Environments

Sign up or log in to save this to your schedule and see who's attending!

This paper describes an effort at the University of Tennessee’s National Institute for Computational Sciences (NICS) to integrate Apache Spark into the widely used TORQUE HPC batch environment. The similarities and differences between the execution of a Spark program and that of an MPI program on a cluster are used to motivate how to implement Spark/TORQUE integration. An implementation of this integration, pbs-spark-submit, is described, including demonstrations of functionality on two HPC clusters and a large shared-memory system.


Thursday July 30, 2015 10:30am - 11:00am
Majestic F

Attendees (14)