More Related Content
What's hot (20)
Viewers also liked (6)
Similar to GPU Computing With Apache Spark And Python (20)
More from Jen Aman (20)
Recently uploaded (20)
GPU Computing With Apache Spark And Python
- 1. GPU Computing With Apache Spark And Python Siu Kwan Lam Continuum Analytics
- 2. • I’m going to use Anaconda throughout this presentation. • Anaconda is a free Mac/Win/Linux Python distribution: • Based on conda, an open source package manager • Installs both Python and non-Python dependencies • Easiest way to get the software I will talk about today • https://www.continuum.io/downloads
- 3. Overview • Why Python? • Using GPU in PySpark • An example: Image registration • Accelerate: Drop-in GPU-accelerated functions • Numba: JIT Custom GPU-accelerated functions • Tips & Tricks
- 4. WHY PYTHON?
- 5. Why is Python so popular? • Straightforward, productive language for system administrators, programmers, scientists, analysts and hobbyists • Great community: • Lots of tutorial and reference materials • Vast ecosystem of useful libraries • Easy to interface with other languages
- 6. But… Python is slow!
- 7. But… Python is slow! • Pure, interpreted Python is slow. • Python excels at interfacing with other languages used in HPC: • C: ctypes, CFFI, Cython • C++: Cython, Boost.Python • FORTRAN: f2py • Secret: Most scientific Python packages put the speed critical sections of their algorithms in a compiled language.
- 8. Is there another way? • Switching languages for speed in your projects can be a little clunky • Generating compiled functions for the wide range of data types can be tedious • How can we use cutting edge hardware, like GPUs?
- 9. IMAGE REGISTRATION An example for using GPU in PySpark
- 10. Image Registration • An experiment to demonstrate GPU usage • The problem: • stitch image fragments • fragments are randomly orientated, translated and scaled. • phase-correlation for image registration • FFT heavy
- 12. Basic Algorithm • FFT 2D heavy • Expensive • Potential speedup on GPU • KMeans Clustering • Histogram as metric Group similar images Image registration within each group image set unused images new images Progress?
- 13. Setup • conda can create a local environment for Spark for you: conda create -n spark -c anaconda-cluster python=3.5 spark accelerate ipython-notebook source activate spark IPYTHON_OPTS="notebook" ./bin/pyspark # starts jupyter notebook
- 14. Performance Bottleneck • Most of the time spent in 2D FFT
- 16. Accelerate • Commercial licensed • Hardware optimized numerical functions • SIMD optimized via MKL • GPU accelerated via CUDA
- 17. CUDA Library Bindings: cuFFT >2x speedup incl. host<->device round trip on GeForce GT 650M MKL accelerated FFT
- 18. CPS with GPU drop-in • Replace numpy FFT with accelerate version
- 19. CPS with GPU drop-in • Replace numpy FFT with accelerate version CPU-GPU transfer CPU-GPU transfer
- 20. NUMBA JIT CUSTOM GPU-ACCELERATED FUNCTIONS
- 21. Numba • Opensource licensed • A Python JIT as a CPython library • Array/numerical subset • Targets CPU and GPU
- 22. Supported Platforms • Experimental support for ARMv7 (Raspberry Pi 2) OS HW SW • Windows (7 and later) • 32 and 64-bit x86 CPUs • Python 2 and 3 • OS X (10.7 and later) • CUDA-capable NVIDIA GPUs • NumPy 1.7 through 1.11 • Linux (~RHEL 5 and later) • HSA-capable AMD GPUs • Experimental support for ARMv7 (Raspberry Pi 2)
- 23. How Does Numba Work? Python Function (bytecode) Bytecode Analysis Functions Arguments Numba IR Machine Code Execute! Type Inference LLVM/NVVM JIT LLVM IR Lowering Rewrite IR Cache @jit def do_math(a, b): … >>> do_math(x, y)
- 24. Ufuncs—Map operation for ND arrays
- 25. Decorator for creating ufunc List of supported type signatures Code generation target Ufuncs—Map operation for ND arrays
- 27. Numba in Spark • Compiles to IR on client • Or not if type information is not available yet • Send IR to workers • Finalize to machine code on workers
- 28. cuFFT explicit memory transfer CPS with cuFFT + GPU ufuncs
- 29. TIPS & TRICKS
- 30. Operate in Batches • GPUs have many-cores • Best to do many similar task at once • GPU kernel launch has overhead • prefer mapPartitions, mapValues over map
- 31. Under-utilization of GPU • PySpark spawns 1 Python process per core • Only 1 CUDA process per GPU at a time • Under-utilize the GPU easily • GPU context-switching between processes
- 32. Under-utilization of GPU (Fix) • nvidia-cuda-mps-control • Originally for MPI • Allow multiple process per GPU • Reduce per-process overhead • Increase GPU utilization • 10-15% speedup in our experiment
- 33. Summary • Anaconda: • creates Spark environment for experimentation • manages Python packages for use in Spark • Accelerate: • Pre-built GPU functions within PySpark • Numba: • JIT custom GPU functions within PySpark
- 35. Extras
- 36. NUMBA: A PYTHON JIT COMPILER
- 37. Compiling Python • Numba is a type-specializing compiler for Python functions • Can translate Python syntax into machine code if all type information can be deduced when the function is called. • Code generation done with: – LLVM (for CPU) – NVVM (for CUDA GPUs).
- 38. How Does Numba Work? Python Function (bytecode) Bytecode Analysis Functions Arguments Numba IR Machine Code Execute! Type Inference LLVM/NVVM JIT LLVM IR Lowering Rewrite IR Cache @jit def do_math(a, b): … >>> do_math(x, y)
- 39. Numba on the CPU
- 40. Array Allocation Looping over ndarray x as an iterator Using numpy math functions Returning a slice of the array 2.7x speedup! Numba decorator (nopython=True not required) Numba on the CPU
- 41. CUDA Kernels in Python
- 42. CUDA Kernels in Python Decorator will infer type signature when you call it NumPy arrays have expected attributes and indexing Helper function to compute blockIdx.x * blockDim.x + threadIdx.x Helper function to compute blockDim.x * gridDim.x
- 43. Calling the Kernel from Python Works just like CUDA C, except we handle allocating and copying data to/from the host if needed
- 44. Handling Device Memory Directly Memory allocation matters in small tasks.
- 45. NUMBA IN SPARK
- 47. Using Numba with Spark
- 48. Using CUDA Python with Spark • conda can create a local environment for Spark for you: Define CUDA kernel Compilation happens here Wrap CUDA kernel launching logic Creates Spark RDD (8 partitions) Apply gpu_work on each partition
- 49. Compilation and Code Delivery Client Cluster CUDA Python Kernel LLVM IR PTX LLVM IR PTX PTX CUDA Binary Compiles Serialize FinalizeDeserialize
- 50. Compilation and Code Delivery Client Cluster CUDA Python Kernel LLVM IR PTX LLVM IR PTX PTX CUDA Binary Compiles Serialize FinalizeDeserialize This happens on every worker process