ABC-SysBio – Approximate Bayesian Computation in Python with GPU support
4 likes1,327 views
The document discusses approximate Bayesian computation (ABC) in Python with GPU support, focusing on its application in biological modeling and the challenges associated with inferring complex parameters from data. It elaborates on various modeling techniques, the importance of parallel computation using GPUs, and the benefits of different ABC methodologies. Additionally, it highlights the architecture and performance issues of GPU programming relevant to ABC applications.
ABC-SysBio – Approximate Bayesian Computation in Python with GPU support
- 1. ABC-SysBio: Approximate Bayesian Computation in Python with GPU support Chris Barnes, Michael Stumpf (among many others!) Centre for Bioinformatics & Institute of Mathematical Sciences & Centre for Integrative Systems Biology at Imperial College London 20th May 2010 1 / 28
- 2. Challenges in biological modelling 2 / 28
- 3. Challenges in biological modelling Biological systems Complex models with many parameters Not all parameters can be measured in vivo At least some parameters, if not all, must be inferred from data 2 / 28
- 4. Challenges in biological modelling Biological systems Complex models with many parameters Not all parameters can be measured in vivo At least some parameters, if not all, must be inferred from data Data Time course Single cell fluorescent microscopy, flow cytometry, count data Can be challenging to perform, not all species observed 2 / 28
- 5. Challenges in biological modelling Biological systems Complex models with many parameters Not all parameters can be measured in vivo At least some parameters, if not all, must be inferred from data Data Time course Single cell fluorescent microscopy, flow cytometry, count data Can be challenging to perform, not all species observed Stochasticity Apparent that many biological processes are highly stochastic Stochastic models make inference much harder Often we cannot write down the likelihood function 2 / 28
- 6. Different time course modelling techniques 3 / 28
- 7. Different time course modelling techniques Infer model parameters from time course data Select between competing models of a process 3 / 28
- 8. Approximate Bayesian Computation Model Data, X θ2 X (t) t θ1 4 / 28
- 9. Approximate Bayesian Computation Model Data, X θ2 X (t) t θ1 4 / 28
- 10. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t θ1 4 / 28
- 11. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 4 / 28
- 12. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 13. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 14. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 15. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 16. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 17. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 18. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 19. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 20. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 21. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 22. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 23. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 24. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 25. Approximate Bayesian Computation Model Data, X θ2 X (t) Simulation, Xs (θ) t d = ∆(Xs (θ), X ) θ1 Reject θ if d > Accept θ if d ≤ 4 / 28
- 26. Approximate Bayesian Computation (ABC) Bayesian Inference Posterior ∝ Likelihood × prior p(θ|X ) ∝ p(X |θ) p(θ) Approximate inference methods Sample from approximate posterior: p(θ|∆(Xs (θ), X ) ≤ ). where ∆(Xs , X ) is distance between simulation and data. It can be shown, as → 0 p(θ|∆(Xs (θ), X ) ≤ ) → p(θ|X ) ABC flavours ABC rejection Pritchard et al. Mol. Biol. Evol. (1999) ABC MCMC Marjoram et al. PNAS (2003) ABC SMC Toni and Stumpf, Bioinformatics (2010) 5 / 28
- 27. ABC SMC Prior, π(θ) πT (θ|∆(Xs , X ) < T ) 6 / 28
- 28. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) 6 / 28
- 29. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 30. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 31. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 32. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 33. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 34. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 35. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 36. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 37. ABC SMC Prior, π(θ) Define set of intermediate distributions, πt , t = 1, ...., T 1 > 2 > ...... > T πt−1 (θ|∆(Xs , X ) < t−1 ) πt (θ|∆(Xs , X ) < t ) πT (θ|∆(Xs , X ) < T ) Sequential importance sampling: Sample from proposal, ηt (θt ) and weight wt (θt ) = πt (θt )/ηt (θt ) R ηt (θt ) = πt−1 (θt−1 )Kt (θt−1 , θt )dθt−1 Kt (θt−1 , θt ) is Markov perturbation kernel 6 / 28
- 39. What can ABC-SysBio do? Inputs Outputs Models in SBML format or python (Approx) posterior distribution over code models and parameters Time series data, distance schedule Diagnostic plots Example 5 : Inference in repressilator model 8 / 28
- 40. Model selection Example 1 : SIR model selection Simulated data Model posteriors Model 1 fit 9 / 28
- 41. Computation on graphical processing units (GPUs) ’GPUs are massively multithreaded many-core chips’ Parallel architectures Multiple instruction multiple data (MIMD) Multiple independent processors execute different instructions on different data Clusters, GRID computing Main drawback: cost Single instruction multiple data (SIMD) Multiple processors execute same instruction on different data Supercomputers from 70s-80s based on this architecture GPUs follow this paradigm and are cheap Main drawback: Programming paradigm differs from CPU, not all applications can be ported GPGPU: General purpose GPU GPUs evolved from dedicated computer graphics to general-purpose parallel processors Dedicated computation GPUs manufactured by NVIDIA, ATI 10 / 28
- 42. NVIDIA GPUs : Compute Unified Device Architecture (CUDA) CUDA Refers to both hardware and software architectures Provides C APIs for programming NVIDIA GPUs Tesla C1060 : 30 multi x 8 processors = 240 cores http://www.nvidia.com/object/cuda_home_new.html http://www.many- core.group.cam.ac.uk/course/ http://www.drdobbs.com/high- performance- computing/207200659 "ProgrammingMassivelyParallelProcessors",Kirk, Hwu http://mathema.tician.de/software/pycuda 11 / 28
- 43. CUDA architecture Software: Thread hierarchy Hardware: Tesla C1060 12 / 28
- 44. Memory structure speed scope global 150x slower device, host local 150x slower thread texture faster (cached) device, host constant faster (cached) device, host shared fastest thread block registers fastest thread 13 / 28
- 45. ABC on GPU Suitability ABC SMC time dominated by simulation of timeseries Simulations can be done in parallel albeit with different parameters 14 / 28
- 46. ABC on GPU Suitability ABC SMC time dominated by simulation of timeseries Simulations can be done in parallel albeit with different parameters CUDA performance issues Shared vs global memory. Shared fast. Global slow If using global memory is it coalesced? Warp divergence. ’if else’ statements cause divergence Register vs local memory. Reduce registers to increase occupancy. Local memory slow Double vs float 14 / 28
- 47. ABC on GPU Suitability ABC SMC time dominated by simulation of timeseries Simulations can be done in parallel albeit with different parameters CUDA performance issues Shared vs global memory. Shared fast. Global slow If using global memory is it coalesced? Warp divergence. ’if else’ statements cause divergence Register vs local memory. Reduce registers to increase occupancy. Local memory slow Double vs float Unsuitability ’Good quality’ pseudo random number generation ODE integration : stiff solver requires double precision, adaptive MJP simulations : Gillespie is sequential, threads are independent SDE simulations : Euler-Maruyama, time step same for all threads, possibly most to gain 14 / 28
- 48. Occupancy vs latency Warps (32 threads) are run sequentially on a processor Hardware tries to hide ant memory latency (eg waiting for global memory call) Performance is a tradeoff between occupancy and latency Example : SSA (Gillespie) simulation of immigration-death process 15 / 28
- 49. Bacterial two component systems (TCS) TCSs are abundant in bacteria, plants and fungi, but apparently absent from animals. They regulate response to environmental stimuli. H ∼P D ∼P ATP 16 / 28
- 50. Bacterial two component systems (TCS) The ArcB-ArcA TCS TCSs are abundant in bacteria, plants and The ArcB-ArcA system in Escherichia coli uses a fungi, but apparently absent from animals. phospho-relay mechanism. They regulate response to environmental stimuli. ATP Transmitter H ∼P Domain (H1) H ∼P Receiver D ∼P Domain (D1) D ∼P Phosphotransfer H ∼P Domain (H2) D ∼P ATP 16 / 28
- 51. Bacterial two component systems (TCS) The ArcB-ArcA TCS TCSs are abundant in bacteria, plants and The ArcB-ArcA system in Escherichia coli uses a fungi, but apparently absent from animals. phospho-relay mechanism. They regulate response to environmental stimuli. ATP Transmitter P H ∼P Domain (H1) H ∼P Receiver D ∼P Domain (D1) D ∼P Phosphotransfer H ∼P Domain (H2) D ∼P ATP 16 / 28
- 52. Bacterial two component systems (TCS) The ArcB-ArcA TCS TCSs are abundant in bacteria, plants and The ArcB-ArcA system in Escherichia coli uses a fungi, but apparently absent from animals. phospho-relay mechanism. They regulate response to environmental stimuli. ATP Transmitter P H ∼P Domain (H1) H ∼P Receiver D ∼P Domain (D1) D ∼P Phosphotransfer H ∼P Domain (H2) D ∼P ATP Orthodox system Unorthodox system 16 / 28
- 53. Gillespie simulation of orthodox TCS : timing comparisons MJP simulation using Gillespie, 5 reactions and 5 species pyCUDA on GPU vs C++ on single CPU With no compiler optimizations on either 30x speed up Maybe 10x more realistic Simulation timing simulations per second gpu / cpu 30 q single cpu q q QuadroFX1700 q QuadroFX1700 q QuadroFX3700 q QuadroFX3700 q TeslaC1060 2.5 q TeslaC1060 q 25 q q q q 2.0 q q 20 q q log10(time (s) ) q q q q q q q 15 1.5 q q q q q q q q q q q q q q q q q q 10 q q q qq q q q q q q q q q q q qq q q q qq q q q 1.0 q q q q q q q q q q q q q q 5 0.5 q 0 1 2 3 4 5 log10(n simulations) 17 / 28
- 54. Mechanistic modelling of metapopulation dynamics Metapopulations Predator-prey systems are unstable and prone to extinction One mechanism for promoting stability is spatial heterogeneity A metapopulation is a set of linked sub populations or ’patches’ Limited dispersal and asynchronous dynamics between patches increases total persistence 18 / 28
- 55. Mechanistic modelling of metapopulation dynamics Metapopulations Predator-prey systems are unstable and prone to extinction One mechanism for promoting stability is spatial heterogeneity A metapopulation is a set of linked sub populations or ’patches’ Limited dispersal and asynchronous dynamics between patches increases total persistence Inference of stochastic migration models from time series data Important for understanding how to maximize metapopulation persistence Obvious implications for conservation Relatively unexplored Gillespie and Golightly (2009) Demonstrate methods that can be used on field data 18 / 28
- 56. The contenders Beetles : Callosobruchus chinensis Also known as ’bean weavil’ Live on beans or seeds Adults lay eggs on beans Larvae chew their way into the bean Wasps : Anisopteromalus calandrae Bean weavil’s natural parasitoid Adults lay eggs on beetles Larvae kill and eat beetle Can be released into grain stores as pest control 19 / 28
- 57. Experimental setup Laboratory microcosm 4 × 4 clear plastic boxes (73×73×30 mm) 20 / 28
- 58. Experimental setup Laboratory microcosm 4 × 4 clear plastic boxes (73×73×30 mm) Establish bruchid beetle on black eyed peas 20 / 28
- 59. Experimental setup Laboratory microcosm 4 × 4 clear plastic boxes (73×73×30 mm) Establish bruchid beetle on black eyed peas Introduce wasp populations 20 / 28
- 60. Experimental setup Laboratory microcosm 4 × 4 clear plastic boxes (73×73×30 mm) Establish bruchid beetle on black eyed peas Introduce wasp populations Control inter cell migration using gates Limited dispersal Unlimited dispersal 20 / 28
- 61. Metapopulation structure affects persistence (Bonsall et al (2002) ) 21 / 28
- 62. Time series data : ’unlimited dispersal’ x 4 replicates 20 25 20 20 20 15 15 15 15 X X X X 10 10 10 10 5 5 5 5 0 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 t t t t 25 20 20 25 20 20 15 15 15 15 X X X X 10 10 10 10 5 5 5 5 0 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 t t t t 25 25 25 30 20 20 25 20 20 15 15 15 X X X X 15 10 10 10 10 5 5 5 5 0 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 t t t t 30 25 30 20 25 20 25 20 15 20 15 X X X X 15 15 10 10 10 10 5 5 5 5 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 0 20 40 60 80 t t t t 22 / 28
- 63. Time series data : ’limited dispersal’ x 4 replicates 30 30 40 40 25 25 30 30 20 20 15 15 X X X X 20 20 10 10 10 10 5 5 0 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 t t t t 30 40 20 25 30 20 30 15 20 15 X X X X 20 10 10 10 10 5 5 0 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 t t t t 40 30 50 30 25 40 30 20 30 20 20 X X X X 15 20 10 10 10 10 5 0 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 t t t t 40 40 30 50 25 40 30 30 20 30 20 20 X X X X 15 20 10 10 10 10 5 0 0 0 0 20 40 60 80 0 20 40 60 80 0 20 40 60 80 0 0 20 40 60 80 t t t t 23 / 28
- 64. Stochastic model Xi , Yi denote numbers of beetles, wasps in cell i Beetle logistic growth process rate Xi → 2Xi b1 Xi Xi → ∅ d1 Xi2 Wasp-Beetle interation: Lotka-Volterra process rate Xi + Yi → 2Yi pXi Yi Yi → ∅ d2 Yi Migration process rate Xi → Xj mXc Xi Xi + Xi → Xi + Xj mXd Xi (Xi − 1)/2 Yi → Yj mYc Yi Yi + Yi → Yi + Yj mYd Yi (Yi − 1)/2 24 / 28
- 65. Movement models Q: ”Given a migration event occurs, where does the individual move?” 25 / 28
- 66. Movement models Q: ”Given a migration event occurs, where does the individual move?” Global movement: Xi → Xj where i = j 25 / 28
- 67. Movement models Q: ”Given a migration event occurs, where does the individual move?” Local movement: Xi → Xj where j ∈ nearest neighbours of i 25 / 28
- 68. Inference: Global vs local movement 1) Global movement 2) Local movement 26 / 28
- 69. Inference: Parameters of the global movement model prey birth prey m (const) m2 m2 m2 m2 m2 m2 0.4 0.8 1.2 0 1 2 3 4 5 prey death prey m (dens) m1 m1 m1 m1 m1 m1 m2 m2 m2 m2 0.00 0.05 0.10 0.15 0 1 2 3 4 5 predation pred m (const) m1 m1 m1 m1 m1 m1 m2 m2 m2 m2 0.10 0.15 0.20 0.25 0 1 2 3 4 5 pred death pred m (dens) m1 m1 m1 m1 m1 m1 m2 m2 m2 m2 0.20 0.25 0.30 0.35 0.40 0 1 2 3 4 5 27 / 28
- 70. Thanks! Acknowledgements Juliane Liepe, Erika Cule Michael Stumpf Michael Bonsall Tina Toni Xia Sheng Kamil Erguler Paul Kirk Justina Norkunaite Suhail Islam [email protected] http://www3.imperial.ac.uk/theoreticalsystemsbiology http://abc-sysbio.sourceforge.net/ 28 / 28