Scribd
Upload
38 views23 pages

Random Walks

Module 520 introduces random walks as a fundamental concept in stochastic processes, using a gambling game as a primary example. It develops and solves a finite difference equation related to the game, explores expected gains, and discusses implications for Markov chains and continuous processes. The module includes exercises, a model exam, and a bibliography to support further investigation into the applications of finite mathematics in probability.

Uploaded by

230110326
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
38 views23 pages

Random Walks

Module 520 introduces random walks as a fundamental concept in stochastic processes, using a gambling game as a primary example. It develops and solves a finite difference equation related to the game, explores expected gains, and discusses implications for Markov chains and continuous processes. The module includes exercises, a model exam, and a bibliography to support further investigation into the applications of finite mathematics in probability.

Uploaded by

230110326
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
Module 520 Random Walks: An Introduction to Stochastic Processes Ron Barnes Applications of Finite Mathematics COMAF nc, Suit 20, 57 Bedford Steet, Lexington, MA 02173 (617) 862-7878 “Moouves AND MONOGRAPHS IN UNDERGRADUATE “MATHEMATICS AND Its APpuCATIONS (UMAP) Project ‘The goal of UMAP was to develop, through a community of users and developers, a system of instructional modules in undergraduate mathemat- ics and its applications to be used to supplement existing courses and from ‘which complete courses may eventually be built. ‘The Project was guided by a National Advisory Board of mathemati- cians, scientists, and educators. UMAP was funded by a grant from the National Science Foundation and is now supported by the Consortium for ‘Mathematics and Its Applications (COMAP), Inc, a non-profit corporation engaged in research and development in mathematics education. COMAP Stare Paul [Link] Editor Solomon A. Garfunkel ‘Executive Director, COMAP Laurie Aragén Development Director Roland D. Cheyney Project Manager Philip A. McGaw Production Manager Laurie M. Holbrook Copy Editor Dale Horn Design Assistant Robin Altomonte Distribution ‘Sharon McNulty, Executive Assitant ‘The Project would like to thank James J. Halavin of Colorado School of Mines, and Peter Purdue of the University of Kentucky for their reviews, and all others who assisted in the production of this unit. ‘This material was prepared with the support of National Science Foun dation Grant No. SED80-07731. Recommendations expressed are those of the author and do not necessarily reflect the views of the NSF or of COMAP. ‘© Copyright 1962 by COMAP, Inc. All rights reserved. Reprinted in 1992, 1994, a. ue 15. RANDOM WALKS? AN INTRODUCTION 70 STOCHASTIC PROCESSES Department of Mathenatics University of Kouston by Ron Barnes Houston, TX 77002 INTRODUCTION... 2-2 ee ee ‘THE PROBLEM EXAMPLES OP THE PROBLEM... THE woDEL . SOLUTION OP THE MODEL UTILIZING OUR EXAMPLE COMMENTS 2 ee te ee GENERAL SOLUTION. » 2. ee eee ‘SOME IMPLICATIONS OP THESE SOLUTION FORMULAS EXPECTED GAIN COMMENTS ON THE PREVIOUS MODELS DURATION OF GAMES 2... ee eee GENERALIZED APPLICATIONS AND AREAS FOR PURTHER INVESTIGATION MODEL EXAM . SOLUTIONS TO THE MODEL EXAM APPENDIX: BIBLIOGRAPRY SOLUTIONS TO THE EXERCISES... . . . PINITE DIFFERENCE EQUATIONS 10 10 a3 au 16 18 19 Intempdular Description Sheet: UMP Unit 520 Tighe: BANDOW WALKS: AN INTRODUCTION TO STOCHASTIC PROCESSES Author! on Barnes Departuent of Msthenstics Driversity of ouster Bosstor, 117002, Bexiow stase/Date: Math Field: Finite Kathenatica AspLisation Field: Probability Abstract: Random valks are ietroduced by an exemple of 9 genbling game. “The aveociated finite difference equation is developed and solved. ‘The example is generalized and the general solution developed. Exercises introduce the ides of ¢ fei gene and other generelizations. The concept of eapected gain and the Carat mie ore introdiced aad their vsafulnses is denonstrated. General izatione to Markov chains and continuous, pros Simulation and applications i fe noted, Exercises, * nodal exam, and tolutione are. given. 14 bibLiography eve included, Upon completion students howid be able to: 1) Fecognize problens that vill invelve fisite difference equations: 2) develop and solve some tuch problene} snd 3) see sone of the inportant applications of canden procecees aud the value of applied mathenatice in formulating mn s/siee Stochastic Processes Prereguisitea: Te" 'high eehoot algebes 2, Solve « quadratic equation by che quadratic formula. L_uwropuctzoN In practically any gambling situation the odds are not even, In general the "house™ has a slight advantage. For example, in a roulette game there may be 36 nunbers on the wheel, while the house only pays odds of 35 tol. Similar~ ly in'one version of dice, the odés of rolling a7 is 6 to Land yet the house only pays off at a rate of 4 to 1. TE you were to repeat the sane bet (say, red on a rou- Lette table), we can characterize by p the probability of your winning en individual bet and q the probability that you lose an individual bet. Since p 2-00097 - .00097 x 32 dg = 1-00097 - ,003108 = .997866, 6. comenns Af you were to engage in any betting game vhere the odds against you were 2 to 1 (i.e., g = 2/3 and p = 4/3) and you started with $5, then the probability of your eventually losing everything is .997866, So it would real- ly be a vaste of time to play in this situation. However, the value of this discussion is that we can generalize the situation to let 2 and a take on other val- ues. If we considered the same problem, only assuming you have 1 = $9 and your opponent has a - Z'= $1, then a = $10 and 9 + 1.00097 - .00097(2)%, but 2 = $9 scans 5 = 1.00097 - .00097(219 iy = 1.00097 - .49664 a9 + «50650. Hence, if you played this gane with odds against you of 2 to 3 (and atmost every gambling cane gives much better odds than this), if you had more money than your opponent Gin fact, $9 and your opponent only $1), then your odds of eventually getting wiped out is only .50430, Tt can be shown that if the odds are, say, 6 to 4 (i.es5 p= 4, q = 6) and if you have z = $100 and your ‘opponent $10, then the probability of your getting wiped out is much less than .50. Notice as your probability of Losing everything, gz, gets smaller, your opponent's proba~ bility of getting wiped out increases. In fact, with this method it is possible to win money at the gambling tables even though the odds are against you as long as you agree to play to a winning which is small in comparison with the ‘amount of money you have available to bet. For an inter- esting example of one individual who aid just this at Monte carlo, see An Introduction to Probability Theory and its Applications by W. Feller. 7L_GENERAL soLuezoN In general to solve the equation 1. , = Pays + M9g-1 Subject to the conditions Go = 1s dq * 0 we get the general solution a aie = (l= tai)” ig : - tao = 2 eae However, if p= g = 1/2, although the above formula fails, it can be shown that the tvo solutions in this case are gz = 1 and gz = = and the general solution is gz = A+ Bz. Substituting qo = 1, qq = 0 gives qy © 1 ~ (2/a) for p rae Beecsiaes 1 pA ge ae e vitiaate Faia, 4 = Gy Lind gg the probability of 2, Derive the general solution for PAs Pees, + opp BY using the quadratic formule on the chavacterietic equatio: 3, Show that when p* q = 1/2y then gy #1 and gy = = both satiefy the associated finite ditterence equation Gg * Page * @te-1* 4. tm our exmple the bet at oech stage war 51. Compote the proba DiLity 4g im a ge where p= 1/4, q = 3/b, 2 = Sb a= $10, Hest, if each bet Le to be $2 instead of $1, compute gg. Hote that in this case you consider this as a uev gaze vith'Z = 3, ch of the = Sand each bet is 1 unit, Compare your gg for two methods and decide whick vay of betting is more advantageous 5. If you are playing 4 fair game (p = q = 1/2), discuss the proba bility gy of "wiping out" your opponent if you have $10 and your opponent has £5. What if you have $15 and your opponent hat $307 nat Lf you have $2 and your opponent has Sa ~ 27 Juse accenpe to reason it out. Ho calculations are necessary. 6. If you have $2 and the total is Se and on exch play you can cove shead 2 with probability p (isesy win $2) or go back 1 Close $1) Vith probabitity q, Find the Finite difference equation core spovding to the situation for gy. What do you suspect about the ocited characteristic equation? Donte atteapt Observe the the last equation implies that in a fair game (p= q)y if you have 1/2 of the total stake (Z = 0/2) then the probability of your eventually losing is 1 - 1/2 = 1/2 and, similacly, if you have 1/3 of all the money, then you will lose 2/3 of the time. Since the role of you and your adversary can be reversed, we see that p, = the proba~ bility that you win with 1/3 the stake must equal the prob- ability that your adversary loses with 2/3 of the stake, ies, py = 1 > 2/3 = 1/3, thus pz + az = 1. This means that in‘a fair game, while it is theoretically possible that the game could go on forever with neither side winning all the stake, the probability of this happening (starting with $2), is tg = 0, Tt can also be shown that even in an unfair game (p'# q), that rz = 0 and pz + az = 1. 2. EXPECTED GALE Note that your gain from playing such a game is a - 2 dollars if you win and ~z dollars if you lose (and c, = 0). ‘Thus, your expected (average) gain i (a-zp, + (-24, = (a-z)(0-a,) ~ 29, z- aq, + 2a, - 2, = aQeg) = 2 Im particular, if you are ina fair game (p= q = 1/2), then gz = 1 ~ 2/2 and so your average gain becomes all = (-z/ab} - z= 0. ‘Thus ina fair game, your average winnings should be zero dollars, Notice that if you start a fair game with 1/3 of the total stake, then you should win 1/3 of the time (your opponent's 2/3 stake) and lose 2/3 of the tine (your 1/3 stake)? so your average win and loss cancel out. 10, COMMENTS ON THE PREVIOUS MODELS One of the questions that the previous discussion ight suggest is: if it is possible to win money at the gambling tables, even though the odds are against you (as long as you agree to play for a winning which is small in comparison to the total anount of money you have available to bet), then why don't more people take advantage of this and cause the gambling casinos to go broke? The anever vill be noted after first considering the following example: You bet one unit of money on your first play of a gaze that pays off 2 to 1. Assume your unit bet is $5. Tf you win, you get back your $5 + $5 winning, so you can quit vith a1 unit win. If you lose, you double your bet to 2 units on the next play, and if you win, you get winnings of 1 unit since your return is 4 units which includes your 1 unit from the first bet and 2 from the second, plus your 1 unit winnings. If you lose your second bet, you double your bet to 4 unite on the next play and continue this vay. Observe that since you quit vhenover you win a bet, your winnings will alvays be 1 unit since the geonetric sum of 1+ 244464... 4202 220-1, Theoretically you can never lose with this systen and you can make a fortune by betting, say, in unite of $10,000. Unfortunately, to win with such a ystem, you must have an unlimited amount of money since you may have the great Sisfortune to lose the first 50 times or s0. If, hovever, your financial resources are virtually unlimited, the casino prevents you from exercising this system by putting a Limit on the maximum bet that can be made. Sone casinos may not have such limits, but may take other measures to protect the house, such as the banning of card counters From black jack tables. while it is true that, in some games, you have a very high probability of winning a small stake in comparison vith your total bankroll, the time needed to win any sig nificant amount of noney may be prohibitive. LL._DURATION GP caps Therefore it might be of interest to calculate the average duration of a random walk game (1.8., the average huaber of plays necessary to vin the total stake a or lose all your money and arrive at stake 0). IE we let Dy be the duration of che game, then it can be reasoned that D, satisfies the finite difference equa thon Dy * PDgey + Dey tl for Oc aca vith boundacy conditions D9 = 0, Dg = 0 and it can be shown the general solution is of the form peas apt AS BaD, if pea ee D =Ater- 2, if pag. Tuaxcine Z. toe the boundary conditions to show the eh chat ey Fe TP i ceiort ee ipearue. Prom these formulas we see that if 2 players each had $200 and you bet §1 on a coin toss, then the average dura- tion (p= q = 2/2) would be D, = 200(400- 200) You might be interested in checking this ler nunbers of coin tosses and deter— mine the actual durations of your games. 12._GENERALIZED APPLICATIONS AND AREAS 2OR FURTHER INVESTIGATION Ne note that Feller (1) and Rarlin (2) generalize ran- dom walk ideas to two and three dimensions. Another gener- alization is to generalize it to # continuous analogue in three dimensions. This will lead to the idea of Brownian motion, an example of which is the randon motion of parti- eles suspended ina liguid. Since these ideas require more Sophisticated mathenatice than required for this nodule, ve leave then to the interested reader to investigate from our references, -10- Interesting simulation studies can be made, such as the ones we suggested above in your checking the validity of our average duration formula, Sone studies using con puters have been done and Peller (2) notes the surprising Tesult of gone auch simulation studies; namely, that in ovr coin tossing problen, each of the players has the lead for long periods of time--a simulation indication that winning streeke nay have a mathenatical basis (when you're hot, you're hott). Te should be noted that, in the biological and medical sciences, variability anong observations is so large thet the deterministic models of physics and engineering are in- appropriate and stochastic models (of which our random walk is simple model) are the proper models for investigations. Puri (4) enumerates a variety of examples, including growth of @ population such as a bacterial colony, the spatial @istrfbution of plant and animal communities, spread of an epidenic in a community, spread of cancer growth within a body, and others. We note that J.L. Coolidge (1) was one of the first to note that with our random valk model in an unfair game, the player's best chance of winning a specified sum is to al- waye make the largest unit bet the banker will accept. Thie war alluded to in the exercises in section 7. Row ever, in most gambling games, the size of the bet often changes from one play to another. This corresponds to not always moving fron 7 to 1 +1 with probability p, or from Z to z- 1 with probability q, but possibly moving from, say, z to 2 +15 in one step and perhaps to Z - 20 on the next step. When such a situation like this occurs, the mathe- matical deseription is referred to as a Markov chain (an- other form of @ stochastic process). one exanple of such a process occurs in genetics. If ‘a cell contains N genes with R of these mutant and N - R normal, then when the cell cubdivides, each of the two daughter cells will have N genes. Each of the original R mutant genes splits into two mutant genes and each of the W = R normal genes splits into two normal genes. However, the daughter cells can inherit any one of many possible combinations of mutant and non-mutant genes, For example, Af the criginal cell bad five normal and three mutant genes, the daughters might look like 6 mutant 4 normal 6 normal and @ normal or and ete... 2 normal 4 mutant 2 mutant tes If we consider the process as similar to our original ganb- Ling process, we let 2 = the nunber of mutant genes (R) in the first parent and let a- 2 = 8 - R the nunber of normal genes in the first parent, which inplies a = N then g, = the probability of starting with 2 mutant genes and eventu- ally ending up with some descendant with no mutant genes Gin genetics, a pure strain) while p, = the probability of starting with a parent with 2 mutant genes and eventually fending up with some descendent with all mutant genes (in genetics, this usually results in death to the cell). of course in this process, the novenente at each step are not alvays fron % to +1 or z- 1, In fact, in this case the Probability of making jumps of different sizes have differ- ent values. A Dranch of mathenatics called matrix algebra is useful to any study of the properties of Markov chains. When processes are modeled as Markov chains, the associated Probebilities are often unknown, but can be statistically estimated by data sampling technique ‘The use of such processes is also widespread in the Social sciences, econonics, and business. Ae an example we consider the following problen which was instrumental in the developaent of an entire area of mathematical investi- gation. The paper by Puri (4) will serve as reference on the following discussion. Francis Galton, a British bionetrician was interested An the decay of families of men who occupied famous posi- tions in the past. He wondered if the extinction of famous family nanes was by chance or if the physical comfort, in- tellectual achievenent and other factors led to a decrease in fertility, fe posed the problem as follows (see Purt ma. A nation with N adult males, each with a separate suc name, colonize a district. ‘Theic law of population is Such that, in each generation, ag percent of the advit ales have no male children who reach adult Lifes ay Percent have one such child; ap have two, and so ont Find what proportion of the surnanes will have become extinct after r generations. Cleraynan Henry Watson (see Puri (4)) showed that if ¢ is the probability that 2 family nane will ultinately be- come extinct, then q satisfies the equation xe Dae ) naz. wen Lape. sto sme problem was eventually completely solved by J.P Stetfensen (see Puri (4)) vho noted that if a, 4 1008, then the probability of ultinate extinction is one if the average number of sons born per male parent is less then one, and leon than one if this average number is greater than one. This celebcated Galton-Watson process is special cane of # Rackov process and ite generalizations have led to many fruitful applications in the life and 1 sciencen, 13._MODEL_ EXAM 1. Let py be the probability that you eventually win in a random walk game vith stake z (Py q & a8 before). Show that the explicit formula for calculating pz vhen peais », see fy - 2. For & canon walk with p= 1/3, q= 2/3, 8 show that Py + az © 1 by using the formula in T'sbove frp. “Does this mean that in auch a gane, one of the to players sust aluays vin? 3. Show Py + dg = 1 for any random walk (pq). 4, In a random walk, what is the effect on the value of dg and hence pz when you double or halve the anount of your bet? 5. Try to reason out why the finite difference equation for Dy is given by Dy = PDgqq + Dy +1 for 0c 2 < a. Why is the 1 intfoduced in this equation? 6. A manufacturing company builds plants in @ region when ‘the nuaber of back orders in the region ia z = 25 units (each nit is more than 10 orders). A plant fille 1 unit of orders each day. ‘The company has 2 policy that it closes any plant when the number of back order unite 2 = 0. The company aloo closes the plant ané constructs a larger one if the nunber of back order unite gets to 100 units. Bach day the Probability of getting nore than 10 orders, i.e., one ae unit is known to be p= .4S and og © .S5, is the probability that in a given day 10 or fever orders are made (i.e., 1 unit of back order is lost). Find the average duration of a manufacturing plant's life. Companies often use this information to decide on tax strategies for depreciating their plants and machinery. 14,_SOLUDIONS TO TRE EXERCISES 1. aL ag = Bt ea. 2, Charactarietic equation ie Paleeleo L [Fs Pere es xi bf ACD + Bain Then. = 1 aad g, ° el 2 1rwaaeu2 ast Gea ay eet 1) #2 Get) = ey nue “ 5 2 For p= Why q+ 34s £ = $6.0 = $10 and $1 bets saa ag = IR = 908. Yor p# M/s q* 3/h 2 = 835 as ate = 95 (for «$2 ber), we get = O51. Te is more advantageous to bet the larger wait, To a fais game if you have 2/3 of all the money ($10 out of = total $15), then you should probably win two thirds of the tine. You should have « $5 lead (and vin) jost a8 often a4 your oppo" nent should, However, your $10 stake means that. for your oppo- nent to win, he/she must have two consecutive $5 streaks. Sinilarly, if you heve 1/3 of the money (S15/845), yoo should win vith probability 1/3. a gensrel py = 2/4 where is the amount you have and a - = is your opponent's take. ote chat chis implies that, even in a fair game, Lf you play with « finite stake against a very rich sévereary, then py * 2/4 vill approach 0 as « increases. $0 it can be shove that the probability of your opponent's winning az fappronch 1 Sf Ne/ebe wants co put in a sufficiently large stake. 4° Pheer 7 Mea Characteristic equation B-lyete peter? Aa a 30d depres equation. peas s@® tor p= peas a® tor pes =O & wive 0 = thea whee Asa @? asaeo eres Osos Beas ate) wasnt: ae at =~ td # soe and gee = EE, 4s oye EES ey rar nas- oF ys alee), A, ote to win pg is the sme a6 your opponent losing vith his/her stake of a - 2 and the roles of p and q just reverses 40 9, = 6, [Link] p and q are reversed and also the roles of and a= rin The equation for gg in enerciee 2s te 4s such # game there are an infinite number of ways that the gene cam proceed, Ut is clear that in some gamer, neither sige vill ever ving for example, if you vin the first play, lose the ec ond, vin the third, lose the fourth, and continue this pattern forever. The fact that py ¢ qq = 1 means thet in the infinice foumber of ways such # game can proceed, the nuuber where neither side eventually wins ie euch onall fraction thet ite probabil- ity is 0. A nore precise explanation of this would require & course in calenlus-base n16- * wort yet * carpi (aly = Bota 1 (by aagebrad. Ae Ae wae seen io exere sine of the bet increases, the ‘value of Gy decreases and 0 py increases. So, a better bet is ‘nade by increasing the bet, ot halving it, which is « worse strateny. You can compere a, = ated = (ale) vie et 2 Gaye) TTS es] aod by factoring see that gy < qzq+ Thus, doubling the size of the bat, which is the sme se changing fron a scale of 2x and 24 tte one of + and 4, wil decrease your chances of losing ver q > 1p. Thus, your chances of winning, pgs are Increased. Dg = pDggy ¢ Pye * 1 for 0 € 2

You might also like