Building Scalable Websites with Perl

Building Scalable Websites with Perl

• 1.
  Building Scalable Websiteswith Perl Perrin Harkins Plus Three (http://plusthree.com/)
• 2.
  Who is doingit? • Yahoo – Overture • Amazon – IMDB • InterActiveCorp – Ticketmaster – CitySearch
• 3.
  How are theydoing it? • Not just hardware • Basic techniques
• 4.
  Things we won'tbe covering • mod_perl tuning – http://perl.apache.org/ • DBI tuning – http://search.cpan.org/~timb/ • hardware
• 5.
  Caching
• 6.
  Page-Level Caching • Bestperformance • Pre-generate from batch job page batch cache data job web client server
• 7.
  wget wget ­­mirror ­­convert­links     ­­html­extension ­­reject gif,jpg,png     ­­no­parent     http://app­server/dynamic/pages/ wget page cache web app client data server server
• 8.
  Generate-On-Demand data web mod_proxy app client server cache server
• 9.
  mod_proxy ProxyRequests Off ProxyPass /dynamic/stuff http://app­ server/ ProxyPassReverse /dynamic/stuff  http://app­server/ CacheRoot "/mnt/proxy­cache" CacheSize 500000 CacheGcInterval 12 CacheMaxExpire 36 CacheDefaultExpire 2
• 10.
  Intercepting 404 Errors ErrorDocument 404 /page/generator web page client server cache 404 handler data
• 11.
  Partial-Page Caching 2 hours 5 minutes 24 hours
• 12.
  Mason cache   my $result = $m­>cache­>get(                          $search_term                              );   if (!defined($result)) {       $result = run_search($search_term);       $m­>cache­>set($search_term,                      $result,                                   '30 min');   }
• 13.
  Cache::FastMmap   our $Cache = Cache::FastMmap­>new(                     cache_size  => '500m',                     expire_time => '30m',                                    );   $Cache­>set($key, $value);   my $value = $Cache­>get($key);
• 14.
  Memcached our $Memd = Cache::Memcached­>new({     'servers' => [       "10.0.0.15:11211","10.0.0.15:11212",       "10.0.0.17:11211",        [ "10.0.0.17:11211", 3 ]     ],     'debug'   => 0,     'compress_threshold' => 10_000,                                  }); $Memd­>set($key, $value, 5*60 ); my $value = $Memd­>get($key);
• 15.
  Job Queuing
• 16.
  Get in Line
• 17.
  “search in progress” web client server fork worker redirect web done client reload worker server yet? result http://www.stonehenge.com/merlyn/WebTechniques/col20.html
• 18.
  Spread::Queue client use Spread::Queue::Sender; my $sender = Spread::Queue::Sender­>new(                                 "myqueue"                                        ); $sender­>submit(     "myfunc", { name => "value" }                ); my $response = $sender­>receive();
• 19.
  Spread::Queue worker use Spread::Queue::Worker; my $worker = Spread::Queue::Worker­>new("myqueue"); $worker­>callbacks(            myfunc => &myfunc,                   ); $SIG{INT} = &signal_handler; $worker­>run;
• 20.
  Spread::Queue worker sub myfunc {     my ($worker, $originator, $input) = @_;            my $result = {         response => "I heard you!",                  };     $worker­>respond($originator, $result); }
• 21.
  Resources • “Computer Scienceand Perl Programming” • “Mastering Algorithms with Perl” • “Patterns of Enterprise Application Architecture” by Martin Fowler • http://oreillynet.com/
• 22.
  Thanks! • Craig McLaneand Adam Sussman of Ticketmaster Online • Zack Steinkamp of Yahoo