Importance Of Structured Incident Response Process Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA WRITTEN: 2004 Example..................................................................................................................1 Introduction.............................................................................................................

2 SANS Six Step Incident Response Methodology...................................................4 Incident Response Tools........................................................................................ Example !orporation " #orm Incident Re$isited...................................................% !ommon Mista&es o' Incident Response.............................................................1( !onclusion............................................................................................................12 )lossary................................................................................................................1* DISCL I!ER+ Security is a rapidly changing field of hu an endeavor. !hreats "e face literally change every day# oreover, any security professionals consider the rate of change to $e accelerating. %n top of that, to $e a$le to stay in touch "ith such ever&changing reality, one has to evolve "ith the space as "ell. !hus, even though I hope that this docu ent "ill $e useful for to y readers, please keep in ind that is "as possi$ly "ritten years ago. Also, keep in ind that so e of the '() ight have gone *+*, please Google around.

Example
Right around lunchtime, a helpdes& operator at Example !orporation -- a [Link] manu'acturing company " recei$es calls 'rom se$eral users all reporting computer 'ailures and slo/ net/or& response. Example !orporation0s security in'rastructure includes 'ire/alls, intrusion detection systems, anti-$irus so't/are and operating system logs, all technology in$estments 'rom the 12oom3 years. The helpdes& operator opens a ne/ trou2le tic&et in Remedy, descri2ing the users0 pro2lems and recording the machines0 hostnames. 4ther unrelated support issues continue to pile up and the operator0s attention is directed else/here. Mean/hile, the /orm, /hich caused the a2o$e laptop pro2lems, continues to spread throughout Example0s net/or&. The malicious so't/are made its /ay into Example a'ter 2eing 2rought in 2y one o' the sales people /ho o'ten plugs his laptop into untrusted net/or&s, such as hotels and customer en$ironments, outside the company. #ith most o' the Example0s security monitoring capa2ilities deployed in a 5M6 and on a net/or& perimeter, the remainder o' Example0s $ulnera2le corporate assets are largely unguarded and un/atched. Thus, as the /orm /ends its /ay around Example0s enterprise, the company security team is not e$en a/are o' a de$eloping disaster.

Page 1

Soon, net/or& tra''ic generated 2y the /orm has increased dramatically, as more machines 2ecome in'ected and start spe/ing copies o' the same /orm. #hen the in'ection reaches critical le$els and starts to a''ect the per'ormance o' monitored ser$ers, the security team is noti'ied 2y a 'lood o' pager alerts7 chaos ensues. #hile some try installing anti-$irus updates other apply 'ire/all 2loc&s 8pre$enting not only /orm scanning, 2ut also the do/nload o' updates9 and yet others try to scan 'or $ulnera2le machines that contri2utes to the net/or&-le$el denial-o'-ser$ice. A'ter hours o' uncoordinated acti$ities, most o' the /orm-carrying machines are disco$ered and the re-in'ection rate is 2rought under control. A management re:uested in$estigation 2egins and computer 'orensic consultants are 2rought in. ;o/e$er, /hat remained o' the initial in'ection e$idence /as either destroyed or extremely hard to 'ind due to 1mitigation3 acti$ities that /ere implemented. No one remem2ered the original Remedy incident recorded 2y the helpdes& operator since the helpdes& system /as not deemed rele$ant 'or security in'ormation. The in$estigation /as a2le to conclude only that the malicious so't/are /as 2rought in 'rom outside the company -- the speci'ic initial in'ection $ector /as ne$er determined. The 'inancial and technological damage is easy to see. And yet, the recurring security incident descri2ed a2o$e sho/s /hat happens /hen companies lac& a central point 'rom /hich to manage security incidents.

Introduction
Security pro'essionals learn to constantly chant the mantra 1pre$ention-detectionresponse.3 Each o' these three components is &no/n to 2e o' crucial importance to the organi.ation0s security posture. ;o/e$er, unli&e detection and pre$ention, the response is impossi2le to a$oid. #hile it is not uncommon 'or the [Link] to ha$e /ea& pre$ention and nearly non-existent detection capa2ilities, response /ill ha$e to 2e there since the [Link] /ill o'ten 2e 'orced into response mode 2y the attac&ers 82e it the internal a2user, omnipresent 1script &iddie3 or the elusi$e 1u2er-hac&er39 or their e$il creations 8$iruses, /orms and spy/are9. The [Link] /ill li&ely 2e ade to respond in some /ay a'ter the incident has ta&en place. E$en in cases /here ignoring the incident that happened might 2e the chosen option, the [Link] /ill implicitly 'ollo/ a response p"an, e$en i' as ine''ecti$e as to do nothing. In light o' this, 2eing prepared 'or incident response is li&ely to 2e one o' the most cost e''ecti$e security measures the [Link] ta&es. Timely and e''ecti$e incident response is directly related to decreasing the incident-induced loss to the [Link]. It can also help to pre$ent an expensi$e and hard-to-repair reputation damage, /hich o'ten occurs 'ollo/ing the security incident. Se$eral industry sur$eys ha$e identi'ied that pu2lic company<s stoc& price may plunge

Page 2

se$eral percent as a result o' a pu2licly disclosed incident 8http+==///.security'[Link]=ne/s=111>%9. Incidents that are &no/n to /rea& catastrophic results upon the [Link] may in$ol$e malicious hac&ing, $irus out2rea&s, economic espionage, intellectual property the't, net/or& access a2use, the't o' IT resources and other policy $iolations. Most o' us in the security industry are already 'amiliar /ith the traditional challenges /e 'ace e$ery day7 too much security data to si't through, too many 'alse alarms to deal /ith, and not enough 2udget or resource to handle an e$ergro/ing num2er o' security incidents. 4ne additional and o'ten o$erloo&ed challenge in$ol$es the security management process itsel'. ?argely ignored in many o' today0s IT enterprises, a clearly de'ined, documented, and repeata2le incident management process de'ined in an incident response plan is 'undamental to ensuring 'ast and accurate handling o' security incidents. E$en i' an explicit incident response plan is lac&ing, a'ter the incident occurs the :uestions such as these might 2e as&ed 2y the company management+ #hat to do no/@ ;o/ to put it the /ay it /as@ ;o/ to pre$ent recurrence@ ;o/ /e should ha$e prepared@ Should /e try to 'igure /ho is responsi2le@

Ans/ering these :uestions re:uires &no/ledge o' your computing en$ironment, company culture and internal procedures, implemented technical security and policy countermeasures. E''ecti$e incident response 'uses together technical and non-technical resources, 2ound 2y the incident response policy, procedures and plans. Such policy should 2e continuously re'ined and impro$ed, 2ased on the [Link]<s incident history, Aust as the main security policy should 2e. To 2uild an initial incident resolution management 'rame/or& one can use SANS Six Step incident response methodology. This approach /as originally de$eloped 'or BS 5epartment o' Energy, adopted else/here in the BS go$ernment and then [Link] 2y the SANS Institute 8http+==///.[Link]=rr=/hitepapers=incident=9 The methodology includes the 'ollo/ing six steps+ #$ 2$ %$ 4$ &$ )$ Preparation Identification Containment Eradication Reco'er( *o""o+,-p

Page 3

SANS Six Step Incident Response Methodology

4$erall, the SANS methodology allo/s an [Link] to gi$e structure to the other/ise chaotic incident response /or&'lo/. The steps o' the SANS methodology are 2oth clearly de'ined and easy to 'ollo/, and most importantly, /or& in the high-stress post-incident en$ironments 'or /hich they /ere designed. Collo/ing the steps is as easy as selecting and appropriately [Link] the procedures 'or each case at hand. Bsing the SANS pre-de'ined procedures assures that an incident response /or&'lo/ /ill 2ecome relati$ely painless and the crucial steps /ill not 2e missed. Additionally, such a system /ill 'acilitate 2oth training and colla2oration 2et/een $arious response team mem2ers, /ho can share the /or&load 'or increased e''iciency. Cinally, integrating the SANS methodology into an o$erall incident response planning assures today0s IT [Link] that they ha$e a comprehensi$e approach in-place to tac&le security incidents. It also demonstrates compliance /ith industry 12est practices3, /hich is sometime associated /ith regulatory compliance. ;a$ing a repeata2le incident management process is highlighted in se$eral recent regulations, such as ;IDAA. ?et0s spend Aust a moment re$ie/ing a 'e/ &ey 'eatures o' the SANS Six Step Incident Response methodology+ The Preparation stage co$ers e$erything one should do 2e'ore handling the 'irst incident. It in$ol$es 2oth technology issues, such as preparing response and 'orensics tools, learning the en$ironment, con'iguring systems 'or optimal response and monitoring, as /ell as 2usiness issues -- such as assigning responsi2ility, 'orming a team and esta2lishing escalation procedures. Additionally, this stage co$ers the steps necessary to increase a company0s security posture and thus decrease the li&elihood and damage 'rom 'uture incidents. Security audits, patch management, employee security a/areness program and other security tas&s all ser$e to prepare the [Link] 'or incident action. Euilding a culture o' security and a secure computing en$ironment also ser$es as incident preparation. Speci'ically, esta2lishing a real-time system and net/or& security e$ent monitoring program /ill help to recei$e early /arnings a2out the hostile acti$ities as /ell as collect e$idence a'ter the incident. Dro$iding a single $ie/ into your security in'rastructure goes a long /ay to/ards 2eing more prepared and e:uipped to deal /ith the incidents as they occur as /ell as cleaning up in the a'termath. Single e$idence storage allo/s per'orming sophisticated data analysis, leading to 2etter a/areness o' threats and $ulnera2ilities. Identification is /hat happens 'irst /hen an incident is suspected or detected. 5etermining /hether the o2ser$ed e$ent does in 'act constitute an incident 8as

Page 4

de'ined a2o$e9 is o' crucial importance. !are'ul record &eeping is $ery important, since such documentation /ill 2e hea$ily used at later stages o' the response process. 4ne should record e$erything that /as o2ser$ed in relation to the incident, /hether online or in the physical en$ironment. 5uring this stage, it is important that people responsi2le 'or incident handling maintain the proper chain o' custody 8explained here http+==en./i&[Link]=/i&i=!hainFo'Fcustody as 1document or paper trail sho/ing the [Link], custody, control, trans'er, analysis, and disposition o' physical and electronic e$idence.39. !ontrary to popular opinion, this is important e$en /hen the case is ne$er destined to end up in court. Collo/ing esta2lished and appro$ed procedures /ill help the in$estigation that is internal to the company. Garious security technologies play a role in incident identi'ication. Cor example, 'ire/all, I5S, ser$er and application logs re$eal e$idence o' potentially hostile acti$ities, coming 'rom 2oth outside and inside the protected perimeter. ?ogs are o'ten tantamount in 'inding the party responsi2le 'or those acti$ities. Security e$ent correlation is essential 'or high :uality incident identi'ication, due to its a2ility to unco$er patterns in incoming security e$ent 'lo/. !ollecting $arious audit logs and correlating them in near real-time goes a long /ay to/ards ma&ing the identi'ication step o' the response process less la2orious. Additionally, incident identi'ication is greatly helped 2y 1:uali'ying3 the I5S and other alerts using other en$ironment context, such as system and application $ulnera2ilities, running applications as /ell as 2usiness $alue. Containment is /hat &eeps the incident 'rom spreading and thus incurring higher 'inancial or other loss. 5uring this stage, the incident responders /ill inter$ene and attempt to limit the damage, such as 2y tightening net/or& or host access controls, changing system pass/ords, disa2ling accounts, etc. #hile completing the a2o$e steps, one should ma&e e$ery e''ort to &eep all the potential e$idence intact, 2alancing the needs o' system o/ners and incident in$estigators. The 2ac&up o' a''ected systems is also essential at this step. This is done to preser$e the system 'or 'urther in$estigation as /ell as remediation. The important decision on /hether to continue operating the a''ected assets should 2e made 2y the appropriate authorities during this stage. Automated containment measures, such as 'ire/all 2loc&ing, system recon'iguration or 'orced 'ile integrity chec&s, and the use o' intrusion pre$entions solution 8in the inline mode9 can also 2e used, i' dri$en 2y e$ent correlation and more intelligent analytics. ;o/e$er, automated containment /ill li&ely 2ecome /idely accepted in the 'uture. Eradication is the only stage /hen the 'actors leading to the incident are eliminated or mitigated. Such 'actors o'ten include system $ulnera2ilities, unsa'e system con'igurations, out-o'-date protection so't/are or e$en imper'ect physical access control. Also, the non-technology controls such as 2uilding access policies or &ey card pri$ileges might 2e adAusted at this stage. In the case o' a

Page 5

hac&er-related incident, the a''ected systems are li&ely to 2e restored 'rom the last clean 2ac&up or re2uilt 'rom the operating system $endor media /ith all applications reinstalled. Time is most critical during the eradication stage. The 'irst response should satis'y se$eral o'ten con'licting criteria, such as accommodating the system o/ners re:uests, preser$ing e$idence, stopping the spread o' damage /hile complying to all the appropriate [Link]<s policies. Reco'er( is the stage /here the [Link]<s operations return to normal. Systems are restored and con'igured to pre$ent recurrence and are returned to regular use. To insure that the ne/ly esta2lished controls are /or&ing, the [Link] might /ant to maintain increased monitoring o' the a''ected assets 'or some period o' time. Return to production is al/ays a critical step. I' done too early, there is a signi'icant ris& o' recurrenceH i' done too late, it ris&s upsetting the 2usiness o/ners. Thus, it should 2e clearly documented in the incident procedures during the preparation stage. *o""o+,-p is an extremely important stage o' the incident response process. Iust as the preparation stage a2o$e, proper incident 'ollo/-up helps to ensure that lessons are learned 'rom the incident and that the o$erall security posture impro$es as a result. Additionally, 'ollo/-up is important in order to pre$ent the recurrence o' similar incidents. Additionally, a report on the incident is o'ten su2mitted to the senior management. It co$ers the actions ta&en, [Link] the lessons learned and also ser$es as a &no/ledge repository in case o' similar incidents in the 'uture. Collo/-up steps o'ten need to 2e distri2uted to a /ider audience than the rest o' the in$estigation process. Enterprise-/ide security &no/ledge 2ase helps to address this challenge. It /ill ensure that IT resource o/ners /ill 2e more prepared to com2at 'uture threats. To optimi.e the distri2ution o' incident in'ormation, one can use $arious 'orms and templates, prepared in ad$anced 'or di''erent types o' incidents. Droperly [Link] past incident cases should also 2e added to an [Link]-/ide security &no/ledge 2ase, in addition to the industry security resources and $ulnera2ility &no/ledge. Such materials can later 2e used 'or training ne/ incident responders as /ell as 2roader IT audience. A summary o' suggested actions might also 2e sent to the senior management.

Incident Response Tools

#hile people and processes are important, tools is /hat completes the security triangle. #hen the incident is suspected, the response team /ill need the tools to $eri'y its status, assess damage that /as incurred as /ell as can 2e occurred and then proceed to contain and reco$er 'rom the incident. This in$ol$es a /ide

Page 6

range o' tools 'rom intrusion detection to 'orensics and $ulnera2ility management. Eac&up tools should also not 2e o$erloo&ed. Tools help'ul 'or incident management can 2e [Link] as such+ Too"s E$idence collection and storage 5ata analysis and 'orensics !olla2oration Eac&up 5ocumentation Common uses durin. incident response System and security logs, audit trails, dis& images, email and other communication !orrelation, searching and reporting, 'orensics disco$ery acti$ities Incident team communication, /or&'lo/, team management E$idence preser$ation, 1&no/n good3 con'iguration retention, user data reco$ery Actions logged 'or audit and impro$ement, reporting, incident team per'ormance measurement, lessons learned, 'uture team training

Some tools are help'ul in more than one o' the a2o$e category. Cor example, a Security In'ormation Management 8SIM9 solution o'ten holds most o' the e$idence 'rom the scene o' the in'ormation security incident. Incident handling is a natural SIM product 'unctionality aimed at gathering and [Link] security e$ent data around incidents and also en'orcing proper response /or&'lo/ in order to 'acilitate e''ecti$e and prompt response to security incidents. Speci'ically, a SIM can Cacilitates the e''ecti$e handling process Integrates e$idence stora.e and ana"(sis En'orces proper access contro" to e$idence Ena2les team colla2oration Simpli'ies reso"ution monitorin. and reporting Ma&es security measura/"e In general, it esta2lishes a single control point o' the security response capa2ilities 2y com2ining the maAor potential e$idence storage /ith the in$estigati$e plat'orm. 4ther tools that an incident team needs to 2e $ery 'amiliar /ith include dis& image 'orensics tools, co$ering the /hole li'ecycle 'rom ma&ing a 'orensics copy o' the suspect0s /or&station to 'inal e$idence presentation to an internal authority or la/ en'orcement. Those tools do re:uire signi'icant training, especially i' used 'or cases /here court trial is li&ely.

Example !orporation " #orm Incident Re$isited

Page 7

A net/or& helpdes& operator recei$es calls 'rom se$eral users " all reporting computer 'ailures and slo/ net/or& response. Bsing a ne/ly esta2lished process, a trained team and right tools, an incident case is opened according to the plan and user complaints 'rom that department are [Link] and presented to all rele$ant parties, including the security team contact. The a''ected machines together /ith the in'ormation on their o/ners are also added to corresponding case 'ields. The operator then assigns the case to the security e$ent monitoring team, as mandated 2y his instructions, deri$ed 'rom the incident plan. Bpon recei$ing the assignment through the case management system, a monitoring team mem2er run se$eral :ueries searching 'or suspicious e$ents to and 'rom the a''ected machines " all as part o' the incident identification procedure de'ined 2y the company. ;e disco$ers that a net/or& I5S has detected an email /orm 2eing transmitted 'rom outside the en$ironment. The monitoring team mem2er shares the incident case /ith the security analyst team, running the intrusion detection, so they can $eri'y the impact o' the I5S e$ents, 2ased on the a''ected asset 2usiness role and importance. Many e$ents reported 2y the anti-$irus systems running on some o' the user<s des&tops /ere also reported 'rom the a''ected ID addresses. As a next step, an analyst selects a Containment procedure 'rom the &no/ledge 2ase, /hich in$ol$es :uarantining the in'ected machines 2y applying a 'ire/all rule to pre$ent the spread o' the /orm. The procedure is added to the incident case and then implemented. Next, it is necessary to clean the in'ected D!s. The ![Link] procedure in$ol$es installing and running 'ull scan using a 'reshly updated copy o' anti-$irus so't/are. The security engineering team together /ith security analyst team $eri'ies compliance o' the ne/ly installed anti-$irus system /ith the company<s anti-$irus policy. The recommended *o""o+,up procedure includes a mandated company-/ide des&top anti-$irus deployment 'rom a dedicated ser$er. The procedure is then su2mitted 'or management appro$al and, once appro$ed, the remediation team assures that the anti-$irus so't/are is pushed out to all company des&top D!0s and the incident case is closed. ;ere is another example o' ho/ a company /ith a /ell-tuned incident response process handles an attac& against the /e2 ser$er. A security analyst on duty recei$ed an email noti'ication /hen a correlated e$ent on a success'ul attac& /as triggered 2y SIM solution. An analyst has disco$ered that a real-time correlation rule /as matched 2y a series o' e$ents directed against the auxiliary /e2 ser$er. Ey logging into their SIM and running a report, the analyst has 'ound out that the triggered rule aims to detect high-se$erity attac&s against the /e2 ser$er, /hich

Page 8

are preceded 2y the reconnaissance acti$ity, such as a ser$er $ersion :uery. The /e2 ser$er /as 'irst pro2ed 'or its type and $ersion and later attac&ed 2y a &no/n exploit detected 2y the net/or& intrusion detection system. The company security monitoring procedure mandated that such 2e in$estigated. Thus, the analyst clic&ed on the correlated e$ent in the corresponding report and chose to add it to a ne/ incident case. ;e then added a note saying that he recei$ed an email noti'ication and started the in$estigation in accordance /ith the security procedure. A'ter the case /as registered 2y the system, the analyst proceeded to in$estigate the related e$ents. ;e opened the report to $ie/ the ra/ security e$ents that triggered the correlation. Such e$ents included pro2es against multiple ser$ers 'ollo/ed 2y an attac&. ;e loo&ed at the attac& details and 'ound out that the I5S signature 'or the exploit matched the ser$er type and the operating system. ;e added all the related e$ents to the incident case as /ell. Curther, he run an :uery to loo& 'or more traces o' the same attac&er0s ID address 8the source9 in the e$ent data2ase. Multiple entries indicati$e o' scanning, denied connections on the 'ire/all and T!D port J( attempts across the enterprise /ere disco$ered. The report results /ere also added to the incident case. At that stage it /as o2$ious that a consistent attac& /as in progress. The note /as added to the case Identi'ication section saying that the incident is con'irmed and se$eral ser$ers might ha$e 2een impacted. The analyst then searched all e$ents in$ol$ing the attac&er /e2 ser$er. No suspicious acti$ity has originated 'rom it. ;o/e$er, since the ser$er /as not a 2usiness critical asset, it /as possi2le to ta&e it o''line 'or in$estigation. This decision /as recorded in the !ontainment section o' the incident case and the ser$er /as ta&en o''line. The detailed ser$er in$estigation that 'ollo/ed has not re$ealed any signs o' a success'ul compromise. ;o/e$er, the ser$er logs contained e$idence o' a multiple 'ailed exploit attempts. The ser$er /as also 'ound missing se$eral critical patches. Their lac& /as apparently not detected 2y the attac&er. It /as decided to patch the ser$er 2e'ore the regular maintenance /indo/ and to return it online. It /as also decided to increase the logging le$el on the ser$er. The respecti$e note /as made in the Mitigation section o' the incident case and the a2o$e steps /ere per'ormed. A'ter the ser$er /as returned into operation, the analyst has assigned the case to the incident manager /ho had the authority to re$ie/ the per'ormed steps and to close the case. The manager added se$eral notes to the 'ollo/-up section, /hich

Page 9

suggested that ser$ers in that su2net 2e scanned 'or $ulnera2ilities more o'ten. The case /as then closed.

!ommon Mista&es o' Incident Response

#hile many [Link] are on the path to/ards [Link] their incident response, many pit'alls lay in /ait 'or them on the path to incident management nir$ana. This section [Link] se$eral mista&es that companies ma&e in their security incident response. 0 # Not 1a'in. a p"an The 'irst mista&e is simply not creating an incident response plan 2e'ore incidents start happening. ;a$ing a plan in place 8e$en a plan that is not /ell-thought9 ma&es a /orld o' di''erenceK Such plan should co$er all the stages o' incident response process 'rom preparing the in'rastructure to 'irst response all the /ay to learning the lessons o' a success'ully resol$ed incident. I' you ha$e a plan, then a'ter the initial panic phase, 8<4h, my, /e are 2eing hac&edKKK<9 you can :uic&ly mo$e into a set o' planned acti$ities, including a chance to contain the damage and cur2 the incident losses. ;a$ing a chec&list to 'ollo/ and a roster o' people to call is o' paramount importance in a stress'ul post-incident en$ironment. To Aump-start the planning acti$ity one can use a ready-made methodology, such as SANS Institute -step incident response process, co$ered a2o$e. #ith a plan and a methodology your team /ill soon 2e 2attle hardened and ready to respond to the next $irus 'aster and more e''iciently. As a result, you might manage to contain the damage to your [Link]. 0 2 *ai"in. to increase monitorin. and sur'ei""ance The second mista&e is not deploying increased monitoring and sur$eillance a'ter an incident has occurred. This is a&in to shooting yoursel' in the 'oot during the incident response. E$en though some companies cannot a''ord 24=% security monitoring, there is no excuse 'or not increasing monitoring a'ter an incident has occurred. At the $ery least, one o' the 'irst things to do a'ter an incident is to cran& up all the logging, auditing and monitoring capa2ilities in the a''ected net/or& and systems. This simple act has the potential to ma&e or 2rea& the in$estigation 2y pro$iding crucial e$idence 'or identi'ying the cause o' the incident and resol$ing it. It o'ten happens that later in the response process, the in$estigators disco$er that some critical piece o' log 'ile /as rotated a/ay or an existing monitoring 'eature /as 'orgotten in an <o''< state. ;a$ing plenty o' data on /hat /as going

Page 10

on in your IT en$ironment right a'ter the incident /ill not Aust ma&e the in$estigation easier, it /ill li&ely ma&e it success'ul. Another side 2ene'it, is that increased logging and monitoring /ill allo/ the in$estigators to con'irm that they indeed ha$e 'ollo/ed the esta2lished chain o' custody 0%$ 2ein. unprepared for a court /att"e The third mista&e is o'ten tal&ed a2out, 2ut rarely a$oided. Some experts ha$e proclaimed that e$ery security incident needs to 2e in$estigated as i' it /ill end up in court. In other /ords, maintaining 'orensic :uality and 'ollo/ing the esta2lished chain o' custody needs to 2e assured during the in$estigation. E$en i' the case loo&s as i' it /ill not go 2eyond the suspect<s manager or the human resources department 8in the case o' an internal o''ense9 or e$en the security team itsel' 8in many external hac&ing and $irus incidents9, there is al/ays a chance that it /ill end up in court. !ases ha$e gone to court a'ter ne/ e$idence /as disco$ered during an in$estigation, and, /hat /as thought to 2e a simple issue o' inappropriate #e2 access 2ecame a criminal child pornography case. Moreo$er, /hile you might not 2e expecting a legal challenge, the suspect might sue in retaliation 'or a disciplinary action against him or her. A seasoned incident in$estigator should al/ays consider this possi2ility. In addition, 'ollo/ing a high standard o' in$estigati$e :uality al/ays helps since the e$idence /ill 2e that much more relia2le and compelling, i' it can 2e 2ac&ed up 2y a thorough and /ell-documented procedure. 04$ Puttin. it /ac3 t1e +a( it +as The 'ourth mista&e is reducing your incident response to Lputting it 2ac& the /ay it /asL. This o'ten happens i' the company is under deadline to restore the 'unctionality. #hile this moti$e is understanda2le, there is a distinct possi2ility that 'ailing to 'ind out /hy the incident occurred /ill lead to repeat incidents, on the same or di''erent systems. Cor example, in the case o' a hac&ing incident, i' an unpatched machine that /as compromised is re2uilt 'rom the original 4S media, 2ut the exploited $ulnera2ility is not remo$ed, the hac&ers are $ery li&ely to come 2ac& and ta&e it o$er again. Moreo$er, the same 'ate /ill li&ely 2e'all other exposed systems. Thus, /hile returning to operation might 2e the primary goal, don0t lose sight o' the secondary goal+ 'iguring out /hat happened and ho/ to pre$ent it 'rom happening again. It 'eels 2ad to 2e on the recei$ing end o' the success'ul attac&,

Page 11

2ut it 'eels much /orse to 2e hit t/ice 2y the same threat and ha$e you de'enses 'ell in 2oth cases. Incident response should not 2e $ie/ed as a type o' L'ire'ightingL although you0d 'ight plenty o' 'ires in the process. It can clearly help in case o' a 'ire, 2ut it can also help pre$ent 'ires in the 'uture. 0&$ Not "earnin. from mista3es The 'inal mista&e sounds simple, 2ut it is all too common. It is simply not learning 'rom mista&esK !reating a great plan 'or incident response and 'ollo/ing it /ill ta&e the [Link] a long /ay to/ard securing the company, 2ut /hat is e:ually important is re'ining your plan a'ter each incident, since the team and the tools might ha$e changed o$er time. Another critical component is documenting the incident as it is occurring, not Aust a'ter the 'act. This assures that the Lgood, the 2ad and the uglyL o' the handling process /ill 2e captured, studied and lessons /ill 2e dra/n 'rom it. The results o' such e$aluations should 2e communicated to all the in$ol$ed parties, including IT resource o/ners and system administrators. Ideally, the [Link] should 2uild an incident-related &no/ledge 2ase, so that procedures are consistent and can 2e repeated in practices. The latter is $ery important 'or regulatory compliance as /ell and /ill help satis'ying some o' the Sar2anes-4xley re:uirements 'or auditing the controls to in'ormation.

!onclusion
#hile the a2o$e cases are simplistic in nature they readily sho/ the need 'or any security management system to ha$e not only an incident response plan 2ut also an integrated incident handling system to ensure complete and e''ecti$e response planning deployment. ;a$ing a highly e''icient plan helps [Link] sa$e money 2y limiting the impact on core 2usiness 'rom security incidents and increasing the e''iciency o' existing security in'rastructure in$estments. 4$erall, the SANS process allo/s one to gi$e structure to the other/ise chaotic incident response /or&'lo/. It de'ines the steps that /ill then 2e 'ollo/ed under incidentinduced stress /ith high precision. In 'act, many o' the a2o$e steps may 2e 2uilt 'rom the pre-de'ined procedures. Collo/ing the steps /ill then 2e as easy as selecting and sometimes [Link] the procedures 'or each case at hand. Incident handling /or&'lo/ /ill 2ecome more streamlined and the crucial steps /ill not 2e missed and documented properly. Bsing pre-de'ined procedures also helps train the incident response sta'' on proper actions 'or each process step. The automated system may 2e 2uilt to &eep trac& o' the response /or&'lo/, to suggest proper procedures 'or $arious steps and to securely handle incident e$idence. Additionally, such a

Page 12

system /ill 'acilitate colla2oration 2et/een $arious response team mem2ers, /ho can share the /or&load 'or increased operational e''iciency. #hat is e$en more important, monitoring incident resolution acti$ities allo/s the [Link] to implement e''ecti$e security metrics. It is one thing to count num2er o' alerts or e$ents 'lo/ing 'rom $arious sensors, 2ut to ta&e security assessment to the next le$el one needs to measure the per'ormance o' the /hole security process, in$ol$ing 2oth people 8such as security team mem2ers /or&ing on the incident cases9 and technologies. 2O-T T4E -T4OR+ !his is an updated author $io, added to the paper at the ti e of reposting in ,++-. 5r. Anton !hu$a&in 8http+==///.chu$a&in.org9 is a [Link] security expert in the 'ield o' log management and D!I 5SS compliance. ;e is an author o' 2oo&s LSecurity #arriorL and LD!I !omplianceL and a contri2utor to LMno/ Nour Enemy IIL, LIn'ormation Security Management ;and2oo&L and others. Anton has pu2lished [Link] o' papers on log management, correlation, data analysis, D!I 5SS, security management 8see list ///.in'o-secure.org9 . ;is 2log http+==///.security/[Link] is one o' the most popular in the industry. In addition, Anton teaches classes and presents at many security con'erences across the /orldH he recently addressed audiences in Bnited States, BM, Singapore, Spain, Russia and other countries. ;e /or&s on emerging security standards and ser$es on the ad$isory 2oards o' se$eral security start-ups. !urrently, Anton is de$eloping his security consulting practice, 'ocusing on logging and D!I 5SS compliance 'or security $endors and Cortune O(( [Link]. 5r. Anton !hu$a&in /as 'ormerly a 5irector o' D!I !ompliance Solutions at Pualys. Dre$iously, Anton /or&ed at ?og?ogic as a !hie' ?ogging E$angelist, tas&ed /ith educating the /orld a2out the importance o' logging 'or security, compliance and operations. Ee'ore ?og?ogic, Anton /as employed 2y a security $endor in a strategic product management role. Anton earned his Dh.5. degree 'rom Stony Eroo& Bni$ersity.

)lossary
Security event is a single o2ser$a2le occurrence as reported 2y a security de$ice or application or noticed 2y the appropriate personnel. Thus, 2oth I5S alert and security-related helpdes& call /ill :uali'y as security e$ents. Security incident is an occurrence o' one or se$eral security e$ents that ha$e a potential to cause undesired 'unctioning o' IT resources or other related

Page 13

pro2lems. Thus, that limits our discussion to in'ormation security incidents, /hich co$er computer and net/or& security, intellectual property the't and many other issues. Incident response 8or IR9 is a process o' identi'ication, containment, eradication and reco$ery 'rom computer incidents per'ormed 2y a responsi2le security team. It is /orth/hile to note, that the security team might consist o' Aust one person, /ho might only 2e a part-time incident responder. ;o/e$er, /hoe$er ta&es part in dealing /ith the incident conse:uences implicitly 2ecomes part o' the incident response team, e$en i' such team does not exist as organi.ation0s part. Incident case is a collection o' e$idence and associated /or&'lo/ related to a security incident. Thus, the case is a history o' /hat happened, /hat /as done /ith e$idence supporting 2oth items a2o$e. It might include $arious documents such as reports, security e$ent data, results o' audio inter$ie/s, images 'iles and other etc. Incident report is a document prepared as a result o' an incident case in$estigation. Incident report might 2e cryptographically signed or ha$e other assurances o' its integrity. Most incident in$estigations /ill result in the report su2mitted to appropriate authorities 8either internal or outside the company9, /hich might contain some or e$en all data associated /ith the case. It is /orth/hile to note that the term evidence is used throughout the chapter indicates any data disco$ered in the process o' incident response.

Page 14