SlideShare a Scribd company logo

Automatic patch generation learned from human written patches

Sung Kim
Sung Kim

The document describes research on automatically generating software patches from examples of human-written patches. It discusses the GenProg automated program repair system, which has been able to fix 55 out of 105 bugs using genetic programming and random mutation. The researchers aim to improve on GenProg by training a machine learning model on a large dataset of human-written patches to learn patterns for generating high-quality patches.

1 of 171
Downloaded 54 times
1
Automatic Patch Generation Learned from Human-Written Patches Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim The Hong Kong University of Science and Technology, China 24 May 2013 the 35th International Conference on Software Engineering (ICSE 2013)
2
2 Repair
3
2 Repair
4
2 Repair GenProgGenProg
5
3 GenProg C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
6
3 GenProg State-of-the-art C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
7
3 GenProg State-of-the-art Genetic Programming C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
8
3 GenProg State-of-the-art Genetic Programming Random Mutation C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
9
3 GenProg State-of-the-art Genetic Programming Random Mutation Systematically Evaluated C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
10
4 Buggy Code
11
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
12
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
13
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Null Pointer Exception Buggy Code
14
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 5 GenProg repairs bugs
15
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
16
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
17
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg This patch passes ALL test cases.
18
6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
19
6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
20
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
21
7 Would you accept?
22
7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
23
7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
24
7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
25
8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
26
8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 9.4% 90.6% 17  Students 68  Developers
27
9
28
9
29
9 •Random
30
9 •Random •Unnatural
31
9 •Random •Unnatural •Harder  to  understand
32
10
33
10
34
10 Random Evolution
35
10 Random Evolution Explores the large search space
36
10 Random Evolution Explores the large search space but sometimes produces random results
37
10 Random Evolution Intelligent Design Explores the large search space but sometimes produces random results
38
10 Random Evolution Intelligent Design Explores the large search space Leverages human knowledgebut sometimes produces random results
39
11 Human-written Patches
40
11 Human-written Patches Readable
41
11 Human-written Patches Readable Natural
42
11 Human-written Patches Readable Natural Easy to understand
43
11 Human-written Patches Readable Natural Easy to understand We can learn how to generate patches from human knowledge.
44
12
45
12 JDT
46
12 >60,000 Patches JDT
47
12 Manual Classification >60,000 Patches JDT
48
12 Manual Classification #patches Patterns >60,000 Patches JDT
49
12 Manual Classification #patches Patterns >60,000 Patches JDT
50
12 Manual Classification #patches Patterns Top frequent patterns account for >20~30% >60,000 Patches JDT
51
13 Common Fix Patterns Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3) Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3)
52
13 Common Fix Patterns Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()} Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()}
53
14 PAR Pattern-based Automatic Program Repair
54
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15
55
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15 + - + Human Knowledge
56
16 + - + Using Human Knowledge for patch generation
57
16 + - + Fix Templates Using Human Knowledge for patch generation
58
16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation
59
16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation 10
60
16 + - + Fix Templates Program Edit Script Manually created from fix patterns Using Human Knowledge for patch generation 10
61
16 + - + Fix Templates Program Edit Script Manually created from fix patterns Highly reusable Using Human Knowledge for patch generation 10
62
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 17
63
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 18 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
64
19 Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
65
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
66
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
67
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
68
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
69
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
70
19 TP TF Fault Locations Fault locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
71
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 20 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
72
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 21
73
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 22 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
74
23 How a Fix Template works
75
23 How a Fix Template works
76
23 How a Fix Template works
77
+ - + 23 How a Fix Template works
78
+ - + 23 How a Fix Template works
79
+ - + 23 How a Fix Template works
80
+ - + 23 AST Analysis →Collects necessary AST nodes How a Fix Template works
81
+ - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability How a Fix Template works
82
+ - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability Program Editing →Modifies the source code How a Fix Template works
83
Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
84
Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
85
+ - +Null Pointer Checker Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
86
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
87
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
88
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
89
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
90
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
91
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
92
25 List of Templates
93
25 List of Templates Parameter Replacer Method Replacer Parameter Adder and Remover Expression Replacer Expression Adder and Remover Object Initializer Range Checker Collection Size Checker Null Pointer Checker Class Cast Checker
94
26 Creating Patch Candidates
95
26 Creating Patch Candidates
96
26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates
97
26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates
98
26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates ... Patch Candidates
99
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 27 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
100
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 28 Patch Evaluation
101
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation
102
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
103
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
104
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired PassTest Cases
105
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired PassTest Cases
106
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired Pass Fail Test Cases
107
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired + - + - + + Fix Template Patch Candidate Fault Location Pass Fail Test Cases
108
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
109
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
110
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
111
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
112
31 Evaluation: Experiment Design
113
31 Evaluation: Experiment Design
114
31 Evaluation: Experiment Design PAR GenPro
115
31 Evaluation: Experiment Design PAR GenPro
116
31 Evaluation: Experiment Design PAR GenPro
117
31 Evaluation: Experiment Design PAR GenPro # #
118
32 RQ1(Fixability): How many bugs are fixed successfully? RQ2(Acceptability):Which approach can generate more acceptable bug patches? Evaluation: Research Questions #
119
33 Subject # bugs LOC # test cases Rhino 17 51,001 5,578 AspectJ 18 180,394 1,602 log4j 15 27,855 705 Math 29 121,168 3,538 Lang 20 54,537 2,051 Collections 20 48,049 11,577 Total 119 351,406 25,051 Experiment Subjects
120
34 RQ1: Fixability
121
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30
122
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27
123
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27 16
124
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27 16 PAR GenProg 27 16 >
125
35 RQ2: Acceptability
126
35 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
127
36
128
36 PAR
129
36 PAR GenProg
130
36 PAR GenProg 5
131
37 User Study #1 Ranking Patches
132
37 User Study #1 Bug Description Ranking Patches
133
37 User Study #1 Bug Description Buggy Code Ranking Patches
134
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Ranking Patches
135
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Ranking Patches
136
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
137
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
138
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Rank patches 17  Students 68  Developers Ranking Patches
139
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 2 1 3 Rank patches 17  Students 68  Developers Ranking Patches
140
User Study #1: Results Student  group  (avg.  ranking) 38 (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
141
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
142
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better) Significantly Different
143
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different
144
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different Significantly Different
145
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different Significantly Different PAR generates better ranking patches than GenProg
146
39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
147
39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg User Study #2: Pair-wise Comparison between Human-written Patches Vs. PAR or GenProg
148
40
149
40 PAR 27
150
40 PAR 27 GenProg 16
151
41 User Study #2 Pair-wise Comparison
152
41 User Study #2 Bug DescriptionPair-wise Comparison
153
41 User Study #2 Bug Description Buggy Code Pair-wise Comparison
154
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Pair-wise Comparison
155
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Pair-wise Comparison
156
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
157
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Patch #1 Patch #2 Both Not sure Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
158
User Study #2: Results GenProg 42 PAR
159
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR
160
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure
161
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49%
162
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32%
163
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32% PAR generates more acceptable patches than GenProg
164
43 Limitations • Fix templates are written manually. • But it is one-time cost, these are highly reusable. •We entirely re-implemented GenProg by Java. •All subjects are collected from open source projects. • Some participants may not be thoroughly qualified.
165
44 Summary
166
44 Summary Observed common patches #patches Patterns
167
44 Summary Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location
168
44 Summary Can fix more bugs with more acceptability Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%
169
45 Future Work Automatic Fix Template Identification • More templates can fix more bugs. More Test Cases • More test cases may lead us to better patches.
170
46 Acknowledgement Special Thanks to: Westley Weimer, Claire Le Goues, Thomas Zimmermann, and Christian Bird
171
47 Summary Fix Templates and PAR Can fix more bugs with more acceptability Observed common patches #patches Patterns if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%
Automatic Patch Generation Learned from Human-Written Patches Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim The Hong Kong University of Science and Technology, China 24 May 2013 the 35th International Conference on Software Engineering (ICSE 2013)
2 Repair
2 Repair
2 Repair GenProgGenProg
3 GenProg C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
3 GenProg State-of-the-art C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
3 GenProg State-of-the-art Genetic Programming C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
3 GenProg State-of-the-art Genetic Programming Random Mutation C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
3 GenProg State-of-the-art Genetic Programming Random Mutation Systematically Evaluated C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
4 Buggy Code
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Null Pointer Exception Buggy Code
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 5 GenProg repairs bugs
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg This patch passes ALL test cases.
6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
7 Would you accept?
7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 9.4% 90.6% 17  Students 68  Developers
9
9
9 •Random
9 •Random •Unnatural
9 •Random •Unnatural •Harder  to  understand
10
10
10 Random Evolution
10 Random Evolution Explores the large search space
10 Random Evolution Explores the large search space but sometimes produces random results
10 Random Evolution Intelligent Design Explores the large search space but sometimes produces random results
10 Random Evolution Intelligent Design Explores the large search space Leverages human knowledgebut sometimes produces random results
11 Human-written Patches
11 Human-written Patches Readable
11 Human-written Patches Readable Natural
11 Human-written Patches Readable Natural Easy to understand
11 Human-written Patches Readable Natural Easy to understand We can learn how to generate patches from human knowledge.
12
12 JDT
12 >60,000 Patches JDT
12 Manual Classification >60,000 Patches JDT
12 Manual Classification #patches Patterns >60,000 Patches JDT
12 Manual Classification #patches Patterns >60,000 Patches JDT
12 Manual Classification #patches Patterns Top frequent patterns account for >20~30% >60,000 Patches JDT
13 Common Fix Patterns Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3) Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3)
13 Common Fix Patterns Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()} Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()}
14 PAR Pattern-based Automatic Program Repair
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15 + - + Human Knowledge
16 + - + Using Human Knowledge for patch generation
16 + - + Fix Templates Using Human Knowledge for patch generation
16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation
16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation 10
16 + - + Fix Templates Program Edit Script Manually created from fix patterns Using Human Knowledge for patch generation 10
16 + - + Fix Templates Program Edit Script Manually created from fix patterns Highly reusable Using Human Knowledge for patch generation 10
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 17
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 18 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
19 Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
19 TP TF Fault Locations Fault locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 20 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 21
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 22 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
23 How a Fix Template works
23 How a Fix Template works
23 How a Fix Template works
+ - + 23 How a Fix Template works
+ - + 23 How a Fix Template works
+ - + 23 How a Fix Template works
+ - + 23 AST Analysis →Collects necessary AST nodes How a Fix Template works
+ - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability How a Fix Template works
+ - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability Program Editing →Modifies the source code How a Fix Template works
Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
+ - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
25 List of Templates
25 List of Templates Parameter Replacer Method Replacer Parameter Adder and Remover Expression Replacer Expression Adder and Remover Object Initializer Range Checker Collection Size Checker Null Pointer Checker Class Cast Checker
26 Creating Patch Candidates
26 Creating Patch Candidates
26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates
26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates
26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates ... Patch Candidates
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 27 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 28 Patch Evaluation
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired PassTest Cases
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired PassTest Cases
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired Pass Fail Test Cases
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired + - + - + + Fix Template Patch Candidate Fault Location Pass Fail Test Cases
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
31 Evaluation: Experiment Design
31 Evaluation: Experiment Design
31 Evaluation: Experiment Design PAR GenPro
31 Evaluation: Experiment Design PAR GenPro
31 Evaluation: Experiment Design PAR GenPro
31 Evaluation: Experiment Design PAR GenPro # #
32 RQ1(Fixability): How many bugs are fixed successfully? RQ2(Acceptability):Which approach can generate more acceptable bug patches? Evaluation: Research Questions #
33 Subject # bugs LOC # test cases Rhino 17 51,001 5,578 AspectJ 18 180,394 1,602 log4j 15 27,855 705 Math 29 121,168 3,538 Lang 20 54,537 2,051 Collections 20 48,049 11,577 Total 119 351,406 25,051 Experiment Subjects
34 RQ1: Fixability
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27 16
34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27 16 PAR GenProg 27 16 >
35 RQ2: Acceptability
35 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
36
36 PAR
36 PAR GenProg
36 PAR GenProg 5
37 User Study #1 Ranking Patches
37 User Study #1 Bug Description Ranking Patches
37 User Study #1 Bug Description Buggy Code Ranking Patches
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Ranking Patches
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Ranking Patches
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Rank patches 17  Students 68  Developers Ranking Patches
37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 2 1 3 Rank patches 17  Students 68  Developers Ranking Patches
User Study #1: Results Student  group  (avg.  ranking) 38 (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better) Significantly Different
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different Significantly Different
User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different Significantly Different PAR generates better ranking patches than GenProg
39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg User Study #2: Pair-wise Comparison between Human-written Patches Vs. PAR or GenProg
40
40 PAR 27
40 PAR 27 GenProg 16
41 User Study #2 Pair-wise Comparison
41 User Study #2 Bug DescriptionPair-wise Comparison
41 User Study #2 Bug Description Buggy Code Pair-wise Comparison
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Pair-wise Comparison
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Pair-wise Comparison
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Patch #1 Patch #2 Both Not sure Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
User Study #2: Results GenProg 42 PAR
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49%
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32%
User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32% PAR generates more acceptable patches than GenProg
43 Limitations • Fix templates are written manually. • But it is one-time cost, these are highly reusable. •We entirely re-implemented GenProg by Java. •All subjects are collected from open source projects. • Some participants may not be thoroughly qualified.
44 Summary
44 Summary Observed common patches #patches Patterns
44 Summary Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location
44 Summary Can fix more bugs with more acceptability Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%
45 Future Work Automatic Fix Template Identification • More templates can fix more bugs. More Test Cases • More test cases may lead us to better patches.
46 Acknowledgement Special Thanks to: Westley Weimer, Claire Le Goues, Thomas Zimmermann, and Christian Bird
47 Summary Fix Templates and PAR Can fix more bugs with more acceptability Observed common patches #patches Patterns if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%
Ad

Recommended

Automatic Patch Generation Learned from Human-Written Patches
Automatic Patch Generation Learned from Human-Written PatchesAutomatic Patch Generation Learned from Human-Written Patches
Automatic Patch Generation Learned from Human-Written Patches
Dongsun Kim
 
Demo day (june) at erpnext
Demo day (june) at erpnextDemo day (june) at erpnext
Demo day (june) at erpnext
Nabin Hait
 
How Do Software Engineers Understand Code Changes? FSE 2012
How Do Software Engineers Understand Code Changes? FSE 2012How Do Software Engineers Understand Code Changes? FSE 2012
How Do Software Engineers Understand Code Changes? FSE 2012
Sung Kim
 
The Anatomy of Developer Social Networks
The Anatomy of Developer Social NetworksThe Anatomy of Developer Social Networks
The Anatomy of Developer Social Networks
Sung Kim
 
Crowd debugging (FSE 2015)
Crowd debugging (FSE 2015)Crowd debugging (FSE 2015)
Crowd debugging (FSE 2015)
Sung Kim
 
Automatically Generated Patches as Debugging Aids: A Human Study (FSE 2014)
Automatically Generated Patches as Debugging Aids: A Human Study (FSE 2014)Automatically Generated Patches as Debugging Aids: A Human Study (FSE 2014)
Automatically Generated Patches as Debugging Aids: A Human Study (FSE 2014)
Sung Kim
 
REMI: Defect Prediction for Efficient API Testing (

ESEC/FSE 2015, Industria...
REMI: Defect Prediction for Efficient API Testing (

ESEC/FSE 2015, Industria...REMI: Defect Prediction for Efficient API Testing (

ESEC/FSE 2015, Industria...
REMI: Defect Prediction for Efficient API Testing (

ESEC/FSE 2015, Industria...
Sung Kim
 
Source code comprehension on evolving software
Source code comprehension on evolving softwareSource code comprehension on evolving software
Source code comprehension on evolving software
Sung Kim
 
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
Sung Kim
 
A Survey on Dynamic Symbolic Execution for Automatic Test Generation
A Survey on Dynamic Symbolic Execution for Automatic Test GenerationA Survey on Dynamic Symbolic Execution for Automatic Test Generation
A Survey on Dynamic Symbolic Execution for Automatic Test Generation
Sung Kim
 
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Sung Kim
 
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
Sung Kim
 
Personalized Defect Prediction
Personalized Defect PredictionPersonalized Defect Prediction
Personalized Defect Prediction
Sung Kim
 
A Survey on Automatic Test Generation and Crash Reproduction
A Survey on Automatic Test Generation and Crash ReproductionA Survey on Automatic Test Generation and Crash Reproduction
A Survey on Automatic Test Generation and Crash Reproduction
Sung Kim
 
STAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash ReproductionSTAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash Reproduction
Sung Kim
 
Transfer defect learning
Transfer defect learningTransfer defect learning
Transfer defect learning
Sung Kim
 
Tensor board
Tensor boardTensor board
Tensor board
Sung Kim
 
Survey on Software Defect Prediction
Survey on Software Defect PredictionSurvey on Software Defect Prediction
Survey on Software Defect Prediction
Sung Kim
 
Time series classification
Time series classificationTime series classification
Time series classification
Sung Kim
 
Concepts of cutover planning and management
Concepts of cutover planning and managementConcepts of cutover planning and management
Concepts of cutover planning and management
Sanjay Choubey
 
Salesforce Intro
Salesforce IntroSalesforce Intro
Salesforce Intro
Rich Helton
 
Salesforce Presentation
Salesforce PresentationSalesforce Presentation
Salesforce Presentation
Chetna Purohit
 
Making JavaScript Libraries More Approachable
Making JavaScript Libraries More ApproachableMaking JavaScript Libraries More Approachable
Making JavaScript Libraries More Approachable
Pamela Fox
 
C言語静的解析ツールと Ruby 1.9 trunk
C言語静的解析ツールと Ruby 1.9 trunkC言語静的解析ツールと Ruby 1.9 trunk
C言語静的解析ツールと Ruby 1.9 trunk
ikegami__
 
A JIT Smalltalk VM written in itself
A JIT Smalltalk VM written in itselfA JIT Smalltalk VM written in itself
A JIT Smalltalk VM written in itself
ESUG
 
Automata presentation turing machine programming techniques
Automata presentation turing machine programming techniquesAutomata presentation turing machine programming techniques
Automata presentation turing machine programming techniques
Basit Hussain
 
Objectives Assignment 09 Applications of Stacks COS.docx
Objectives Assignment 09 Applications of Stacks COS.docxObjectives Assignment 09 Applications of Stacks COS.docx
Objectives Assignment 09 Applications of Stacks COS.docx
dunhamadell
 
Analyzing Program Similarities and Differences Across Multiple Languages
Analyzing Program Similarities and Differences Across Multiple LanguagesAnalyzing Program Similarities and Differences Across Multiple Languages
Analyzing Program Similarities and Differences Across Multiple Languages
Universidad de los Andes
 
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
Functional Thursday
 
Plagiarism introduction
Plagiarism introductionPlagiarism introduction
Plagiarism introduction
Merin Paul
 

More Related Content

Viewers also liked (14)

How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
Sung Kim
 
A Survey on Dynamic Symbolic Execution for Automatic Test Generation
A Survey on Dynamic Symbolic Execution for Automatic Test GenerationA Survey on Dynamic Symbolic Execution for Automatic Test Generation
A Survey on Dynamic Symbolic Execution for Automatic Test Generation
Sung Kim
 
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Sung Kim
 
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
Sung Kim
 
Personalized Defect Prediction
Personalized Defect PredictionPersonalized Defect Prediction
Personalized Defect Prediction
Sung Kim
 
A Survey on Automatic Test Generation and Crash Reproduction
A Survey on Automatic Test Generation and Crash ReproductionA Survey on Automatic Test Generation and Crash Reproduction
A Survey on Automatic Test Generation and Crash Reproduction
Sung Kim
 
STAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash ReproductionSTAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash Reproduction
Sung Kim
 
Transfer defect learning
Transfer defect learningTransfer defect learning
Transfer defect learning
Sung Kim
 
Tensor board
Tensor boardTensor board
Tensor board
Sung Kim
 
Survey on Software Defect Prediction
Survey on Software Defect PredictionSurvey on Software Defect Prediction
Survey on Software Defect Prediction
Sung Kim
 
Time series classification
Time series classificationTime series classification
Time series classification
Sung Kim
 
Concepts of cutover planning and management
Concepts of cutover planning and managementConcepts of cutover planning and management
Concepts of cutover planning and management
Sanjay Choubey
 
Salesforce Intro
Salesforce IntroSalesforce Intro
Salesforce Intro
Rich Helton
 
Salesforce Presentation
Salesforce PresentationSalesforce Presentation
Salesforce Presentation
Chetna Purohit
 
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2...
Sung Kim
 
A Survey on Dynamic Symbolic Execution for Automatic Test Generation
A Survey on Dynamic Symbolic Execution for Automatic Test GenerationA Survey on Dynamic Symbolic Execution for Automatic Test Generation
A Survey on Dynamic Symbolic Execution for Automatic Test Generation
Sung Kim
 
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Sung Kim
 
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
Sung Kim
 
Personalized Defect Prediction
Personalized Defect PredictionPersonalized Defect Prediction
Personalized Defect Prediction
Sung Kim
 
A Survey on Automatic Test Generation and Crash Reproduction
A Survey on Automatic Test Generation and Crash ReproductionA Survey on Automatic Test Generation and Crash Reproduction
A Survey on Automatic Test Generation and Crash Reproduction
Sung Kim
 
STAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash ReproductionSTAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash Reproduction
Sung Kim
 
Transfer defect learning
Transfer defect learningTransfer defect learning
Transfer defect learning
Sung Kim
 
Tensor board
Tensor boardTensor board
Tensor board
Sung Kim
 
Survey on Software Defect Prediction
Survey on Software Defect PredictionSurvey on Software Defect Prediction
Survey on Software Defect Prediction
Sung Kim
 
Time series classification
Time series classificationTime series classification
Time series classification
Sung Kim
 
Concepts of cutover planning and management
Concepts of cutover planning and managementConcepts of cutover planning and management
Concepts of cutover planning and management
Sanjay Choubey
 
Salesforce Intro
Salesforce IntroSalesforce Intro
Salesforce Intro
Rich Helton
 
Salesforce Presentation
Salesforce PresentationSalesforce Presentation
Salesforce Presentation
Chetna Purohit
 

Similar to Automatic patch generation learned from human written patches (18)

Making JavaScript Libraries More Approachable
Making JavaScript Libraries More ApproachableMaking JavaScript Libraries More Approachable
Making JavaScript Libraries More Approachable
Pamela Fox
 
C言語静的解析ツールと Ruby 1.9 trunk
C言語静的解析ツールと Ruby 1.9 trunkC言語静的解析ツールと Ruby 1.9 trunk
C言語静的解析ツールと Ruby 1.9 trunk
ikegami__
 
A JIT Smalltalk VM written in itself
A JIT Smalltalk VM written in itselfA JIT Smalltalk VM written in itself
A JIT Smalltalk VM written in itself
ESUG
 
Automata presentation turing machine programming techniques
Automata presentation turing machine programming techniquesAutomata presentation turing machine programming techniques
Automata presentation turing machine programming techniques
Basit Hussain
 
Objectives Assignment 09 Applications of Stacks COS.docx
Objectives Assignment 09 Applications of Stacks COS.docxObjectives Assignment 09 Applications of Stacks COS.docx
Objectives Assignment 09 Applications of Stacks COS.docx
dunhamadell
 
Analyzing Program Similarities and Differences Across Multiple Languages
Analyzing Program Similarities and Differences Across Multiple LanguagesAnalyzing Program Similarities and Differences Across Multiple Languages
Analyzing Program Similarities and Differences Across Multiple Languages
Universidad de los Andes
 
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
Functional Thursday
 
Plagiarism introduction
Plagiarism introductionPlagiarism introduction
Plagiarism introduction
Merin Paul
 
Java 8 Puzzlers as it was presented at Codemash 2017
Java 8 Puzzlers as it was presented at Codemash 2017Java 8 Puzzlers as it was presented at Codemash 2017
Java 8 Puzzlers as it was presented at Codemash 2017
Baruch Sadogursky
 
Opal compiler
Opal compilerOpal compiler
Opal compiler
Jorge Ressia
 
Binary tree operations in data structures
Binary tree operations in data structuresBinary tree operations in data structures
Binary tree operations in data structures
Kalpana Mohan
 
An approach to Programming Contests with C++
An approach to Programming Contests with C++An approach to Programming Contests with C++
An approach to Programming Contests with C++
VNIT-ACM Student Chapter
 
2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices
2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices
2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices
Johannes Hoppe
 
Continuation Passing Style and Macros in Clojure - Jan 2012
Continuation Passing Style and Macros in Clojure - Jan 2012Continuation Passing Style and Macros in Clojure - Jan 2012
Continuation Passing Style and Macros in Clojure - Jan 2012
Leonardo Borges
 
Data Representation of Strings
Data Representation of StringsData Representation of Strings
Data Representation of Strings
Prof Ansari
 
Good Code
Good CodeGood Code
Good Code
Kevlin Henney
 
HacknamStyle Plaid CTF write-up
HacknamStyle Plaid CTF write-upHacknamStyle Plaid CTF write-up
HacknamStyle Plaid CTF write-up
vanhoefm
 
Java Pitfalls and Good-to-Knows
Java Pitfalls and Good-to-KnowsJava Pitfalls and Good-to-Knows
Java Pitfalls and Good-to-Knows
Miquel Martin
 
Making JavaScript Libraries More Approachable
Making JavaScript Libraries More ApproachableMaking JavaScript Libraries More Approachable
Making JavaScript Libraries More Approachable
Pamela Fox
 
C言語静的解析ツールと Ruby 1.9 trunk
C言語静的解析ツールと Ruby 1.9 trunkC言語静的解析ツールと Ruby 1.9 trunk
C言語静的解析ツールと Ruby 1.9 trunk
ikegami__
 
A JIT Smalltalk VM written in itself
A JIT Smalltalk VM written in itselfA JIT Smalltalk VM written in itself
A JIT Smalltalk VM written in itself
ESUG
 
Automata presentation turing machine programming techniques
Automata presentation turing machine programming techniquesAutomata presentation turing machine programming techniques
Automata presentation turing machine programming techniques
Basit Hussain
 
Objectives Assignment 09 Applications of Stacks COS.docx
Objectives Assignment 09 Applications of Stacks COS.docxObjectives Assignment 09 Applications of Stacks COS.docx
Objectives Assignment 09 Applications of Stacks COS.docx
dunhamadell
 
Analyzing Program Similarities and Differences Across Multiple Languages
Analyzing Program Similarities and Differences Across Multiple LanguagesAnalyzing Program Similarities and Differences Across Multiple Languages
Analyzing Program Similarities and Differences Across Multiple Languages
Universidad de los Andes
 
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
[FT-11][suhorng] “Poor Man's” Undergraduate Compilers
Functional Thursday
 
Plagiarism introduction
Plagiarism introductionPlagiarism introduction
Plagiarism introduction
Merin Paul
 
Java 8 Puzzlers as it was presented at Codemash 2017
Java 8 Puzzlers as it was presented at Codemash 2017Java 8 Puzzlers as it was presented at Codemash 2017
Java 8 Puzzlers as it was presented at Codemash 2017
Baruch Sadogursky
 
Opal compiler
Opal compilerOpal compiler
Opal compiler
Jorge Ressia
 
Binary tree operations in data structures
Binary tree operations in data structuresBinary tree operations in data structures
Binary tree operations in data structures
Kalpana Mohan
 
An approach to Programming Contests with C++
An approach to Programming Contests with C++An approach to Programming Contests with C++
An approach to Programming Contests with C++
VNIT-ACM Student Chapter
 
2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices
2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices
2013-02-21 - .NET UG Rhein-Neckar: JavaScript Best Practices
Johannes Hoppe
 
Continuation Passing Style and Macros in Clojure - Jan 2012
Continuation Passing Style and Macros in Clojure - Jan 2012Continuation Passing Style and Macros in Clojure - Jan 2012
Continuation Passing Style and Macros in Clojure - Jan 2012
Leonardo Borges
 
Data Representation of Strings
Data Representation of StringsData Representation of Strings
Data Representation of Strings
Prof Ansari
 
Good Code
Good CodeGood Code
Good Code
Kevlin Henney
 
HacknamStyle Plaid CTF write-up
HacknamStyle Plaid CTF write-upHacknamStyle Plaid CTF write-up
HacknamStyle Plaid CTF write-up
vanhoefm
 
Java Pitfalls and Good-to-Knows
Java Pitfalls and Good-to-KnowsJava Pitfalls and Good-to-Knows
Java Pitfalls and Good-to-Knows
Miquel Martin
 
Ad

More from Sung Kim (13)

DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence LearningDeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
Sung Kim
 
Deep API Learning (FSE 2016)
Deep API Learning (FSE 2016)Deep API Learning (FSE 2016)
Deep API Learning (FSE 2016)
Sung Kim
 
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Sung Kim
 
A Survey on Automatic Software Evolution Techniques
A Survey on Automatic Software Evolution TechniquesA Survey on Automatic Software Evolution Techniques
A Survey on Automatic Software Evolution Techniques
Sung Kim
 
Software Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled DatasetsSoftware Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled Datasets
Sung Kim
 
MSR2014 opening
MSR2014 openingMSR2014 opening
MSR2014 opening
Sung Kim
 
Defect, defect, defect: PROMISE 2012 Keynote
Defect, defect, defect: PROMISE 2012 Keynote Defect, defect, defect: PROMISE 2012 Keynote
Defect, defect, defect: PROMISE 2012 Keynote
Sung Kim
 
Predicting Recurring Crash Stacks (ASE 2012)
Predicting Recurring Crash Stacks (ASE 2012)Predicting Recurring Crash Stacks (ASE 2012)
Predicting Recurring Crash Stacks (ASE 2012)
Sung Kim
 
Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...
Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...
Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...
Sung Kim
 
Software Development Meets the Wisdom of Crowds
Software Development Meets the Wisdom of CrowdsSoftware Development Meets the Wisdom of Crowds
Software Development Meets the Wisdom of Crowds
Sung Kim
 
BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)
BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)
BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)
Sung Kim
 
Self-defending software: Automatically patching errors in deployed software ...
Self-defending software: Automatically patching errors in deployed software ...Self-defending software: Automatically patching errors in deployed software ...
Self-defending software: Automatically patching errors in deployed software ...
Sung Kim
 
ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)
ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)
ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)
Sung Kim
 
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence LearningDeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
Sung Kim
 
Deep API Learning (FSE 2016)
Deep API Learning (FSE 2016)Deep API Learning (FSE 2016)
Deep API Learning (FSE 2016)
Sung Kim
 
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Sung Kim
 
A Survey on Automatic Software Evolution Techniques
A Survey on Automatic Software Evolution TechniquesA Survey on Automatic Software Evolution Techniques
A Survey on Automatic Software Evolution Techniques
Sung Kim
 
Software Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled DatasetsSoftware Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled Datasets
Sung Kim
 
MSR2014 opening
MSR2014 openingMSR2014 opening
MSR2014 opening
Sung Kim
 
Defect, defect, defect: PROMISE 2012 Keynote
Defect, defect, defect: PROMISE 2012 Keynote Defect, defect, defect: PROMISE 2012 Keynote
Defect, defect, defect: PROMISE 2012 Keynote
Sung Kim
 
Predicting Recurring Crash Stacks (ASE 2012)
Predicting Recurring Crash Stacks (ASE 2012)Predicting Recurring Crash Stacks (ASE 2012)
Predicting Recurring Crash Stacks (ASE 2012)
Sung Kim
 
Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...
Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...
Puzzle-Based Automatic Testing: Bringing Humans Into the Loop by Solving Puzz...
Sung Kim
 
Software Development Meets the Wisdom of Crowds
Software Development Meets the Wisdom of CrowdsSoftware Development Meets the Wisdom of Crowds
Software Development Meets the Wisdom of Crowds
Sung Kim
 
BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)
BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)
BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)
Sung Kim
 
Self-defending software: Automatically patching errors in deployed software ...
Self-defending software: Automatically patching errors in deployed software ...Self-defending software: Automatically patching errors in deployed software ...
Self-defending software: Automatically patching errors in deployed software ...
Sung Kim
 
ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)
ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)
ReCrash: Making crashes reproducible by preserving object states (ECOOP 2008)
Sung Kim
 
Ad

Automatic patch generation learned from human written patches

  • 1. Automatic Patch Generation Learned from Human-Written Patches Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim The Hong Kong University of Science and Technology, China 24 May 2013 the 35th International Conference on Software Engineering (ICSE 2013)
  • 5. 3 GenProg C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  • 6. 3 GenProg State-of-the-art C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  • 7. 3 GenProg State-of-the-art Genetic Programming C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  • 8. 3 GenProg State-of-the-art Genetic Programming Random Mutation C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  • 9. 3 GenProg State-of-the-art Genetic Programming Random Mutation Systematically Evaluated C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  • 11. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
  • 12. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
  • 13. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Null Pointer Exception Buggy Code
  • 14. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 5 GenProg repairs bugs
  • 15. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
  • 16. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
  • 17. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg This patch passes ALL test cases.
  • 18. 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
  • 19. 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
  • 20. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
  • 22. 7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
  • 23. 7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
  • 24. 7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
  • 25. 8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
  • 26. 8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 9.4% 90.6% 17  Students 68  Developers
  • 27. 9
  • 28. 9
  • 32. 10
  • 33. 10
  • 36. 10 Random Evolution Explores the large search space but sometimes produces random results
  • 37. 10 Random Evolution Intelligent Design Explores the large search space but sometimes produces random results
  • 38. 10 Random Evolution Intelligent Design Explores the large search space Leverages human knowledgebut sometimes produces random results
  • 43. 11 Human-written Patches Readable Natural Easy to understand We can learn how to generate patches from human knowledge.
  • 44. 12
  • 51. 13 Common Fix Patterns Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3) Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3)
  • 52. 13 Common Fix Patterns Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()} Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()}
  • 54. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15
  • 55. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15 + - + Human Knowledge
  • 58. 16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation
  • 59. 16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation 10
  • 60. 16 + - + Fix Templates Program Edit Script Manually created from fix patterns Using Human Knowledge for patch generation 10
  • 61. 16 + - + Fix Templates Program Edit Script Manually created from fix patterns Highly reusable Using Human Knowledge for patch generation 10
  • 62. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 17
  • 63. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 18 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
  • 64. 19 Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 65. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 66. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 67. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 68. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 69. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 70. 19 TP TF Fault Locations Fault locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  • 71. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 20 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
  • 72. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 21
  • 73. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 22 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
  • 74. 23 How a Fix Template works
  • 75. 23 How a Fix Template works
  • 76. 23 How a Fix Template works
  • 77. + - + 23 How a Fix Template works
  • 78. + - + 23 How a Fix Template works
  • 79. + - + 23 How a Fix Template works
  • 80. + - + 23 AST Analysis →Collects necessary AST nodes How a Fix Template works
  • 81. + - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability How a Fix Template works
  • 82. + - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability Program Editing →Modifies the source code How a Fix Template works
  • 83. Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 84. Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 85. + - +Null Pointer Checker Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 86. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 87. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 88. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 89. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 90. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 91. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  • 93. 25 List of Templates Parameter Replacer Method Replacer Parameter Adder and Remover Expression Replacer Expression Adder and Remover Object Initializer Range Checker Collection Size Checker Null Pointer Checker Class Cast Checker
  • 96. 26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates
  • 97. 26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates
  • 98. 26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates ... Patch Candidates
  • 99. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 27 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
  • 100. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 28 Patch Evaluation
  • 101. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation
  • 102. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
  • 103. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
  • 104. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired PassTest Cases
  • 105. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired PassTest Cases
  • 106. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired Pass Fail Test Cases
  • 107. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired + - + - + + Fix Template Patch Candidate Fault Location Pass Fail Test Cases
  • 108. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  • 109. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  • 110. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  • 111. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  • 118. 32 RQ1(Fixability): How many bugs are fixed successfully? RQ2(Acceptability):Which approach can generate more acceptable bug patches? Evaluation: Research Questions #
  • 119. 33 Subject # bugs LOC # test cases Rhino 17 51,001 5,578 AspectJ 18 180,394 1,602 log4j 15 27,855 705 Math 29 121,168 3,538 Lang 20 54,537 2,051 Collections 20 48,049 11,577 Total 119 351,406 25,051 Experiment Subjects
  • 126. 35 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
  • 127. 36
  • 128. 36 PAR
  • 132. 37 User Study #1 Bug Description Ranking Patches
  • 133. 37 User Study #1 Bug Description Buggy Code Ranking Patches
  • 134. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Ranking Patches
  • 135. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Ranking Patches
  • 136. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
  • 137. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
  • 138. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Rank patches 17  Students 68  Developers Ranking Patches
  • 139. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 2 1 3 Rank patches 17  Students 68  Developers Ranking Patches
  • 140. User Study #1: Results Student  group  (avg.  ranking) 38 (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
  • 141. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
  • 142. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better) Significantly Different
  • 143. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different
  • 144. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different Significantly Different
  • 145. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenPro g Human Significantly Different Significantly Different PAR generates better ranking patches than GenProg
  • 146. 39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
  • 147. 39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg User Study #2: Pair-wise Comparison between Human-written Patches Vs. PAR or GenProg
  • 148. 40
  • 152. 41 User Study #2 Bug DescriptionPair-wise Comparison
  • 153. 41 User Study #2 Bug Description Buggy Code Pair-wise Comparison
  • 154. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Pair-wise Comparison
  • 155. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Pair-wise Comparison
  • 156. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
  • 157. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Patch #1 Patch #2 Both Not sure Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
  • 158. User Study #2: Results GenProg 42 PAR
  • 159. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR
  • 160. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure
  • 161. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49%
  • 162. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32%
  • 163. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32% PAR generates more acceptable patches than GenProg
  • 164. 43 Limitations • Fix templates are written manually. • But it is one-time cost, these are highly reusable. •We entirely re-implemented GenProg by Java. •All subjects are collected from open source projects. • Some participants may not be thoroughly qualified.
  • 167. 44 Summary Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location
  • 168. 44 Summary Can fix more bugs with more acceptability Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%
  • 169. 45 Future Work Automatic Fix Template Identification • More templates can fix more bugs. More Test Cases • More test cases may lead us to better patches.
  • 170. 46 Acknowledgement Special Thanks to: Westley Weimer, Claire Le Goues, Thomas Zimmermann, and Christian Bird
  • 171. 47 Summary Fix Templates and PAR Can fix more bugs with more acceptability Observed common patches #patches Patterns if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%