Art of Computer Science Research Planning
Download as PPTX, PDF1 like6,302 views
This document summarizes a talk given by Prof. Abhik Roychoudhury about skills needed for a PhD. He discusses obvious skills like analyzing papers and identifying research trends. Less obvious skills include choosing impactful problems and determining if one has the right background. The least obvious skill is determining what constitutes a research contribution, which is qualitative rather than quantitative. The talk provides examples of different types of contributions and emphasizes choosing an interesting research area and topic, considering its relevance over time, potential for translation, and avoiding negative perceptions from the community.
![Nature of contribution
in CS
• Technical Contribution
• [Transplantation / Adaptation style] Amir Pnueli
brought ideas from philosophy to build temporal
logics for specifying properties of programs.
• [Cross-fertilization style] Clarke, Emerson and Sifakis
developed model checking as an automated
procedure for checking programs
• Today, these are being used to check critical
properties of SW in the car you are driving.
• Also, used for checking Intel processors we are using
ISSTA 2019 Doctoral Symposium](https://image.slidesharecdn.com/doctoralsymposium-190719041607/85/Art-of-Computer-Science-Research-Planning-7-320.jpg)
![Imagination
• Satisfying the intellectual curiosity
• Beyond utility
• Does it touch their mind
• [Contrast]
• Does this attempt a useful problem
• [Testing – our example]
• Is this a practical technology
• Not strictly enforced when an area starts
• [Symbolic Execution was not practical when it
started]
ISSTA 2019 Doctoral Symposium](https://image.slidesharecdn.com/doctoralsymposium-190719041607/85/Art-of-Computer-Science-Research-Planning-20-320.jpg)
![Perception
• Avoid negative perceptions such as:
• Been there done that.
• [Example] Use formal methods to verify critical software
• [Perception] Tried it (20 years ago in 1998) too slow !
• [Reality] More practical in 2018-17 due to powerful
solvers being developed?
• You need to write papers with clarity to avoid such
perceptions.
ISSTA 2019 Doctoral Symposium](https://image.slidesharecdn.com/doctoralsymposium-190719041607/85/Art-of-Computer-Science-Research-Planning-26-320.jpg)
Art of Computer Science Research Planning
- 1. ISSTA19 Doctoral Symposium Art of Research Planning Prof. Abhik Roychoudhury National University of Singapore ISSTA 2019 Doctoral Symposium All comments in this talk are my own, and for discussion only.
- 2. Skills needed in PhD Obvious ones • Reading a paper and quickly judging a contribution. • Compare and contrast the contributions of each paper, and chart the growth of a research area over time. • Identify certain emerging research trends alluded to in recent publications, and check their feasibility ISSTA 2019 Doctoral Symposium
- 3. Obvious Skills needed Obvious ones • Why are they obvious? • They involve effort and discipline, of course! • They involve analytical skills, given the papers you need to analyze them, and distill out the positioning of the papers. • Involves looking outside, and doing so analytically • In research, you are expected to be analytical • It is no big deal ISSTA 2019 Doctoral Symposium
- 4. Less Obvious Skills needed Less Obvious ones • Choosing the right problems at the right time • Why will it be impactful if you study it now? • What technical skills and background are needed, AND • Why you are the right person right now to tackle this problem? • Looking inside yourself ISSTA 2019 Doctoral Symposium
- 5. Least Obvious Skills needed Least Obvious ones • Learning to decide what is a research contribution? • An (ISSTA) paper, A degree, x number of citations ?? •NO!! • Contribution is how you progress the field • It is argumentative and Qualitative in nature, as opposed to quantitative • Paper in a top conference is only a quantitative measure ISSTA 2019 Doctoral Symposium
- 6. Nature of contribution in CS • Conceptual Contribution • Focus is on the imagination such as invention of program representations by Fran Allen, who was a school-teacher in Peru (NY) • She revolutionized compilation, so that you can compile your programs today for your assignment ISSTA 2019 Doctoral Symposium
- 7. Nature of contribution in CS • Technical Contribution • [Transplantation / Adaptation style] Amir Pnueli brought ideas from philosophy to build temporal logics for specifying properties of programs. • [Cross-fertilization style] Clarke, Emerson and Sifakis developed model checking as an automated procedure for checking programs • Today, these are being used to check critical properties of SW in the car you are driving. • Also, used for checking Intel processors we are using ISSTA 2019 Doctoral Symposium
- 8. Nature of contribution in CS • Utilitarian Contribution • e.g. build a novel computer system which acts a great enabler. • Need to be careful that it is not implementation of known ideas. • Conveys important design goals and principles. • Invention of Unix operating system by Ken Thompson, which has influenced Apple’s MacOS which you and I are running right now. • Technical and utilitarian contributions often go hand- in-hand. ISSTA 2019 Doctoral Symposium
- 9. Make your own Contribution ISSTA 2019 Doctoral Symposium You have several ideas, and you can even evaluate their Technical, Conceptual Contributions Which one to choose?
- 10. Research area ISSTA 2019 Doctoral Symposium Is it interesting to YOU? Is it topical? Are you the only one interested? Is it useful even in indirect fashion? …. Not all the questions need to be answered as YES Ask yourself
- 11. Growth ISSTA 2019 Doctoral Symposium Will my research area remain relevant ? Will it contribute to grow? What are the disruptive technologies which may aid or prevent its growth. Ask yourself Super-hot areas can become super-cold quickly, Why (Question for the group) Certain areas are broad and ever-green e.g. PL, SE. Comes with choices
- 12. Research Topic • Who would be interested? • Who would fund it, if they had 1B? • Only a thought experiment • Does not preclude theoretical research • Level of interest in a topic in research community is not necessarily linked translation. ISSTA 2019 Doctoral Symposium
- 13. Translation • Translation is not an immediate need • Yet it remains a key goal. • Tough to find a balance • Never translate anything • Focus only on translation? • Try to see a pathway … ISSTA 2019 Doctoral Symposium
- 14. Examples from real-life • I focus on my own community • Software testing via symbolic execution • Proposed in 1976 and then again in 2005. • Aided by growth of Constraint Solver capability • Uptake from some companies in 10 years • Some of the research is quite theoretical • Heavy investment for a company initially • Pays off in long term, used in multi-national corporations today. ISSTA 2019 Doctoral Symposium
- 15. Example area ISSTA 2019 Doctoral Symposium Software Seed Tests Engine (calculate path for seed tests, convert it to logic, mutate logical formula to find inputs which go through other paths, solve mutated logic formula and get more inputs … ) Test suite covering many paths Aided by growth in solver technology
- 16. Another example direction ISSTA 2019 Doctoral Symposium Debugging Buggy Program Failing Test Explanation Less theoretical in nature Often involves huge search Searching in time Searching in space Lot of heuristics Very active 1997 – 2016 …. Less papers now? Lot of sctivity in. related field of Automated Program Repair. Keep an eye. You could be the leader of the next wave.
- 17. Translation of research • Translation gives your research more credibility • Each research community goes in waves • Interest in a technology often reduces. • Translation over a period of time can sustain that interest. • Translation can also move the area into other directions over a longer time e.g. decades • Why a certain topic is not translated? ISSTA 2019 Doctoral Symposium
- 18. Lack of Translation • Little bit of a story of start-up success • Why is a start-up not successful • Not enough investment • Steady marketing blitz • … • Alternate ways of doing it • Inferior technology • … • Falls through the cracks. • Need investment for translation, may not be in funds. This is the nuanced aspect. ISSTA 2019 Doctoral Symposium
- 19. Research Topic • Who would be interested? • Who would fund it, if they had 1B? • Only a thought experiment • Does not preclude theoretical research • Level of interest in a topic in research community is not necessarily linked translation. • Time-line for deployment • 5-50 years • Thought experiment only ISSTA 2019 Doctoral Symposium
- 20. Imagination • Satisfying the intellectual curiosity • Beyond utility • Does it touch their mind • [Contrast] • Does this attempt a useful problem • [Testing – our example] • Is this a practical technology • Not strictly enforced when an area starts • [Symbolic Execution was not practical when it started] ISSTA 2019 Doctoral Symposium
- 21. Contribution • Technical contribution • Smart-ness of solution • Narrowly focused on Problem solving • Conceptual contribution • Charting out an area • More focused on Problem Formulation • Utilitarian Contribution • Translation and Deployment ISSTA 2019 Doctoral Symposium
- 22. Timing of a topic • Not too ahead • Ultra-futuristic • Not today’s issue • Implementation only • Tomorrow’s issue • Symbolic execution took off after SMT solvers became mature 1995- 2005 roughly. ISSTA 2019 Doctoral Symposium
- 23. Timing of an area • PhD takes 5 years • A research area in CS which is hot in 2019 vey unlikely to be hot in 2024 • Your job will be affected by what is hot in 2023-24 • Example period 1995-2000 • Computer Networking was hot in 1995-96 • There was a huge supply of Networking graduates in 2000-01. ISSTA 2019 Doctoral Symposium
- 24. Timing of CS itself • Boom and bust cycles in CS • We are seeing a boom right now • Hard to know how long it lasts • Example: 1997 vs 2002. • Dot com internet startup boom 1996 – 2001. • Different situation 2003 – 2010 or so. • These are global factors which affects ALL CS graduates! ISSTA 2019 Doctoral Symposium
- 25. Perception • Avoid negative perception of community • Fitting a solution to a problem • You have a hammer everything looks like a nail • Example: Testing via Symbolic Execution • Problem: Testing of XYZ software • Technique: Symbolic Execution • Question: Is symbolic execution an overkill for XYZ software could simpler techniques be used? ISSTA 2019 Doctoral Symposium
- 26. Perception • Avoid negative perceptions such as: • Been there done that. • [Example] Use formal methods to verify critical software • [Perception] Tried it (20 years ago in 1998) too slow ! • [Reality] More practical in 2018-17 due to powerful solvers being developed? • You need to write papers with clarity to avoid such perceptions. ISSTA 2019 Doctoral Symposium
- 27. Perception • Avoid negative perception of community • This is easily achievable • Claiming away your technical contributions. • Mentioning your contributions as more of an implementation than ideas. • Need to write the paper to cater to, and prevent such reviews. ISSTA 2019 Doctoral Symposium
- 28. Perception • Avoid negative perception of community • This must have been done before. • Usually papers not turned down because of not mentioning related work. • One may compare your paper at a high level with another, and claim similarity. • This is where you need to be very clear yourself about the new concepts and ideas in your paper. ISSTA 2019 Doctoral Symposium
- 29. Takeaways • Choose an area • Choose a topic of interest • Why would it interest others • Possible translational value you can claim • Extrapolate to the time you finish PhD • Evaluate the contributions – be clear yourself • Evaluate the timing of your work – is the time right? • Structure the paper to address common negative perceptions, AND ISSTA 2019 Doctoral Symposium
- 30. Takeaways • Contribution and advance of a paper is dependent on so many factors. • Implementation, experiments and experimental results are important • But they are far from being the only factor. • Footnote: What I tell all my students • THINK hard before we start working hard IMPLEMENTING! ISSTA 2019 Doctoral Symposium