Download as PDF, PPTX
![Regression, in 2D
How: Polynomials, splines, neural networks, support vector
machines, Gaussian process models, boosted trees, … [many refs]](https://image.slidesharecdn.com/20180226asurveyofhierdesignmethodologies-180328101051/75/Top-Down-Bottom-Up-A-Survey-of-Hierarchical-Design-Methodologies-5-2048.jpg)
![Symbolic Regression (SR)
(Like regression, but output a symbolic model too)
Y
X
Symbolic model:
y=50.2 + 9.1 • x
+ 3.2 • max(0, x2)
Y
X
[e.g. McConaghy 2005;
McConaghy 2011]](https://image.slidesharecdn.com/20180226asurveyofhierdesignmethodologies-180328101051/75/Top-Down-Bottom-Up-A-Survey-of-Hierarchical-Design-Methodologies-6-2048.jpg)
![SR with Vanilla GP – Koza 1991
[Koza1992]](https://image.slidesharecdn.com/20180226asurveyofhierdesignmethodologies-180328101051/75/Top-Down-Bottom-Up-A-Survey-of-Hierarchical-Design-Methodologies-17-2048.jpg)
Uploaded byTrent McConaghy
Top-Down? Bottom Up? A Survey of Hierarchical Design Methodologies
The document discusses various hierarchical design methodologies within artificial intelligence, focusing on both top-down and bottom-up approaches. It highlights the importance of performance estimation, optimization, and feasibility modeling in complex design systems. The author emphasizes the potential for evolution in design processes and the necessity for structured methodologies to handle increasingly complicated designs.
More Related Content
Similar to Top-Down? Bottom Up? A Survey of Hierarchical Design Methodologies
![Blockchains for AI [With New Applications]](https://cdn.slidesharecdn.com/ss_thumbnails/20170522blockchainsforai-consensus2017-170523111907-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Energy/abundance edition] Nature 2.0: The Cradle of Civilization Gets an Upg...](https://cdn.slidesharecdn.com/ss_thumbnails/20180416setfestival-nature2-180417115903-thumbnail.jpg?width=640&height=640&fit=bounds)
Top-Down? Bottom Up? A Survey of Hierarchical Design Methodologies
- 1. Top-Down? Bottom Up? ASurvey of Hierarchical Design Methodologies Trent McConaghy @trentmc0
- 2. Pragmatic Ways toView AI, by Example
- 3. Classification, in 2D Salary Age Salary Age Didn’tpay Paid bills Credit profile:
- 4.
- 5. Regression, in 2D How:Polynomials, splines, neural networks, support vector machines, Gaussian process models, boosted trees, … [many refs]
- 6. Symbolic Regression (SR) (Likeregression, but output a symbolic model too) Y X Symbolic model: y=50.2 + 9.1 • x + 3.2 • max(0, x2) Y X [e.g. McConaghy 2005; McConaghy 2011]
- 7. AI Has aToolbox of Ways to Solve… •Classification – Fraud detection, spam filtering … •Regression – Stock prediction, sensitivity analysis … •Whitebox regression – Scientific discovery … •Optimization – Airfoil design, circuit simulation … •Structural synthesis – Analog synthesis, robotics … •Pattern recognition – Face recognition, object recog … •System identification – Scientific discovery … •Ranking – Web search, ad serving, social discovery … •Control – Auto-driving autos, spacecraft trajectories … •…
- 8. AI Sub-fields • machinelearning • neural networks • evolutionary computation • fuzzy logic • data mining • artificial general intelligence • pattern recognition • .. • (nee) nonlinear programming • (nee) databases • ..
- 9. AI Sub-fields ofsub fields • machine learning + neural networks – recurrent neural networks – sparse linear regression – deep learning – .. • evolutionary computation – evolutionary programming, evolution strategies – genetic algorithms – genetic programming • ..
- 10. Genetic Programming (GP): Abranch of a branch of AI But a super-cool one..
- 11.
- 13. Hornby & Lohn2005
- 15. GP for SR “Afunction is a tree” f(x) = 4.8*x3 + √x2 4.8 √* x2 + x3 Searches through the space of trees: 1. Initial random population; evaluate 2. Create children from parents via operators; evaluate 3. Select best; goto 2
- 16. GP for SR:Crossover Operator
- 17. SR with VanillaGP – Koza 1991 [Koza1992]
- 18. SR With FancierGP – McConaghy 2005
- 19. Circuit synthesis withGP – Koza 2001
- 20. Circuit synthesis withGP – McConaghy 2005
- 21. Synthesis of jewelry– Hornby 2011
- 22.
- 23.
- 24.
- 25. How do youevolve a 5K-part car?
- 26. How do youevolve a 10G-part chip? Ref. 1st Preamp 2nd Preamp Comp. NAND ROM b0 b5 b6 b7 Latch vin_minvin_plus clk GRAYCODER S/H
- 27. How do youevolve a 37T cell animal?
- 28. • Simple • What99% of optimization does • But, doesn’t scale • Yes, massive compute helps a lot. Eg deep learning • But even for that: what about a 10x larger net? 1000x? Approach the Design “Flat”?
- 29. Top-Down Design? How,Specifically?
- 30. Hierarchy! Hierarchy in aCircuit
- 31. Bottom-up Design? How,Specifically?
- 32. Hierarchy! Bringing Methodto the Madness system SB 1.1 SB 1.2 SB 1.m SB 2.1 subblock 1 subblock 2 subblock n • Divide, conquer, stitch it back together • Difficulty(design) ≈ Difficulty(hardest block)
- 33. Hierarchical Design Methodologies system SB1.1 SB 1.2 SB 1.m SB 2.1 subblock 1 subblock 2 subblock n • Target behavior of each sub-block is well defined • Design involves • A way to estimate performance of a candidate block • A way to search each sub-block • A way to stitch together the blocks • Many possible ways!
- 34. A key requirement:we need a way to estimate performance of each sub-block • Input: design parameters (e.g. device sizes) • Output: performance metrics (e.g. power, …) • Higher-level blocks are lower fidelity (to simulate faster)
- 35. flat top-down constraint-driven design spaceorganization & traversal
- 36.
- 37. Top-Down Constraint-Driven Design SystemSystemdesign Subblock 1 Subblock 2 Subblock 1 Subblock 2 Subblock n constraints con.con.con. • Step 1: map specifications down the hierarchy (via search)
- 38. Top-Down Constraint-Driven Design System SB1.1 SB 1.2 SB 1.m SB 2.1 System design Subblock 1 Subblock 2 Subblock 1 Design Subblock 2 Design Subblock n Design SB 1.1 SB 1.2 SB 1.m SB 2.1 constraints con.con.con. con. con. con. con.
- 39. Top-Down Constraint-Driven Design System SB1.1 SB 1.2 SB 1.m SB 2.1 System design Subblock 1 Subblock 2 Subblock 1 Design Subblock 2 Design Subblock n Design SB 1.1 SB 1.2 SB 1.m SB 2.1 constraints con.con.con. con. con. con. con.
- 40. Top-Down Constraint-Driven Design System SB1.1 SB 1.2 SB 1.m SB 2.1 System design Subblock 1 Subblock 2 Subblock 1 Design Subblock 2 Design Subblock n Design SB 1.1 SB 1.2 SB 1.m SB 2.1 Verify Verify Verify • bottom-up verification to verify performance against specifications
- 41. Top-Down Constraint-Driven Design System SB1.1 SB 1.2 SB 1.m SB 2.1 System design Subblock 1 Subblock 2 Subblock 1 Design Subblock 2 Design Subblock n Design SB 1.1 SB 1.2 SB 1.m SB 2.1 constraints con.con.con. con. con. con. con. • What happens when constraints cannot be met? • General heuristic: backtrack
- 42. flat top-down constraint-driven manual or semi-auto feasibility modeling designspace organization & traversal
- 43. flat top-down constraint-driven manual or semi-auto feasibility modeling automatic bottom-up feasibility modeling designspace organization & traversal
- 44. Feasibility modeling bottom-up •Model feasible performance space of a design • For entire range of design parameters • E.g. for a car: “can have 200mph top speed and 20 mpg separately but not together”. Represent this continuously.
- 45. Top-down constraint-driven design •Pros: • Scale! (Compared to flat) • Feasible runtime • Can re-use feasibility models of building blocks • “Natural choice” for advocates of top-down design • Cons: • Need a way to know what performance combinations of sub-blocks are feasible • Once feasibility modeling is done, still need to do top-down steps (i.e. optimization at each node in hierarchy)
- 46. flat concurrenttop-down constraint-driven design spaceorganization & traversal
- 47. Concurrent hierarchical methodology System SB1.1 SB 1.2 SB 1.m SB 2.1 Subblock 1 Subblock 2 Subblock n optimize all at once via glue constraints • Pros: an alternative to “flat” • Cons: scales badly; more complex than “flat”
- 48. flat concurrenttop-down constraint-driven bottom-up multi- objective design spaceorganization & traversal
- 49. Multi-objective bottom-up design(MOBU) system SB 1.1 SB 1.2 SB 1.m SB 2.1 subblock 1 subblock 2 subblock n • Pareto front of a lower level block is “dimensions” in the design space of each layer that uses it • Effectively “compresses” lower level design spaces into the meaningful part (i.e. into the tradeoffs) • Ref Eeckelaert et al 2005
- 50.
- 51.
- 52. Multi-objective bottom-up design(MOBU) • Pros: • Feasible runtime • Provides system level tradeoffs • Can re-use multi-objective tradeoffs of building blocks • “Model” of tradeoffs is simply the discrete designs themselves • Once once top-level tradeoffs are known, the designs already exist too. No need for a subsequent top-down sizing • Cons: • No full model of feasibility, just of tradeoffs • Cannot refine designs precisely, the way that top-down does
- 53. How do you evolvea 37T cell animal?
- 54. flat concurrenttop-down constraint-driven bottom-up multi- objective design spaceorganization & traversal bottom-up ossification
- 62. Bottom-up Ossification system SB 1.1SB 1.2 SB 1.m SB 2.1 subblock 1 subblock 2 subblock n • Two levels at a time are evolving • Then lower level’s mutation rate →0 (ossifies), and a new upper level emerges
- 63. Bottom-up Ossification system SB 1.1SB 1.2 SB 1.m SB 2.1 subblock 1 subblock 2 subblock n • Two levels at a time are evolving • Then lower level’s mutation rate →0 (ossifies), and a new upper level emerges
- 64. Conclusion • Hierarchical designmethodologies enable ridiculously complicated designs • In machine learning • And in nature! • They are structured, not ad-hoc • Top-down constraint-driven • Multi-objective bottom up • And more • Are you ready for 1000x larger nets? Trent McConaghy @trentmc0
- 65. Survey / furtherreading • Georges Gielen, Trent McConaghy, and Tom Eeckelaert, “Performance Space Modeling for Hierarchical Synthesis of Analog Integrated Circuits”, Proc. Design Automation Conference, 2005. http://trent.st/content/2005_DAC_hierarchy.pdf • For further references, see references of that work