Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Scala Symposium 2014, ThoughtWorks
4 likes5,189 views
The document discusses the experience of a software engineering team at Springer after three years of using Scala for their content delivery platform. It highlights both positive aspects, such as increased productivity and the functional programming features of Scala, and negative aspects, including tool support issues and complexity. The future outlook suggests no immediate plans to move away from Scala, though Java 8 is considered an alternative as the company may explore polyglot programming with smaller teams and applications.
![Complexity
• There’s still a lot of code in our codebase that is hard to read
• It seems to be very easy to shoot yourself in the foot with Scala
• Scala *is* complex (and that’s why scalac will never be as fast as javac)
Invariant/covariant/contravariant types (T, +T and -T)
Refined types (new Foo {...})
Structural types (x: {def y: Int})
Path dependant types (a.B)
Specialized types (@specialized)
Self types (this =>)
Projection types (A#B)
Existential types (M[_])
Type bounds (<:, >:)
Type constraints (=:=, <:< and <%<)
Type members (type T)
Type aliases (type T = Int)
Type classes ( (implicit ...) )
View bounds (<%)
Higher kinded types (* => *)
F-Bounded type polymorphism (M[T <: M[T]])
http://nurkiewicz.github.io/talks/2014/scalar/#/16](https://image.slidesharecdn.com/scala-in-practice-3-years-140420201055-phpapp01/85/Scala-in-practice-3-years-by-Patric-Fornasier-Springr-presented-at-Pune-Scala-Symposium-2014-ThoughtWorks-28-320.jpg)
![Surprises
List(1, 2, 3).toSet
scala.collection.immutable.Set[Int] = Set(1, 2, 3)
http://dan.bodar.com/2013/12/04/wat-scala/](https://image.slidesharecdn.com/scala-in-practice-3-years-140420201055-phpapp01/85/Scala-in-practice-3-years-by-Patric-Fornasier-Springr-presented-at-Pune-Scala-Symposium-2014-ThoughtWorks-36-320.jpg)
![Surprises
List(1, 2, 3).toSet
scala.collection.immutable.Set[Int] = Set(1, 2, 3)
List(1, 2, 3).toSet()
http://dan.bodar.com/2013/12/04/wat-scala/](https://image.slidesharecdn.com/scala-in-practice-3-years-140420201055-phpapp01/85/Scala-in-practice-3-years-by-Patric-Fornasier-Springr-presented-at-Pune-Scala-Symposium-2014-ThoughtWorks-37-320.jpg)
![Surprises
List(1, 2, 3).toSet
scala.collection.immutable.Set[Int] = Set(1, 2, 3)
List(1, 2, 3).toSet()
Boolean = false
http://dan.bodar.com/2013/12/04/wat-scala/](https://image.slidesharecdn.com/scala-in-practice-3-years-140420201055-phpapp01/85/Scala-in-practice-3-years-by-Patric-Fornasier-Springr-presented-at-Pune-Scala-Symposium-2014-ThoughtWorks-38-320.jpg)
More Related Content
!["Year of the Selfie" [INFOGRAPHIC]](https://cdn.slidesharecdn.com/ss_thumbnails/selfieinfographic-140212032727-phpapp01-thumbnail.jpg?width=560&fit=bounds)
Similar to Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Scala Symposium 2014, ThoughtWorks (20)
Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Scala Symposium 2014, ThoughtWorks
- 2. Context
- 3. Context Pat Springer Software Engineer Technical Principal 10+ years Academic Publisher Pretty Large Springer Link Strategic 95 % Online Revenue 66 % Total Revenue 9 Mio items 2TB XML Re-built 3 years ago Content Delivery Platform 52 Mio PageViews / Month 99.9% Availability Scala NoSql Co-Sourced Team TW Ex-TW Springer Global CD
- 4. SpringerLink
- 5. Timeline Start development Cycling Trip Re-joined Product -> platform Paying back tech debt Today 04/11 04/12 08/13 04/14 Inception 05/12 09/12 Bookfair Release RD Live User migration, enhancements Link Live Smart Books Live Enhancements
- 6. Looking back: why scala? • Increase productivity • Be more attractive employer • Team decision
- 7. Looking back: good vs bad Good • Functional programming • Terse syntax • JVM ecosystem • Gentle learning curve • DSL friendly syntax • Motivated team Bad • Tool support • Compilation times • Language complexity #moreRope
- 8. Fast-forward
- 9. Fast-forward (Aug 2013) • 2.5 years into project • 1.5 years of weekly live releases • 100k LOC • >10k commits • >90 committers
- 10. Fast-forward (Aug 2013) • 2.5 years into project • 1.5 years of weekly live releases • 100k LOC • >10k commits • >90 committers not all related to Scala - to be fair • Poor feedback loops • Lots of accidental complexity
- 11. Trend (2 years) LOC ~ 100k
- 12. Trend (2 years) build time LOC
- 13. Trend (2 years) build time LOC • 1:34 min src/main • 6:44 min src/test • 8:18 min total
- 14. What did we do?
- 15. What did we do • Reduced build time • Improved feedback loops • Reduced accidental complexity
- 16. Build time • Reduced size of codebase (broke off vertical slices, pulled out APIs, pulled out libraries, removed unused features, removed low-value tests, etc.) • Reduced usage of certain language features (esp. traits and implicits)
- 18. Trend (Dec 2013) build time # traits LOC unable to compile on 13” macbook
- 19. Trend (Dec 2013) build time # traits LOC unable to compile on 13” macbook
- 20. The problem with traits • Will re-compile on every class the trait is mixed in • Slows down dev-build cycle • Will result in byte code bloat • Will compile *a lot* slower ! For faster compile times: • Use pure traits • Use old-school composition for code re-use • Use pure functions via imports (e.g. import Foo._) • If unavoidable, use inheritance for code re-use
- 21. Build time
- 22. Build time • 1:34 min src/main • 6:44 min src/test • 8:18 min total
- 23. Build time • 1:34 min src/main • 6:44 min src/test • 8:18 min total • 0:24 min src/main • 3:11 min src/test • 3:35 min total
- 24. Build time (on CI server) • Incremental compilation on CI • Only one dedicated CI agent • Physical build servers • CPUs with higher clock speed
- 25. Build time (on CI server) • Incremental compilation on CI • Only one dedicated CI agent • Physical build servers • CPUs with higher clock speed
- 26. Complexity
- 27. Complexity • There’s still a lot of code in our codebase that is hard to read • It seems to be very easy to shoot yourself in the foot with Scala • Scala *is* complex (and that’s why scalac will never be as fast as javac)
- 28. Complexity • There’s still a lot of code in our codebase that is hard to read • It seems to be very easy to shoot yourself in the foot with Scala • Scala *is* complex (and that’s why scalac will never be as fast as javac) Invariant/covariant/contravariant types (T, +T and -T) Refined types (new Foo {...}) Structural types (x: {def y: Int}) Path dependant types (a.B) Specialized types (@specialized) Self types (this =>) Projection types (A#B) Existential types (M[_]) Type bounds (<:, >:) Type constraints (=:=, <:< and <%<) Type members (type T) Type aliases (type T = Int) Type classes ( (implicit ...) ) View bounds (<%) Higher kinded types (* => *) F-Bounded type polymorphism (M[T <: M[T]]) http://nurkiewicz.github.io/talks/2014/scalar/#/16
- 29. Not opinionated For example: http://twitter.github.io/effectivescala/ • Many ways to do the same thing • Coding conventions help, but only so much
- 30. Not opinionated def foo() = "foo" def bar = "bar" ! foo foo() bar bar() // won't compile For example: http://twitter.github.io/effectivescala/ • Many ways to do the same thing • Coding conventions help, but only so much
- 31. Not opinionated def foo() = "foo" def bar = "bar" ! foo foo() bar bar() // won't compile For example: http://twitter.github.io/effectivescala/ • Many ways to do the same thing • Coding conventions help, but only so much def baz(x: String) = x “x”.charAt(0) “x” charAt(0) // won't compile “x”.charAt 0 // won't compile “x” charAt 0 baz("x") baz “x" // won't compile
- 32. Not opinionated def foo() = "foo" def bar = "bar" ! foo foo() bar bar() // won't compile list.foreach { x => println(x) } list.foreach ( x => println(x) ) list.foreach { println(_) } list.foreach ( println(_) ) list foreach { x => println(x) } list foreach ( x => println(x) ) list foreach { println(_) } list foreach ( println(_) ) For example: http://twitter.github.io/effectivescala/ • Many ways to do the same thing • Coding conventions help, but only so much def baz(x: String) = x “x”.charAt(0) “x” charAt(0) // won't compile “x”.charAt 0 // won't compile “x” charAt 0 baz("x") baz “x" // won't compile
- 33. Not opinionated def foo() = "foo" def bar = "bar" ! foo foo() bar bar() // won't compile list.foreach { x => println(x) } list.foreach ( x => println(x) ) list.foreach { println(_) } list.foreach ( println(_) ) list foreach { x => println(x) } list foreach ( x => println(x) ) list foreach { println(_) } list foreach ( println(_) ) if (foo) "x" else "y" foo match { case true => "x" case _ => "y" } For example: http://twitter.github.io/effectivescala/ • Many ways to do the same thing • Coding conventions help, but only so much def baz(x: String) = x “x”.charAt(0) “x” charAt(0) // won't compile “x”.charAt 0 // won't compile “x” charAt 0 baz("x") baz “x" // won't compile
- 36. Surprises List(1, 2, 3).toSet scala.collection.immutable.Set[Int] = Set(1, 2, 3) http://dan.bodar.com/2013/12/04/wat-scala/
- 37. Surprises List(1, 2, 3).toSet scala.collection.immutable.Set[Int] = Set(1, 2, 3) List(1, 2, 3).toSet() http://dan.bodar.com/2013/12/04/wat-scala/
- 38. Surprises List(1, 2, 3).toSet scala.collection.immutable.Set[Int] = Set(1, 2, 3) List(1, 2, 3).toSet() Boolean = false http://dan.bodar.com/2013/12/04/wat-scala/
- 39. Implicits • Can make it very hard to read code • Tool support is very bad • Impacts compilation time • Surprising behaviour (esp. when used with overloaded methods or optional params)
- 40. Tooling def handle(response: HttpResponse, request: HttpRequest) • Tool support is still very basic • Makes it hard to continuously refactor (which means people are less likely to do it)
- 41. Tooling def handle(response: HttpResponse, request: HttpRequest) • Tool support is still very basic • Makes it hard to continuously refactor (which means people are less likely to do it) no luck with “change signature” refactoring support
- 42. Trait entanglements • Makes it difficult to reason about behaviour
- 43. Trait entanglements • Makes it difficult to reason about behaviour trait A { def foo = "a" }
- 44. Trait entanglements • Makes it difficult to reason about behaviour trait A { def foo = "a" } trait B extends A { override def foo = "b" }
- 45. Trait entanglements • Makes it difficult to reason about behaviour trait A { def foo = "a" } trait B extends A { override def foo = "b" } class C extends A with B new C().foo
- 46. Trait entanglements • Makes it difficult to reason about behaviour trait A { def foo = "a" } trait B extends A { override def foo = "b" } class C extends A with B new C().foo "b"
- 47. Trait entanglements • Makes it difficult to reason about behaviour trait A { def foo = "a" } trait B extends A { override def foo = "b" } class C extends A with B new C().foo "b" class D extends B with A new D().foo
- 48. Trait entanglements • Makes it difficult to reason about behaviour trait A { def foo = "a" } trait B extends A { override def foo = "b" } class C extends A with B new C().foo "b" class D extends B with A new D().foo "b"
- 49. Trait entanglements (2) ArticlePageSteps WebDriverSupport AuthorStepsCoverImageSteps SummarySection Waiter SectionPageSteps CommonPageSteps WebElementSupport Assertions Uris TripleEquals OnHost TripleEqualsSupport MachineNames
- 50. Trait entanglements (3) ArticlePageSteps_0 WebDriverSupport_1 AuthorSteps_1 CoverImageSteps_1 SummarySection_1Waiter_1 SectionPageSteps_1 CommonPageSteps_1 CommonPageSteps_2 WebElementSupport_2Assertions_2 WebDriverSupport_2Uris_2 WebElementSupport_3 WebDriverSupport_3Uris_3 TripleEquals_3 OnHost_3 OnHost_4 TripleEqualsSupport_4 MachineNames_4 MachineNames_5
- 51. Trait entanglements (3) ArticlePageSteps_0 WebDriverSupport_1 AuthorSteps_1 CoverImageSteps_1 SummarySection_1Waiter_1 SectionPageSteps_1 CommonPageSteps_1 CommonPageSteps_2 WebElementSupport_2Assertions_2 WebDriverSupport_2Uris_2 WebElementSupport_3 WebDriverSupport_3Uris_3 TripleEquals_3 OnHost_3 OnHost_4 TripleEqualsSupport_4 MachineNames_4 MachineNames_5
- 52. Trait entanglements (4) ArticlePageTests_0 ArticlePageSteps_1 AboutSectionSteps_1 SearchResultsPageSteps_1 Uris_1 ArticleTestFixture_1 JavascriptSupport_1 IssuePageSteps_1 CommonAbstractSteps_1 GoogleAnalyticsSteps_1 FakeEntitlementSteps_1 ExportCitationPageSteps_1 FullTextPageSteps_1 OtherActionsSectionSteps_1 WebDriverSupport_2 AuthorSteps_2 CoverImageSteps_2 SummarySection_2Waiter_2 SectionPageSteps_2 CommonPageSteps_2 CommonPageSteps_3 WebElementSupport_3Assertions_3 WebDriverSupport_3Uris_3 WebElementSupport_4 WebDriverSupport_4Uris_4 TripleEquals_4 OnHost_4 OnHost_5 TripleEqualsSupport_5 MachineNames_5 MachineNames_6
- 53. Trait entanglements (4) ArticlePageTests_0 ArticlePageSteps_1 AboutSectionSteps_1 SearchResultsPageSteps_1 Uris_1 ArticleTestFixture_1 JavascriptSupport_1 IssuePageSteps_1 CommonAbstractSteps_1 GoogleAnalyticsSteps_1 FakeEntitlementSteps_1 ExportCitationPageSteps_1 FullTextPageSteps_1 OtherActionsSectionSteps_1 WebDriverSupport_2 AuthorSteps_2 CoverImageSteps_2 SummarySection_2Waiter_2 SectionPageSteps_2 CommonPageSteps_2 CommonPageSteps_3 WebElementSupport_3Assertions_3 WebDriverSupport_3Uris_3 WebElementSupport_4 WebDriverSupport_4Uris_4 TripleEquals_4 OnHost_4 OnHost_5 TripleEqualsSupport_5 MachineNames_5 MachineNames_6 Imagine many more circle here
- 54. So, what’s next?
- 55. Today • We’ve delivered successfully using Scala • Don’t think we’re more productive (pure gut feeling, though) • We try to stick to the good parts (conventions, functional programming, pattern matching, etc.) • Complexity, slow compilation and lack of tool support are real problems
- 56. The future • No urgency to move away from Scala or re-write existing systems • Java 8 is an alternative • Smaller teams and apps will probably lead to more polyglotism (and less Scala)