An Architect's View

CFML, Clojure, Software Design, Frameworks and more...

An Architect's View

Scala Days 2011 - Fascinating but...

June 4, 2011 ·

It's been a long time since I've been to a conference where serious research papers were presented. Pretty much every conference I've been to in the last 10-15 years has focused on practical use cases even for very bleeding edge topics. Scala Days had a mixture of practical and research but, at least based on the talks I chose, seemed very heavy on research - in several talks I felt like I was back at university in my final year computer science classes. It was all very interesting and thought-provoking / stimulating but I didn't come away fired up with new ideas I could apply at work. I did come away with a list of research papers to read tho'...

It is always good to be exposed to - and to talk to - very smart people, solving very hard problems. Martin Odersky's keynote covered how Scala has grown and what challenges lie ahead, with a heavy focus on parallel processing - showing off the easy to use new parallel collections in Scala 2.9 - and talked about the research work being done on distributed collections. Doug Lea's keynote also focused on parallel processing, covering many different flavors from hardware thru implicit software to explicit software and his theme was that parallelism can no longer be hidden from developers: we must all embrace this and be able to control parallelism in our applications. Doug was entertaining, insightful and managed to paint several groups of people as "delusional" - awesome! Later on, I got to ask him for his thoughts on Clojure and he was very complimentary about Rich Hickey and what he has achieved with Clojure, while cautioning that the approach would not work for everyone (fine by me: I worry about languages that get too popular being dragged down by the inertia of conservatism that is forced upon any widely used technology).

Three talks I enjoyed on the more practical side: Scala Domain Modeling and Architecture (slides) by Hossam Karim; How Scala Experience Improved Our Java Development (PDF) by Sam Reid; Exploring Light-weight Event Sourcing (example code on github) by Erik Rozendaal. The latter is particularly applicable to something I'm working on right now so it was very timely (and it confirmed the approach I want to take - thank you to Peter Bell for initially sowing the seed on that!).

On the more esoteric side: Jon Pretty's talk on inhibiting exceptions was interesting from a "wouldn't it be nice if we could do this?" point of view and Luc Duponcheel's talk about implementing Philip Wadler's Arrow Calculus was a nice exposure to the more mathematical side of functional programming. I learned a couple of interesting things about both Scala and Ruby in Rémy-Christophe Schermesser's "Scala from a Rubyist Point of View".

On the more "WTF" side of things: Alberto Souza's "Unit Testing the Implicit Methods that Uses Infrastructure Code", Heiko Seeberger's "The Ease of Scalaz" and Sadek Drobi's "Anorm: Plain Old SQL, Using Scala Collections, Pattern Matching, and Parsers to Simplify an Unnecessarily Over-complexified Task" all highlighted - for me - how the power of Scala is also it's Achilles' Heel in terms of approachability and adoption. Sadek's talk reflected the opacity of its title where he showed how using Parser Combinators was the "simple" solution to getting column data back from SQL queries in a type-safe manner. Read that again. Now tell me how many problems you have with that premise. Heiko's talk on Scalaz - "Type Classes and Pure Functional Data Structures for Scala" - really brought home how a strong type system creates problems for which the solutions are... more types... and stuff like Monads, Applicative Functors, Covariant Binary Functors and so on. Alberto's talk showed how implicit type conversions can make code much harder to test and how to get around that (or, put another way, the hoops you have to jump thru to mock services that are added thru implicit type conversions). Given that many elegant solutions in Scala seem to rely heavily on implicit type conversions - the so-called "pimp my library" idiom - his talk just added fuel to the fire regarding Scala's perceived complexity. There was nothing wrong with any of these talks in an absolute sense. I actually enjoyed following along thru the twists and turns of the type system and application of higher-order functions. The problem is that this sort of stuff makes Scala very intimidating for a lot of developers...

...and the closing panel discussion focused on that by asking "Scala in the Enterprise: What Will It Take?" I spoke to a number of attendees who were either completely or somewhat new to Scala and almost without fail, they'd felt lost and overwhelmed in nearly every talk they'd attended. Several panel members felt that all Scala needs is improved tooling and some "best practice" documentation to bring over the leading 10% of "Enterprise" developers and that the rest will follow. There was a sense from several audience members that (some of) the Scala "Thought Leaders" feel that most Java developers are retards and even panelist Dick Wall said that when beginners ask questions, the answer from the Scala community must not be something like "Oh, don't do it like that - use a Monad!" because that's intimidating and unhelpful.

Scala is an extremely impressive language. It is a great blend of practicality, incredible expressiveness and solid type safety. The libraries offer clean, type safe ways to manipulate all sorts of data collections with efficient, concise code, both sequentially and in parallel (as of Scala 2.9) and promise the same for distributed processing in the future. It's a huge improvement over Java in so many ways. But that doesn't mean it should become a mainstream tool, used by "everyman" in the Java programming world. I think there's room for successful niche languages and I think that the mainstream is already well-served by popular languages that are a good fit for a large community of developers of varying abilities. I think the folks behind Scala need to figure out how successful it really needs to be as opposed to what it would take to make it go mainstream.

Overall, I found Scala Days fascinating but... at the end of two days I felt more comfortable about my increasing use of Clojure, which has many of the benefits that Scala promotes (functional approach, immutable state, improved support for concurrency, expressiveness) without a lot of the machinery that seems to demand complexity behind the scenes to address type safety and support the expressiveness that is possible.

Tags: scala

4 responses

  • 1 Peter Bell // Jun 5, 2011 at 11:33 AM

    Interesting summary. Firstly, thanks for the links to the Event Sourcing presentation. Even though I've been speaking about this for a while, my new gig is the first time I'm going to get to put this into production, so I'll check out the sample code.

    Your take away from the conference was interesting. I was supposed to attend a Scala conference in New York, but something came up. It sounds like on balance I might not put it at the top of my "to get better at" list - especially now that I have node for some of my functional needs (even if I'm not a huge fan of Javascript as a language).
  • 2 John Nilsson // Jun 11, 2011 at 7:12 AM

    I was thinking about your comment about thinking that Java developers are retards. Is this sentiment really coming from the Scala community?

    I've read some mailing list discussions where similar sentiments has come up. But usually they are from the other side of the fence. Some concerned potential adopter asks "Can my team members really handle this", where the community response might be "If you think your team members are retards, maybe you shouldn't employ them".

    So it seems to me that if there's concern about bad developers not being able to handle Scala, it's coming from outside the community.
  • 3 Sean Corfield // Jun 11, 2011 at 9:12 PM

    @John, I haven't personally encountered that sentiment - it always seems to come up in terms of "some people think..." which is a bit of a strawman, at best.

    I think a lot of organizations are honest enough to know they hire a lot of mediocre people because they just need bodies churning out code. It's why outsourcing has been so "successful". After all, half of all programmers are "below average" by definition!

    I think quite a few people in the Scala community believe they are "above average" and that "below average" programmers couldn't do what they do. I think this is true of early adopters in any new technology that is seen as "advanced", pretty much by definition.

    I don't work in the Java community. I haven't used Java seriously for years - I just use the libraries from other languages on the JVM :) I certainly know a lot of programmers who would struggle with Scala (I think some of them would struggle with Java too). But I also know several who would be perfectly happy with Scala.

    So, overall, I think the attitude exists both within and outside the Scala community. Make sense?
  • 4 notyy // Sep 21, 2011 at 7:12 AM

    I have 10years experience on java,some experience on haskell, and am trying to learn scala. I feel that many java developers need to learn functional programming concept first, in a more pure functional programming language such as haskell. then move to scala for realworld usage on jvm platform.

    learning scala for me is not easy,especially when I see many different ways to solve a problem. are there "best practice" in scala? I am not confident on my code,as I don't know whether it's the best code to solve the problem, even when it do work.

    in the end o my comment, may I mention to my experiment project on code review is warmly welcome.