Reflecting, celebrating, and looking forward

As I write, the Perl 6 Christmas release is taking place. It goes without saying that it’s been a long journey to this point. I’ve walked less than half of it, joining the effort 7-8 years ago through. Back then, I was fresh from university, had enjoyed the courses on compiler construction, type systems, and formal semantics, and was looking for an open source project in one or more of these fields to contribute to.

For many years before I got involved with Perl 6, I’d been using Perl extensively for web application development. It helped me live more comfortably through my college and university years, and I had sufficient work coming in that I kept a few other part-timers in a job too. That would be reason enough for being fond of a language, but there was more. Perl is really the language where I “grew up” as a programmer. It’s the first language where I used regexes and closures, it helped develop my early understanding of OOP, and it introduced me to testing culture. It’s the first language where I went to a conference, and realized the value a language’s community can have. All of this has been foundational to what I’ve done in my career since then, which besides Perl 5/6 has seen me deliver code and training in a wide range of languages, including C, C#, Java, and Python.

Through using Perl I got to know about the Perl 6 project – and it seemed a good fit with my interest in languages, compilers, and types. I knew it was a long-running project with plenty of history, but looking at what it was trying to achieve, I was convinced it was worth trying to help out a little to make it happen. It’s surprising just how successful Perl 6 has been over the years at attracting really great people to work on it – knowing full well its long, and at times difficult, history.

From patcher to architect

As usual with anyone new to an open source project, my first contributions were small. Portability things, small fixes here and there, minor features, and the like. Over time, I found myself taking on increasingly large language features. By 2009 I was a regular contributor, and had a decent grasp of much of the Rakudo compiler code base. I caused Patrick Michaud, the lead developer of Rakudo at the time, a good number of “oh no!” moments, as I didn’t yet have a great picture of the overall design – but was putting in notable features anyway. He, and others, did a great job of steering me gently in the right kind of direction.

2010 probably goes down as the most difficult year of my involvement in the Perl 6 project. The first Rakudo Star, a “useful, usable, early adopter’s release of Perl 6”, was generally graded as “not good enough”. More problematic, as I took some steps back and reflected on how to rectify this, was that the issues went right to the very heart of the architecture of Rakudo itself. Rakudo in those days was a traditional compiler: you fed it code in Perl 6, it spat out code in an intermediate language, which was executed on the Parrot VM. Of course, this is what you learn a compiler does at university, and it’s all very clean and simple…and not at all what Perl 6 demands.

Compile time and runtime are not so cleanly distinct concepts in Perl. They never have been, thanks to things like BEGIN blocks. But in Perl 6, we really wanted to both handle BEGIN time – and have lots of meta-programming stuff going on at BEGIN time – and still be able to have separate compilation. This, it turns out, is a tricky problem. And, as I looked into how to solve it, it became clear that it was going require a deep, drawn out, overhaul. Further, it became clear that this was an effort I was going to need to play architect on and largely lead. History has shown it to be the right call; the architecture put in place then has largely survived intact to today’s release, and it enabled a lot of the great things we simply take for granted today. But at the time, it was a lonely path to walk.

In the years since then, I led the way on getting Rakudo ported to the JVM – and have been happy to see that work taken forward by others. I was also a founder of MoarVM, the VM that the Perl 6 Christmas release primarily targets. Most recently, I took up the long-neglected task of getting Perl 6’s concurrency design into decent shape, doing the initial implementations of the features. It goes without saying that none of this would have been possible without the incredible bunch of people in the Perl 6 community who not only contributed their great technical skills to these efforts, but also a good deal of encouragement and friendship along the way.

For 7 years I made a really great job of just being “this guy who hacks on stuff” while managing to disclaim wearing any particular hat – until Larry went and called me out as architect at the Swiss Perl Workshop this last summer. It’s a role I’m proud to hold for the moment, and I look forward to continuing to contribute to the Perl 6 project for some years to come.

So, about the release…

By this point into writing the post – the Christmas release of Perl 6 has already taken place! Hurrah! But…what does it mean?

First, it’s important to understand that we’ve actually released two things (and that there are a couple more to come in the next days).

The first of these is the specification of the Perl 6 language itself, which is expressed as a suite of over 120,000 tests. Versions of the Perl 6 language will be named after festivals or celebrations; our alpha was Advent, our beta was Birthday, and we’re now at Christmas. This is referred to as “Perl 6 Christmas”, or in short as Perl 6.c. The next major language release will most likely be called Diwali, though I’m not sure we’ve worked out how to spell it yet. :-)

The second is a release of the Rakudo Perl 6 compiler that complies with this specification. We don’t imagine we’ll manage to stop people blurring language specification and language implementation, and know full well that when most people say they want “Perl 6 Christmas” they actually want a compiler that implements that version of the language. All the same, it’s a valuable distinction, as it means we remain open to alternative implementations – something that may not be that important now, but may be in a decade or two.

In the coming days, we’ll also produce a Rakudo Star release – which consists of the compiler along with documentation and a selection of modules – and that will also have an MSI, to make life easier for Windows folks.

What happens next?

For me? Rest. A lot of rest. Really a lot of rest. It’s been an exhausting last few months in the run up to the release, chasing down lots of little semantic details that we really wanted to get straightened out ahead of the freezing of the Perl 6 Christmas language specification. It was worth it, and I’m really happy with the language we’ve ended up with. But now it’s time to take care of myself for a while.

Come 2016, the work will go on. However, the focus will shift. For compiler and runtime folks like me, the focus will be largely on performance and reliability engineering. Now we have a stable language definition, it makes much more sense to invest more heavily in optimizing things. That isn’t to say a great deal of optimization work hasn’t already taken place; many of us have worked hard on that. But there’s a lot more that can, and will, be done.

Even more important than that work, however, will be the work that takes place on the Perl 6 ecosystem in the year to come. Since we announced the Christmas target for a stable language/compiler release, a number of new faces showed up up to help with writing modules and building supporting infrastructure. Now, their work won’t have to contend with us compiler hackers breaking the world under them every week – and that hopefully will encourage more to dive into the ecosystem work also. Maturity here will take time, but there’s plenty of expertise and wisdom on these matters in the Perl community.

Rakudo will stick to a monthly release cycle. We’ll be making a number of process changes to help us deliver those monthlies at a higher quality, especially with regard to not regressing on the 6.c language test suite, key modules, and ecosystem tooling. These changes will also introduce a stability level that lies between bleeding edge commits and monthlies. We also expect the language specification itself to have a small number of minor versions between now and Diwali, and we will treat these a lot like we have the Christmas release, with some extra attention going into the release than normal monthlies will get. Those releases will tend to have seen a greater focus on semantics detail, so will for now serve as our “stable track” for those who want something more occasional than the monthlies. We’ll see how that serves us and our userbase, and adjust as needed. It’s all about keeping the ceremony of contributing and releasing low, while keeping the quality of releases up.

Last but not least…

…I’d like to say thank you. Thank you to all those who have been my fellow contributors on the Perl 6 project, for being among the best people I’ve ever worked with on anything. Thank you to all those who came to my Perl conference talks and read my ramblings here over the years, and provided feedback and encouragement. Thank you to those who donated financially to the Perl 6 project, and so enabled Perl 6 to be part of my day job. And last, but absolutely not least, thank you to those of you who have written and run Perl 6 programs over the years, and shared that you were doing so – because perhaps the greatest reward of all for a compiler/VM hacker is seeing others use their work to build their own great creations.

Together, we’ve breathed life into a new Perl. I’m damn proud of what we’ve built together – and I can’t wait to see how people put it to work.

Merry Christmas!