NQP gets MoarVM support, cursor reduction, and other news

I thought it was high time for a round-up of news about the Perl 6 related stuff that I’ve been working on – as usual, with the help of many others. So, here goes!

MoarVM Backend in NQP master

In the last progress update, I mentioned that MoarVM could host NQP and pass most of the NQP language test suite. We got to that point by using a cross-compiler running on Parrot to compile the NQP sources into MoarVM bytecode. Fast forward a month, and we’ve reached the point of having NQP bootstrapped on MoarVM. What’s the difference? Simply, that you can use NQP running on MoarVM in order to build an NQP from source. This means that both bytecode generation and serialization are now working. Since some of the source files we’re compiling are also fairly large (4000 lines), it’s also been a fairly good hardening exercise; many bugs have been fixed.

The MoarVM support has now been added to the NQP repository. Just as you only need a JVM to build NQP for the JVM, you also only need MoarVM to build an NQP for MoarVM. That is, it’s bootstrapped and can stand alone now. NQP monthly releases from here on will thus come with support for three backends: Parrot, JVM and MoarVM.

Better still, NQP on MoarVM now passes the full NQP test suite. It also does so faster than any other backend. This doesn’t mean MoarVM is faster for all things; if you write NQP code with a tight integer loop, or something long running, the JVM’s JIT will kick in and typically come out ahead after the slow start. MoarVM just gets moving a lot faster than the JVM, which is what matters for running lots of short-lived tests.

This means we’re now ready to start work on getting Rakudo running on MoarVM. I don’t have any predictions just yet on precisely when we might land this; the rate of progress in the next couple of weeks, as we dig into it, will provide an indication. Reaching the point of being able to bootstrap NQP on MoarVM is a significant achievement along the way, though. While NQP is both smaller and simpler than full Perl 6, it still requires being able to execute Perl 6 grammars, eval code, do a whole range of OO-related things (including classes, roles and meta-programming), perform multiple dispatch, handle BEGIN time, and so on. These are all things that a full Perl 6 needs, so it’s very good to have them in place.

Allocating Less Cursors

Allocation profiling work by hoelzro and timotimo (I hope I remembered right; correct me if not!) indicated that both NQP and Rakudo were allocating a huge number of Cursor objects. So what are these? They are objects that keep track of parsing state. They are created as we enter a rule or token in the grammar, thrown away if it fails, or typically placed into a tree of cursors if it passes (though sometimes we only care for pass/fail and so quickly throw it away again). Naturally, we’re going to allocate quite a few of these, but 2.14 million of them being allocated while parsing Rakudo’s CORE.setting seemed decidedly excessive.

I decided to spend some time trying to understand where they all came from. NQP itself tends to be a bit easier to analyze than Rakudo, so I started there, and added some instrumentation to record the number of times each production rule or token was hit and led to allocation of a Cursor. I then used this to gather statistics on a fairly large source file from the NQP build. It started out at 284,742 Cursor allocations.

The first big win from this came when I realized that a huge number of these Cursors were just allocated and returned, to indicate failure. A Cursor is born in a failed state, and many places were just creating them and returning them without any further work, to say “I failed to match”. Thing is, failed Cursor objects all look about the same. The only thing they differ by is type, and for each Cursor type we have a ParseShared instance. Thus, it wasn’t too hard to create a shared “fail Cursor” and just look it up in a bunch of places, rather than allocating. That shaved off over 50,000 allocations. A related realization about improving MARKED and MARKER led to another 30,000 or so allocations chopped. Note that none of this really leads to overall reduced memory usage; all of these objects were quickly collectable. But allocation makes GC run more often, and thus carries some amount of runtime cost.

Further examination of the statistics showed hotspots that seemed unreasonable. The problem wasn’t that they allocated a Cursor needlessly, but rather than we should never have been calling the grammar rule in question so many times. This led to a number of grammar optimizations. In one case, just making sure that a certain rule got a declarative prefix meant the LTM mechanism could very quickly rule it out, saving 15,000 calls to the production rule, and thus cursors. The other big discovery was that the ident built-in was not having an NFA properly generated for it. This is notable because ident features in various paths in the grammar, and thus meant we were failing to quickly rule out a lot of impossible paths through the grammar using the NFA (which is the cheap way to rule things out). With a few other tweaks, all told, we were down to 172,424 Cursor allocations, just 60% of what we started out with.

The statistics for Rakudo’s CORE.setting showed we started out doing 2,196,370 Cursor allocations. Some of the fixes above also aided Rakudo directly, making a good cut into this number. However, analysis of the statistics revealed there were more wins to be had. So far, we managed to bring it down to 1,231,085 – a huge reduction. There’s likely a lot more wins to be had here, but I think we’ve picked all of the large, low-hanging fruit by now.

Rakudo Debugger on JVM

I spent some time getting the Rakudo Debugger to also work on the JVM. I’ve still got some work to do on making it easily buildable, and so it can be installed with Panda on the JVM like it can on Parrot. But the rest of the work is done, and I was able to step through programs, set breakpoints, examine variables and so forth. So, that’s another of the “things missing on the JVM port” mostly gone.

Rakudo on JVM spectest

I also did a little work to fix more of the spectests that pass on Rakudo Parrot, but fail on Rakudo JVM. The result: 99.9% of the spectests that pass on Rakudo Parrot also pass on Rakudo JVM. To put an exact number on it, that’s 27 tests different. Around half are Unicode related, which isn’t so surprising since we’ve borrowed Java strings for the time being; in the end, we’ll need to do an NFG implementation on the JVM. The others still need to be hunted down. I think it’s fair to say that you’re rather unlikely to run into this 0.1% in day-to-day usage of Rakudo on the JVM, however. In fact, the thing you’re most likely to miss today is NativeCall – which arnsholt has been working on, and I plan to join in with during the coming days.

All in all, things are coming along very nicely with Rakudo’s JVM support. It was just 11 months ago – not even a year – that I started to build a little bit of infrastructure so we’d be able to create a tree-like data structure in NQP and get JVM bytecode made from it. By now, thanks to the contributions of many besides me, Rakudo runs on the JVM and is very capable there. When it comes to concurrency, it’s the most capable Rakudo you can get. And, once it gets past the slow startup, it’s usually the fastest. It’ll be interesting to see where we stand on these things in another six months or a year’s time. Here’s hoping for another productive time ahead!

Posted in Uncategorized | 2 Comments

Material from the Rakudo and NQP Internals course

A little over a month ago, lizmat++ contacted my employer, Edument AB, to discuss using our training services to deliver a 2-day workshop aimed at current and potential Rakudo and NQP contributors. For those of you who don’t know, aside from working on Perl 6 I also work teaching/mentoring on various topics (typically software architecture, TDD, and advanced C# programming). The goal was for me to spend a couple of days explaining a bunch of topics – including NQP, grammars, QAST, 6model, and bounded serialization – to help people know where to start or to help them get into unfamiliar areas of the code.

The workshop took place this last weekend in Frankfurt. I’d expected we might have 5-6 people sign up; in the end, we had around 15 people attending! The workshop involved a combination of teaching and exercises, with plenty of chances to ask questions. Happily, there were tasty lunches and dinners too (though I can’t take any of the credit for this side of things). I greatly enjoyed teaching the course; I like both working on Perl 6 and doing my teaching work at Edument, and being able to do both at once was delightful! :-)

One aim of developing the course was to help with the state of documentation of the NQP and Rakudo internals. Typically at Edument, we only give out our material to those attending a delivery of our courses, and even then not the original source! However, for this course, we’ve made an exception and released the course material under a Creative Commons license. So, while I hope that we’ll be able to put together future “live” deliveries of the material for other new (or potential) contributors, this means it will now always be available to the community at large. :-)

I hope this will prove a valuable resource for those contributing or interested in contributing to Perl 6 development, and would like to take a moment to thank Edument for once again being supportive of my work on the Perl 6 project!

Posted in Uncategorized | 4 Comments

A MoarVM Progress Update

A while back, I wrote here to introduce the MoarVM project, which aims to build a Virtual Machine specially tailored for the needs of NQP and Rakudo. Since then, a lot of progress has been made and the project has attracted a number of new contributors. So, it’s time for a round-up of the latest MoarVM news.

MoarVM Hosts NQP

Back when I first introduced MoarVM, we had a partly complete cross-compiler that used NQP on Parrot to parse NQP source code and build its AST, then turned it into MoarVM bytecode. This enabled us to cross-compile some of the NQP test suite to MoarVM. Since then, we’ve been improving the cross-compiler so it is able to translate NQP itself into MoarVM bytecode, as well as making the VM capable of doing the things that NQP requires to run.

In the last week, we’ve reached the point of having a self-hosted NQP on MoarVM that passes most of the NQP test suite. What does this really mean? That MoarVM is capable enough not only to run programs written in NQP, but also to run the NQP compiler (meaning it has pretty complete Perl 6 grammar support), its own backend code-generator and to be able to evaluate the result of this compilation.

We’re down to 5 files from t/nqp that either fail outright or fail some tests. I expect we’ll be able to get these fixed within the next week, and also make progress on the serialization tests. Beyond this, it will be time to start working on getting this self-hosted NQP able to build itself. At this point, the cross-compiler will no longer be needed, and the MoarVM backend code will migrate to the main NQP repository, joining the Parrot and JVM backends.

How slow is it?

I’ve been pushing pretty hard for us to focus on getting things working before getting things fast; for now keeping the code easy to work on and implementing features on the path to having a Rakudo on MoarVM is far, far more important. That said, there is already an encouraging result. While it’s not quite a fair test given the five of the 77 test files that do not run to completion, currently MoarVM can run the NQP test suite in 16s on my main development machine, This compares to 21s on Parrot (which means we come out ahead at startup time, execution time, or a mixture). While it’s not incredibly faster (though of course running lots of small test files is startup-dominated), it isn’t a bad starting point given we’ve barely scratched the surface of what we’ll be able to do in terms of optimization (for example, no JIT yet, known algorithmic issues in handling of GC inter-generational pointers, and comparatively little time invested in optimization all over the VM). By the way, thanks to its much greater start-up time, NQP on the JVM takes a huge 139s. However, much like in the story of the tortoise and the hare, leave things to run for long enough and the JVM backend still typically comes out ahead, sometimes very significantly so.

On memory usage, to compile and run a simple “hello, world” in NQP, MoarVM’s memory usage clocks in at just over half that which Parrot uses, which in turn clocks in at about half that of the hungry JVM. :-)

While the JVM backend doesn’t come out looking too awesome here resource usage wise unless you’re doing something long-running, it’s worth remembering that the JVM port of Rakudo is just a couple of months old, and the whole porting effort less than a year old. In that sense, we’ve a lot of room for improvement yet.

Goodbye APR, welcome libuv

Originally, MoarVM was using the Apache Portable Runtime for IO, multi-platform thread support and a handful of other things. By now, we’ve moved over to using libuv. This isn’t because there was anything inherently wrong with the APR, and while we’ve ended up not using it, I’ve come out with a positive impression of it. So, thanks to the APR developers! The primary reason we moved to using libuv instead was to ensure we can provide support for asynchronous I/O on a relatively wide variety of platforms. We were also using a mix of the atomic primitives from the APR and from libatomic_ops; we’ve now standardized completely on libatomic_ops for this.

Build and Dependencies

There’s a bunch of stuff I’m decent at doing, but build systems is not one of them. My first pass was a case of, “make something work, and hope somebody replaces it.” Happily, that has now happened, so the build system is in much better shape. We’ve also got things in place for building MoarVM as a shared library, and for pulling in varius dependencies while keeping them at enough of a distnace that we should be able to depend on system versions of them (trying to balance easy source builds with keeping future packagers happier).

Contributors

Since becoming a public project, MoarVM has grown its contributor base. In total, 16 authors now show up in the commit log and I’d say we’re at around 5 core developers who commit notable things relatively often. I’m happy that people have felt inspired to contribute, and that things are progressing fairly swiftly now (I hoped we’d be a little further by this point, but my tuit supply over the summer was not what I’d hoped). Currently, it looks like the October NQP release will be the first to include MoarVM support, and from there we’ll focus on getting Rakudo onto MoarVM.

Posted in Uncategorized | 4 Comments

YAPC::EU 2013 Slides

I’m just back from this year’s YAPC Europe in Kiev. I’ve liked Kiev since the first time I visited many years ago, and after around two years since the last visit, I was glad of an excuse to return. It was the same beautiful, rather hilly city I remembered, though decidedly warmer than I remember it – probably because this is the first visit I made there in summer. In the company of the many good Perl folks attending YAPC, I enjoyed plenty of nice food, and even caught some good Belgian beers thanks to the wonderful Belgian Beer Cafe.

This year I submitted three talks, expecting one or two would be accepted. Instead, all three were! So, I talked about:

  • Rakudo on JVM – this session explained the motivation for adding a JVM backend, the compiler architecture that enabled it, how it was implemented, the current status, the support so far for Java interoperability and the plan from here. Seems to have been well received.
  • Concurrency, Parallelism and Asynchrony – this session showed the work I have been doing to build basic support for parallel, asynchronous and concurrent programming in Perl 6. This was the best attended and also, I believe, the most discussed of my talks this year. There is still much work to do in this area, but what’s done so far caught some interest. Once I’m recovered from YAPC, I’ll dig back into it.
  • MoarVM – this session talked about the motivation for building a new VM, its overall design, the current status and what’s in store. It’s also the first talk I ever gave on MoarVM. The most lightly attended and most hastily prepared, but still it seemed to be appreciated by those who attended.

Enjoy the slides, and hopefully the videos will make it online soon too.

I also agreed to attend this years Austrian Perl Workshop, where the hills will be alive with the sound of Perl 6 in the lovely Salzburg, sometime in November. :-)

Posted in Uncategorized | 1 Comment

Rakudo JVM News: More tests, plus Thread and Promise prototypes

Last time I wrote, the Rakudo on JVM port was passing around 92% of the specification tests that Rakudo on Parrot can. In the last couple of weeks, we’ve continued hunting down and fixing failures. I’m happy to report that we have already passed the 98% mark – well beyond the 95% I was aiming for by the July release! I’m optimistic that we may be able to push that up to 99% in the coming days. Either way, we’re closing in on the goal spectest wise, meaning the focus should soon move to getting tools like Panda working, followed by the module ecosystem. Happily, arnsholt++ has already started working on the NativeCall support that many modules depend on.

One of the reasons for adding a JVM backend is to unblock work on Rakudo’s support for asynchronous, parallel and concurrent programming. With a YAPC::EU talk on these topics looming, and hating to take to the stage without anything to demonstrate, I’ve started working on this area. It’s early days yet, but here is a quick look at what’s already possible.

There is some basic support for doing stuff in threads.

say "Creating a couple of threads...";

my $t1 = Thread.start({ sleep 1; say "Thread 1 done"; });
my $t2 = Thread.start({ sleep 2; say "Thread 2 done"; });

say "Waiting for joins...";
.join for $t1, $t2;
say "Joined!";

This does what you’d naturally expect. However, threads are kind of like the assembly language of parallel programming: occasionally you want to work at that level, but usually it’s much better to work in terms of higher level constructs. Thus, while you can do the above, I don’t suggest it. So what’s available at a higher level? Well, so far, promises are.

say "Creating two promises...";

my $a = async { sleep 2; say "omg slept 2"; 27 }
my $b = async { sleep 1; say "omg slept 1"; 15 }

say "Scheduler has $*SCHEDULER.outstanding() tasks";
say "Waiting for results...";
say $a.result + $b.result;

The async construct evaluates to a Promise object (name subject to change; Future or Task are other options we could steal from other languages). A Promise is an object that represents a piece of ongoing work. It is not backed by a thread of its own; instead, it is scheduled onto a pool of threads that are spun up on demand, up to a limit. Alternatively, a Promise could be backed by some kind of asynchronous I/O. The point is that it doesn’t much matter what the exact nature of the work is, just that there’s a common way to talk about concurrent work and write combinators over them.

When you ask for the result of a Promise and it is not available, you will block until it is available. If the inside of the async block died, then the exception will be thrown at the point the result method is called. There is also a method “then” for chaining on extra work to be done once the promise is either completed or fails due to an exception:

my $a2 = $a.then(-> $res { say "Got $res.result() from promise a" });

This returns another Promise, thus allowing chaining. There is also a sub await that for now just calls the result method for you, on a whole list of promises if you pass them. Here’s an example:

say [+] await dir('docs/announce').map({
    async { .IO.lines.elems }
});

This creates a Promise per file in the directory that will count the number of lines in the file. Then, await will wait for each of the promises to give a result, handing them back as they come in to the reduction. Note that in the future, this could probably just be:

say [+] hyper dir('docs/announce').map(*.IO.lines.elems)

But we didn’t implement that yet, and when it does happen then it will most likely not work in terms of simply creating a promise per element.

Remember that promises are much lighter than threads! We’re not spinning up dozens of threads to do the work above, just spreading the load over various threads. And yes, the parallel version does run faster on my dual core laptop than the one that isn’t using async/await.

Future plans for promises include combinators to wait for any or all of them, an API for backing them with things other than work in the thread pool, and making await a bit smarter so that it can suspend an ongoing piece of work in the thread pool when it blocks on another promise, thus freeing up that thread for other work.

Of course, all of this is early and experimental; any/all of the above can change yet, and it’s a long, long way from being exercised to the degree that many other parts of Rakudo have been. Expect changes, and expect many more things to land in the coming months; on my list I have asynchronous I/O and an actors module, and I know pmichaud++ has been thinking a bit about how to evolve the list model to support both race and hyper.

Anyway, that’s the latest news. Next time, hopefully there will be yet more spectest progress and some nice sugar for sorear++’s ground work on Java interop, which is what I used to build the threads/promises implementation.

Posted in Uncategorized | 9 Comments

Rakudo on JVM Progress Update

In the spirit of “release early, release often”, last month’s Rakudo compiler release was the first one to come with very basic support for running on the JVM. It supported a small number of the features that Rakudo on Parrot does. Of note, it could pass the sanity tests, a small test suite we keep in the Rakudo repository. The sanity tests ensure that the compiler is functional enough to compile Test.pm, which is a prerequisite for running the specification tests. Essentially, they make sure that we can crawl our way to the starting line, where the real race begins. :-)

So, since that release, we’ve been working hard on getting Rakudo on JVM able to pass more of the specification test suite, gradually increasing its capabilities, hunting bugs, adding missing bits of infrastructure and guts code in order to get features working, and so forth. Progress has been rather swift, and today’s automated test run by Coke++ shows that Rakudo on JVM is now passing 92% of the tests that Rakudo on Parrot does!

So what does this number really mean? It means that Rakudo passes 92% of the individual test cases on the JVM that it does on Parrot. To put that into perspective, that’s more tests than either of Niecza (82%) or Pugs (36%), meaning that Rakudo on JVM is now the second most spectest-passing Perl 6 implementation. Not bad, to say around 8 months ago not a single line of implementation code related to Rakudo JVM targeting had been written.

However, the raw number should be taken with a pinch of salt. Here’s why. The test suite is not especially uniform in terms of tests it dedicates to each feature. For example, there are hundreds of passing tests dedicated to providing good coverage of Unicode character property matching in regexes, and trigonometry easily has many hundreds too. By contrast, the slurp function has more like dozens of tests – since you can comfortably cover it that way. However, these are failing still, and I’m pretty sure that more Perl 6 users depend on slurp that Unicode character properties and trig.

Anyway, my aim is that we’ll be some way past the 95% mark in time for the July compiler release. And yes, I’ll make sure slurp gets fixed before then! :-) I suspect the last couple of percent will be the most tedious, but it feels good to be reaching towards them.

In the coming weeks, I expect the focus to start shifting from the compiler to the ecosystem: getting Panda working with Rakudo on JVM, and starting to run the module tests and work through the issues we find there. Calling Java libraries and work on parallelism are also planned, not to mention digging in to optimization work, since this initial phase has very much been focused on “get things working”. All in all, plenty to be working on, and plenty to look forward to.

Finally, a week or so ago I spent a couple of hours being interviewed by Nikos Vaggalis for Josette Garcia’s blog. It was a very wide-ranging interview and I managed to spout sufficiently in response to the questions that it’s being published over three posts in the coming weeks. You can find the first part of the interview here; enjoy!

Posted in Uncategorized | 3 Comments

MoarVM: A virtual machine for NQP and Rakudo

Over the course of the last year, we’ve been working to make both NQP and Rakudo more portable. This has primarily been done to enable the JVM porting work. While the JVM is an important platform to target, and initial work suggests it can give us a faster and more stable Rakudo, there are some use cases, or users, that the JVM will not cater to so well. Startup time will likely remain a little high for one-liners and quick scripts. Other potential users simply don’t want to use the JVM, for whatever reason. That’s fine: there’s more than one way to do it, and our strategy has been to enable that by adding support for the JVM without breaking support for Parrot. Additionally, pmurias will be working on a JavaScript backend for a GSoC project.

Today I’d like to introduce some work that, all being well, will lead to an additional “way to do it” arriving over the next several months. A small team, composed of myself, diakopter, japhb and masak, have been quietly working on taking the design of the 6model object system and building a runtime around it. Thus, we’ve created the “Metamodel On A Runtime” Virtual Machine, or the “MOAR VM”, which we’ve taken to writing as “MoarVM”.

This is not a release announcement. At present, MoarVM runs neither NQP nor Rakudo, though a cross-compiler exists that allows translating and passing much of the NQP test suite. We’re revealing it ahead of YAPC::NA, so it can be openly discussed by the Perl 6 team. The goal from the start has been to run NQP, then run Rakudo. The JVM porting work has established the set of steps that such an effort takes, namely:

  1. Build an NQP cross-compiler that targets the desired platform. Make it good enough to compile the MOP, built-ins and the classes at the heart of the regex/grammar engine. Make it pass most of the NQP test suite.
  2. Make the cross-compiler good enough to cross-compile NQP itself.
  3. Close the bootstrap loop, making NQP self host on the new platform.
  4. Do the Rakudo port.

At the moment, the JVM work is well into the final step. For MoarVM, we’ve reached the second step. That is to say, we already have a cross-compiler that compiles a substantial range of NQP language constructs into MoarVM bytecode, including the MOP, built-ins and regex-related classes. Around 51 of the NQP test files (out of a total of 62) pass. Work towards getting the rest of NQP to cross-compile is ongoing.

Since anybody who has read this far into the post probably has already got a whole bunch of questions, I’m going to write the rest of it in a question-and-answer style.

What are the main goals?

To create a VM backend for Rakudo Perl 6 that:

  • Is lightweight and focused on doing exactly what Rakudo needs, without any prior technical or domain debt to pay off.
  • Supports 6model and various other needs natively and, hopefully, efficiently.
  • Is a quick and easy build, with few dependencies. I was rather impressed with how quick LuaJIT can be built, and took that as an inspiration.
  • Enable the near-term exploration of JIT compilation in 6model (exploring this through invokedynamic on the JVM is already underway too).

What’s on the inside?

So far, MoarVM has:

  • An implementation of 6model. In fact, the VM uses 6model as its core object system. Even strings, arrays and hashes are really 6model objects (which in reality means we have representations for arrays and hashes, which can be re-used by high-level types). This is the first time 6model has been built up from scratch without re-using existing VM data structures.
  • Enough in place to support a sizable subset of the nqp:: op space. The tests from the NQP test suite that can be translated by the cross-compiler cover a relatively diverse range of features: the boring easy stuff (variables, conditionals, loops, subs), OO stuff (classes, methods, roles, mixins, and, naturally, meta-programming), multiple dispatch, most of grammars (including LTM), and various other bits.
  • Unicode strings, designed with future NFG support in mind. The VM includes the Unicode Character Database, meaning that character name and property lookups, case changing and so forth can be supported without any external dependencies. Encoding of strings takes place only at the point of I/O or when a Buf rather than a Str is requested; the rest of the time, strings are opaque (we’re working towards NFG and also ropes).
  • Precise, generational GC. The nursery is managed through semi-space copying, with objects that are seen a second time in the nursery being promoted to a second generation, which is broken up into sized heaps. Allocations in the nursery are thus “bump the pointer”, the copying dealing with the resulting fragmentation.
  • Bytecode assembly done from an AST, not from a textual format. MoarVM has no textual assembly language or intermediate language. Of course, there’s a way to dump bytecode to something human-readable for debugging, but nothing to go in the opposite direction. This saves us from producing text, only to parse it to produce bytecode.
  • IO and other platform stuff provided by the Apache Portable Runtime, big integer support provided by libtommath, and re-use of existing atomic ops and hash implementations. We will likely replace the APR with libuv in the future. The general principle is to re-use things that we’re unlikely to be able to recreate ourselves to the same level of quality or on an acceptable time scale, enabling us to focus on the core domain.

What does this mean for the Rakudo on JVM work?

Relatively little. Being on the JVM is an important goal in its own right. The JVM has a great number of things in its favor: it’s a mature, well-optimized, widely deployed technology, and in some organizations the platform of choice (“use what you like, so long as it runs on the JVM”). No matter how well Moar turns out, the JVM still has a couple of decades head start.

Additionally, a working NQP on JVM implementation and a fledgling Rakudo on JVM already exist. Work on making Rakudo run, then run fast, on the JVM will continue at the same kind of pace. After all, it’s already been taking place concurrently with building MoarVM. :-)

What does this mean for Rakudo on Parrot?

In the short term, until MoarVM runs Rakudo well, this shouldn’t really impact Rakudo on Parrot. Beyond that is a more interesting question. The JVM is widely deployed and battle-hardened, and so is interesting in its own right whatever else Rakudo runs on. That’s still not the case for Parrot. Provided MoarVM gets to a point where it runs Rakudo more efficiently and is at least as stable and feature complete, it’s fairly likely to end up as a more popular choice of backend. There are no plans to break support for Rakudo on Parrot.

Why do the initial work in private?

There were a bunch of reasons for doing so. First and foremost, because it was hard to gauge how long it would take to get anywhere interesting, if indeed it did. As such, it didn’t seem constructive to raise expectations or discourage work on other paths that may have led somewhere better, sooner. Secondly, this had to be done on a fairly limited time budget. I certainly didn’t have time for every bit of the design to be bikeshedded and rehashed 10 times, which is most certainly a risk when design is done in a very public way. Good designs often come from a single designer. For better or worse, MoarVM gets me.

Why not keep it private until point X?

The most immediate reason for making this work public now is because a large number of Perl 6 core developers will be at YAPC::NA, and I want us to be able to openly discuss MoarVM as part of the larger discussions and planning with regard to Perl 6, NQP and Rakudo. It’s not in any sense “ready” for use in the real world yet. The benefits of the work being publicly known just hit the point of outweighing the costs.

What’s the rough timeline?

My current aim is to have the MoarVM backend supported in NQP by the July or August release of NQP, with Rakudo support to come in the Rakudo compiler release in August or September. A “Star” distribution release, with modules and tools, would come after that. For now, the NQP cross-compiler lives in the MoarVM repository.

After we get Rakudo running and stabilized on MoarVM, the focus will move towards 6model-aware JIT compilation, improving the stability of the threads implementation (the parallel GC exists, but needs some love still), asynchronous IO and full NFG string/rope support.

We’ll have a bunch of the right people in the same room at YAPC::NA, so we’ll work on trying to get a more concrete roadmap together there.

Where is…

Posted in Uncategorized | 5 Comments