A few words on Perl 6 versioning and compatibility

Recently, I put together a set of guidelines for Perl 6 versioning and backward compatibility. They have been refined with input from the Perl 6 community. Together, they amount to a read of over 4,000 words, and are fairly detailed. You can consider this post a TL;DR version of the guidelines, giving the big picture idea of what they mean.

There’s more than one thing to version

In common with many languages, Perl 6 has a clean separation of language version and compiler version. Just as C has language versions like C89, C99, and C11, Perl 6 has versions 6.c, 6.d, 6.e, and so forth. To compile C you might be using GCC, Clang, MSVC, or something else, and these have their own version schemes. To compile and run Perl 6 you’ll for now probably be using Rakudo, which has monthly releases with a boringly predictable naming scheme: 2015.11, 2015.12, 2016.01, etc.

Perl 6 language version = a test suite

A given Perl 6 language version is defined by a test suite. An implementation that passes that test suite can claim to provide that version of the Perl 6 language. Rakudo 2015.12 was the first release to provide Perl 6.c, because it passed the test suite. Rakudo 2016.01 also passes the test suite, and thus provides Perl 6.c too.

The master branch of the test suite represents the latest development of the Perl 6 language. Language releases are represented by tags (implying that 6.c, for example, is immutable).

Programs may declare which version they want

The first statement of a program (either a script or a module) may contain a use statement requesting a particular language version:

use v6.c;

Implementations are required to refuse to run code for versions of Perl 6 they do not provide.

No language version declaration implies latest released version

Pretty much self explanatory. Asking for nothing in particular gets you the latest and greatest Perl 6 version the implementation has to offer. Meaning that 20 years down the line, people wanting to show off the latest things won’t have to prefix all their example snippets with use v6.n.

Module installers/directories may be more demanding, and require modules to specify a language version. That’s a decision for the those making such tooling.

An implementation may support many language versions at once

Assuming that we release a Perl 6.d sometime during 2016 (remember that this means “release a test suite defining what 6.d is”), then Rakudo 2016.12 will provide both Perl 6.c and Perl 6.d. To do that, it will be required to pass both the 6.c and 6.d test suites.

Implementations will need to go to moderate effort to do this

New syntax (new operators, new phasers, etc.) introduced by Perl 6.d should not be available to a program declaring use v6.c. This is to ensure that the syntactic additions do not cause breakage. For example, adding a DONE phaser could break any code that had a DONE sub and called it with listop syntax. Rakudo will, for now, support this by guarding such syntax in the parser with a version assertion. (Another strategy is to ship multiple parsers, presumably loading “on demand” those that are actually needed.)

Lexical aspects of the CORE.setting (the builtins) get the same treatment. Recall that in Perl 6, the builtins are not imported, but rather are in the outer lexical scope of the program. If Perl 6.d adds new subs, different implementations of subs, new classes or roles, or new constants, they should go in a CORE.d.setting, or some equivalent. Implementations are free to choose exactly how they will structure things. In Rakudo, we’ll retain CORE.setting with the “base” set of things, and have a CORE.d.setting that overrides and adds or overrides things as needed for Perl 6.d. This means that new Perl 6 language versions can change the behavior of existing operators or subs in ways that are not backward compatible, without breaking code that declared in a use statement it wants a previous version of Perl 6.

Late-bound things (methods, dynamic variables) have different rules. See the full guidelines for the details, but the upshot is that Perl 6 designers have a bit less flexibility in how they evolve methods on built-in types compared to subs, syntax, etc. Backward incompatible changes (as judged by the test suite) are not allowed; new methods or new overloads that don’t conflict with existing behaviors are. The other key difference is that no effort is made to “hide” methods added as part of newer versions of Perl 6 from callers using an earlier version of the Perl 6 language.

TL;DR, supposing you have a module that declares use v6.c:

  • The onus is on Perl 6 implementations to not let your program run into 6.d syntax changes or new subs/constants in the builtins
  • The onus is on you to not use methods that did not exist in 6.c. However, tooling will probably come to exist to help out (for example some kind of Module::FlightCheck that uses rakudobrew to grab the 2016.01 release, which is known to support nothing later than 6.c, and run your module’s tests on it).

If you think the second part of this sucks, then read the guidelines to get an idea why the alternatives all suck harder. :-)

Long Term Support and security releases

Rakudo releases every month, and since each release will provide a range of Perl 6 language versions by passing their official test suites, then in a sense every one is an “official” release. To be very clear, there is no release we should talk about as being “the 6.c release of Rakudo” or “the 6.d release of Rakudo” (though some folks probably will anyway, no matter what I say, and they’ll most likely mean “the first Rakudo release that supported 6.c” or so).

What we will do is declare some releases as “Long Term Support” releases. This label will be applied to releases some time after they have been made, so we can support releases that we know behaved reasonably well in the wild – at least in their first month or so “out there”. For example, suppose that 2016.02 is fairly well received. We might declare it a LTS release, and we’ll declare with that the period of time we intend to “support” it for.

What does support mean? It means that, in the event of security patches or serious bug fixes, we’ll produce bug compatible releases of all current Long Term Support versions of Rakudo. For example, suppose 2016.02 and 2016.10 were marked as LTS releases for a period of 12 months. In December 2016, we find a serious security bug in Rakudo. We’d release a 2016.02.1 and 2016.10.1, which would branch from the 2016.02 and 2016.10 tags and have the required patch(es) cherry-picked in. This would allow upgrades to get the security fix with a very high degree of confidence that existing code will not break.

Minor language versions

One thing we’d prefer to avoid is people declaring dependencies in their Perl 6 code on particular compiler versions. There’s no way to prevent it, but we can try to reduce the temptation to do so. The typical use case would be wanting to depend on a particular bug fix. Fixes get coverage in the language test suite, and so will be part of the next language release – but since the major language versions will tend to have at least a year between them, that could be a bit too long to wait.

Therefore, we’ll also have minor language versions, named 6.c.1, 6.c.2, etc. These give something implementation-independent to depend on. Chances are these will be needed more in the short term than in the long term.

What will we market?

Major language versions, primarily. We’ll use minor language versions to focus on incremental improvement and refinement. The interesting “next big thing” will be the major language versions. Each major language version will get a name. We’ve picked celebrations as the naming theme; 6.c was “Christmas”, and 6.d will be “Diwali”. (That doesn’t mean we’ll be pushing ourselves to actually ship it anywhere near where Diwali falls in the calendar. We already did that with Christmas, taking care to release on Christmas day.) So, look out for a talk on “What’s coming in Perl 6 Diwali” at some conference later on in the year. :-)

Trying out the future

One question the above raises is how to try out the latest implementation work towards the next major version. For that, we’ll use versions such as 6.d.a (6.d “alpha”). So:

use v6.d.a;

Will get you access to the stuff we expect to be in 6.d. Note these lettered versions will really be giving you the current work in progress and come with absolutely no backward-compatibility or stability promises, and support for them will be dropped swiftly after the actual language release of 6.d.

Where will new spectests go? Do I need to tag them somehow?

In the master branch of the repository of all spectests, and they need no tagging up besides the usual fudging. Released language versions are handled as tags, which are immutable.

Are the spectests really enough to specify a language?

I think they’re the best tool we have for the job at the moment. We might want to look towards property based testing ala QuickCheck some more, but that’s still a test-based approach. Natural language doesn’t have the precision, but more critically lacks the verifiability (that is, you can’t run a Perl 6 compiler against a natural language specification). Formal methods, such as operational or denotational semantics, offer greater precision than tests, but the intersection of people who know how to apply those and who want to contribute to Perl 6 is probably tiny. Certainly they lack the accessibility of a test suite expressed in Perl 6, and so would take us away from the goal of Perl 6 being a community’s language.

All that said, it’s fairly clear we need anywhere between 2 and 10 times the current number of tests to have comfortable coverage of both the language and its wide array of built-ins. We’ll be looking into coverage analyses to help us understand where those tests are most lacking.

You didn’t answer my really important question about Perl 6 versioning!

Then leave a comment, and maybe I’ll do a follow-up post to answer it. :-)

This entry was posted in Uncategorized. Bookmark the permalink.

6 Responses to A few words on Perl 6 versioning and compatibility

  1. stmuk says:

    How does this all work from the perspective of end users? Consider some of

    1. Someone packaging Perl 6 for a popular linux distribution

    2. Someone using Perl 6 in production in a company with fairly strict change
    control and/or an uncooperative sysadmin ops team.

    3. Someone moderately interested in Perl 6 but who uses other languages.

    Obviously monthly releases aren’t going to be suitable for any of these people
    and they would be better served with much more infrequent releases. Perl 5
    releases yearly but most of these end users wouldn’t even use updates that
    frequently. To these people Long Term Support (LTS) means support for 3-5
    years (at least) with vendor security patches added to a version usually no
    longer supported upstream.

    I think use of LTS at this point for any Perl 6 release will lead to
    unrealistic expectations for these sort of users since the best we are likely
    to manage is a year. Thinking of a replacement term is hard (“Stable” implies
    other releases are “Unstable”) and the best I can come up with right now is
    “Recommended” — recommended in the sense of being recommended to conservative
    users and those outside the immediate Perl 6 community. Also “Medium Term
    Support” or “Orthodox” are another suggestions.

    There is no mention in your post of Rakudo Star (R*). In the past it was
    always intended that most end users use R* and not the monthly releases and to
    this end only R* releases were announced on http://rakudo.org/. It would make
    sense to continue with these and base them on the LTSish releases. These are
    the obvious foundations on which to build our own binary releases (currently
    Windows MSI, hopefully soon Mac DMG and maybe RPM and DEBs).

    It would also be nice to be able to use a tools like rakudobrew or git to
    automatically install the last LTSish release from a static branch name which
    always points to the last 2016.02.x or whatever.

  2. I’m curious if there will be a built-in means for correctly ordering version numbers. For example, a typical ASCII/POSIX C sort would show v6.d.a coming before v6.d and v6.d.1.

    I can’t think of why I would need to sort Perl 6 spec versions, but you never know, and I’d rather there be a standard correct way to do it than deal with 20 different ad-hoc implementations :)

    • Zoffix Znet says:

      That already exists with the .sort method:
      $ perl6 -e ‘say (v6.d.a, v6.d, v6.d.1, v6.c).sort’
      (v6.c v6.d.a v6.d v6.d.1)
      The `before` and `after` operators can also tell you which version is “larger”:
      $ perl6 -e ‘say v6.d.a before v6.d’
      $ perl6 -e ‘say v6.z before v6.d’

  3. draxil says:

    That was informative, thanks.

  4. Pingback: 2016.7 More Versioning | Weekly changes in and around Perl 6

  5. Pingback: 2016.12 Less Happening | Weekly changes in and around Perl 6

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.