rss.xml

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
  <channel>
    <title>aardvark179.github.io</title>
    <link>https://aardvark179.github.io</link>
    <description></description>
    <pubDate>Mon, 04 Feb 2019 15:48:40 GMT</pubDate>
    <lastBuildDate>Mon, 04 Feb 2019 15:48:40 GMT</lastBuildDate>
    <docs>http://www.rssboard.org/rss-specification</docs>
    <generator>Org-page static site generator (https://github.com/kelvinh/org-page)</generator>
    <item>
      <title>Better support for C extensions in TruffleRuby.</title>
      <link>https://aardvark179.github.io/blog/capi.html</link>
      <description><![CDATA[<section id="content" role="main">
    <div id="outline-container-sec-" class="row" style="padding-top: 70px">
        <div class="col-md-2"></div>
            <h1>Better support for C extensions in TruffleRuby.</h1>
            <div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org562a885">1. Interactions between Truffle languages</a></li>
<li><a href="#orgf7a5212">2. C extensions the old way</a>
<ul>
<li><a href="#orgc239c52">2.1. Managed memory</a></li>
<li><a href="#orgd5e4fc3">2.2. Handles everywhere</a></li>
</ul>
</li>
<li><a href="#org3005066">3. Difficult issues with our old approach</a></li>
<li><a href="#org4ddff6c">4. C extensions the new way</a>
<ul>
<li><a href="#org5e88149">4.1. Wrapping and unwrapping</a></li>
<li><a href="#orge5595c0">4.2. Tidying up wrappers</a></li>
<li><a href="#orged17412">4.3. Keeping objects alive in MRI</a></li>
<li><a href="#org409e900">4.4. Periodic marking</a></li>
</ul>
</li>
<li><a href="#orgb27ad38">5. Testing the new approach and what's next?</a>
<ul>
<li><a href="#org3304a46">5.1. Compatibility</a>
<ul>
<li><a href="#org978ed06">5.1.1. The type of <code>VALUE</code></a></li>
<li><a href="#org5e7456e">5.1.2. <code>RARRAY_PTR</code></a></li>
<li><a href="#org16536e3">5.1.3. Calling functions with the wrong arguments</a></li>
</ul>
</li>
<li><a href="#org22fe821">5.2. Performance</a></li>
<li><a href="#org24e877c">5.3. What's next?</a></li>
</ul>
</li>
<li><a href="#org4b751db">6. Conclusion</a></li>
</ul>
</div>
</div>
<p>
We think it is crucial that any alternative Ruby implementation aiming
to be fully compatible with MRI runs the C extensions. TruffleRuby's
compatibility was recently significantly improved, with much better
support that almost completely removes the need to patch C extensions.
</p>

<p>
We've been able to support C extensions in TruffleRuby for a long time
now, but we've always had to patch them while building the C for them
to work. We've just added much better support which almost completely
removes the need to patch extensions. In this article I'll explain how
Truffle based languages can support polyglot calls, how C extensions
used to work in TruffleRuby, how they now work, and what the remaining
differences are with MRI.
</p>
<div id="outline-container-org562a885" class="outline-2">
<h2 id="org562a885"><span class="section-number-2">1</span> Interactions between Truffle languages</h2>
<div class="outline-text-2" id="text-1">
<p>
All Truffle languages can provide support for other languages to
access data and make calls by defining how foreign access is
resolved. This covers things like asking if an object is executable,
or has a size, reading and writing public accessible fields, and so
forth. So JavaScript can easily call methods on a Ruby object, or
access members of a Ruby array as long as we provide foreign access
for Ruby objects.
</p>

<p>
There is also support for lower level concepts in the foreign access
APIs.  You can convert an object to a native representation, as well
as checking whether that object can behave like a pointer.
</p>

<p>
These facilities don't cover everything though, so Sulong provides a
set of functions for converting almost any object into a native handle
and back again. I say almost any object because boxed primitives such
as <code>java.lang.Integer</code> are not supported, but you can always arrange
to wrap these in an object that is supported (and unwrap them again
when converting back from handles) to avoid the problem.
</p>
</div>
</div>
<div id="outline-container-orgf7a5212" class="outline-2">
<h2 id="orgf7a5212"><span class="section-number-2">2</span> C extensions the old way</h2>
<div class="outline-text-2" id="text-2">
<p>
We used Truffle's polyglot features to call C functions from Ruby, and
Ruby methods from C, without touching the arguments at all. This
allowed Ruby objects to pass easily through C code that is interpreted
by Sulong, but fails if those objects have to be passed to a native
shared library or stored on the native heap. To solve that problem we
would patch C extensions in two ways.
</p>
</div>
<div id="outline-container-orgc239c52" class="outline-3">
<h3 id="orgc239c52"><span class="section-number-3">2.1</span> Managed memory</h3>
<div class="outline-text-3" id="text-2-1">
<p>
The first way to avoid problems with the translation of Ruby objects to
native pointers is to avoid doing as much as possible. By default we'd
replace every array of <code>VALUE</code> objects (which would be stored on the
native heap) with a managed object that could store the object without
conversion. We could also allocate managed structs instead of native
one, but this became complicated with complex structures which had
nested members of fused arrays.
</p>
</div>
</div>
<div id="outline-container-orgd5e4fc3" class="outline-3">
<h3 id="orgd5e4fc3"><span class="section-number-3">2.2</span> Handles everywhere</h3>
<div class="outline-text-3" id="text-2-2">
<p>
The second way to solve these problems was to use the functions Sulong
provided to convert between managed objects and native handles. These
functions were hard to use without introducing memory leaks however,
and did require that we patched every point at which a Ruby object
needed to be converted to or from native memory.
</p>
</div>
</div>
</div>
<div id="outline-container-org3005066" class="outline-2">
<h2 id="org3005066"><span class="section-number-2">3</span> Difficult issues with our old approach</h2>
<div class="outline-text-2" id="text-3">
<p>
So, there are a couple of issues with the old approach and resource
leaks. Let's take a look at the behaviour we would like, the problems
we could see with the old approach, and what we could do to fix
those. We'll start by considering a simple case of three objects A, B,
and C. A will be a GC root (i.e. it's either some sort of global
object or a variable which the garbage collector knows cannot be
removed), and B and C are Ruby objects. A will have an instance
variable of B, and B will have one of C. So A has a hard reference to
B and B has a hard reference to C like this.
</p>

<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/diagram.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
We'll use red arrows to indicate hard references from now on, and red
boxes to indicate GC roots. The good thing here is that if A stops
being a root, or a hard reference is removed then some or all of the
objects can be collected.
</p>


<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/diagram_gc.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
Now, what happens when A, B, and C are C structs which have been
wrapped in Ruby objects? Well, now our hard references are just
<code>VALUEs</code> in the structs, and for the GC to know those are alive each
object must mark those it has references to. We'll use purple to mark
marked links.
</p>


<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/diagram2.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
So in this situation if A stops marking B, or B stops marking C, or A
ceases to be a GC root then those objects can be collected, just as
they were above.
</p>

<p>
Now what happens in TruffleRuby when we use handles?
</p>


<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/diagram3.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
Now we have a problem. The references from A to B, and B to C are weak
(they are simple handle numbers) so the handle table has to associate
those with the objects they represent by keeping a strong
reference. For B or C to be garbage collected we would need to remove
the handle pointing to them. Likewise if A ceases to be a root
something needs to happen for B and C to be collected.
</p>

<p>
We might be able to do that with a finaliser for A which in turn
releases the handle to B, whose finaliser can release the handle to C,
but the situation may not always be so simple. Consider the following
structure of objects.
</p>


<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/diagram4.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
This is a common sort of structure to find in tools like XML
processors such as Nokogiri. Each node has a reference to the parent
document, and to its own children. If we break the hard reference from
the GC root object to the document then it and all of its nodes can be
collected. The same is true if the nodes hold <code>VALUEs</code> and mark them,
but what happens if we use handles?
</p>


<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/diagram5.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
Well, our diagram has certainly got messier! But it's also hard to
know how we should free those objects nicely. Release the handle that
Object held to Document doesn't help, because other handles exist
pointing to it from the nodes, so the whole cluster remains
uncollected. There doesn't seem to be a nice obvious order in which we
could do it. We could solve almost any situation like this by
introducing weak handles, but that requires patching each C extension,
and carefully analysing how to break these cycles. To really be
compatible we need a different approach.
</p>
</div>
</div>
<div id="outline-container-org4ddff6c" class="outline-2">
<h2 id="org4ddff6c"><span class="section-number-2">4</span> C extensions the new way</h2>
<div class="outline-text-2" id="text-4">
<p>
Our previous approach was enough to get several key C extensions
working, but sometimes they required large patches, and avoiding
resource leaks was tricky. We prototyped several approaches, either
making as many parts as possible managed objects to avoid conversion,
or allowing all Ruby objects to be converted to native pointers, but
both these approaches had issues. So we tried a third approach,
wrapping every Ruby object.
</p>
</div>
<div id="outline-container-org5e88149" class="outline-3">
<h3 id="org5e88149"><span class="section-number-3">4.1</span> Wrapping and unwrapping</h3>
<div class="outline-text-3" id="text-4-1">
<p>
The idea is fairly simple. C extensions will never see raw Ruby
objects, they will only ever see wrappers that know how to convert
themselves to native pointers, and at every point where a Ruby object
needs to be extracted from a wrapper we know there should only ever be
a wrapper or a native pointer. This makes it easy to convert back from
a native pointer to a wrapper. Best of all C extensions don't have to
know this is happening, so although it required a lot of changes to
our C code to wrap and unwrap values that is as far as the changes go.
</p>
</div>
</div>
<div id="outline-container-orge5595c0" class="outline-3">
<h3 id="orge5595c0"><span class="section-number-3">4.2</span> Tidying up wrappers</h3>
<div class="outline-text-3" id="text-4-2">
<p>
It was also important that these wrappers didn't cause the objects
they wrapped to live longer than expected. This was a real problem
with the handle conversion we used to do, and we didn't want to make
it worse. Wrappers obviously need to keep a strong reference to the
object they wrap, and objects should also keep a strong reference to
their wrapper, but converting an object to a native pointer should not
stop it from getting garbage collected at some point, but equally it
mustn't be collected too soon.
</p>
</div>
</div>
<div id="outline-container-orged17412" class="outline-3">
<h3 id="orged17412"><span class="section-number-3">4.3</span> Keeping objects alive in MRI</h3>
<div class="outline-text-3" id="text-4-3">
<p>
MRI keeps objects alive in two ways when they are being used in a C
extension. Any object still on the stack will be seen by the GC and
kept alive, but that isn't enough to preserve values which may have
been assigned to a field in a structure. MRI allows these to be kept
alive by associating the structure with a Ruby object, and allowing
that object to mark other objects it has references to. So, when the
garbage collector traverses all the objects in your Ruby heap it calls
these custom mark functions and the objects will be marked as live as
long as the owners are. There's just one problem, we don't have a GC
which can call custom mark functions, we have to work with any GC on
the JVM. We also can't change the GC to look for native pointers on
the stack which should also keep their respective objects alive.
</p>
</div>
</div>
<div id="outline-container-org409e900" class="outline-3">
<h3 id="org409e900"><span class="section-number-3">4.4</span> Periodic marking</h3>
<div class="outline-text-3" id="text-4-4">
<p>
We can solve this by keeping two lists of objects that need to be kept
alive. Then each time we convert a wrapper to a native pointer we will
add the wrapper to the lists, and it will in turn keep its object
alive. One list is for those objects with pointers on the stack. We
can create this list whenever we enter a C extension, and destroy it
again when we finish the call. The other list is a fixed size buffer
of every wrapper converted to a pointer. Whenever this list becomes
full we'll run any marking functions associated with live objects and
attach lists of marked objects to their owners. Let's see what that
looks like with the example we've used before (I've left off the stack
list to keep the diagrams from becoming too cluttered):
</p>

<div class="figure">
<p><object type="image/svg+xml" data="file:///home/duncan/work/aardvark179.github.io/marking.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>

<p>
As you can see, since the handle table now only holds weak references
to objects, once the preservation table has become full and the
markers have been run there are no strong references from the
preservation table or the handle table to the actual objects. We no
longer need to write special finalisers for the object we create, as
long as we ensure the marking functions do not maintain any strong
references to the objects.
</p>
</div>
</div>
</div>
<div id="outline-container-orgb27ad38" class="outline-2">
<h2 id="orgb27ad38"><span class="section-number-2">5</span> Testing the new approach and what's next?</h2>
<div class="outline-text-2" id="text-5">
<p>
I said at the start of this article that we used to have to patch C
extensions, so how much has our compatibility improved; does this new
approach perform well or is more work required to make it fast; and
what are our next steps?
</p>
</div>
<div id="outline-container-org3304a46" class="outline-3">
<h3 id="org3304a46"><span class="section-number-3">5.1</span> Compatibility</h3>
<div class="outline-text-3" id="text-5-1">
<p>
<a href="https://github.com/oracle/truffleruby/commit/1915b9a4">130 additions, and 1,402 deletions</a> is the best kind of commit to be
able to merge. This new approach has allowed us to remove almost
all our patches for C extensions, even for complex ones such as zlib,
OpenSSL, or pg.
</p>

<p>
Notice we've only removed <i>almost</i> all patches. There are however
still some fundamental differences between us and MRI, but they are
much smaller than they were.
</p>
</div>
<div id="outline-container-org978ed06" class="outline-4">
<h4 id="org978ed06"><span class="section-number-4">5.1.1</span> The type of <code>VALUE</code></h4>
<div class="outline-text-4" id="text-5-1-1">
<p>
One is that our <code>VALUE</code> type is a <code>void *</code> in C. This means we can't
do a switch on a <code>VALUE</code>, so we do still need to patch anything that
tries to do that.
</p>
</div>
</div>
<div id="outline-container-org5e7456e" class="outline-4">
<h4 id="org5e7456e"><span class="section-number-4">5.1.2</span> <code>RARRAY_PTR</code></h4>
<div class="outline-text-4" id="text-5-1-2">
<p>
We also can't yet translate a pointer to a Ruby array's contents to
native. This requires storing the contents in native memory so that
they can be read and altered from C, but ensuring that the view of the
array from Ruby remains consistent with any changes made via a C
extension. The work to support this is in progress and we expect to
resolve this area of incompatibility very soon.
</p>
</div>
</div>
<div id="outline-container-org16536e3" class="outline-4">
<h4 id="org16536e3"><span class="section-number-4">5.1.3</span> Calling functions with the wrong arguments</h4>
<div class="outline-text-4" id="text-5-1-3">
<p>
There are also some small differences imposed by our use of Sulong to
interpret C extensions. One is function declarations may need to be
changed. For example a function declared as taking two arguments must
be passed two arguments, even if the second one is never used, and
<code>int</code> and pointer types may not be as interchangeable as they can be
in native C. We also have trouble with varargs functions in managed
code being called from native libraries, but none of these differences
causes widespread problems and most can be patched without changing the
behaviour of any C extensions.
</p>
</div>
</div>
</div>
<div id="outline-container-org22fe821" class="outline-3">
<h3 id="org22fe821"><span class="section-number-3">5.2</span> Performance</h3>
<div class="outline-text-3" id="text-5-2">
<p>
Translating between Ruby objects and native pointers requires updating
a global hash table, which is relatively expensive. We reduce that
cost by tagging the pointers for common types, so true, false, nil,
and so forth always convert to the same native value and never need to
touch the hash table. Likewise fixnums can be tagged to cover most of
their range, and we can probably use a similar technique for floating
point numbers.
</p>

<p>
We are also doing work to reduce the costs of our GC marking
technique. We can make an assumption that GC marking functions are
not used and avoid those costs as long as we can safely recover as
soon as markers are introduced, and we can reduce the cost of
preserving objects on the stack in many situations.
</p>

<p>
Having applied some of these techniques we have benchmarked repeated
calls to C extensions being up to 3 times faster than MRI 2.4 in some
cases, though not yet for all our tests. We'll continue to work on
performance in this area.
</p>
</div>
</div>
<div id="outline-container-org24e877c" class="outline-3">
<h3 id="org24e877c"><span class="section-number-3">5.3</span> What's next?</h3>
<div class="outline-text-3" id="text-5-3">
<p>
As I mentioned above we're still working on some changes to improve
compatibility even more, and we'll continue to benchmark and improve
performance. We expect any remaining problems to be related to
specific functions in the Ruby C APIs rather than being more
fundamental compatibility issues, and we'll be expanding our testing
of gems considerably in the near future to help find and resolve
these.
</p>
</div>
</div>
</div>
<div id="outline-container-org4b751db" class="outline-2">
<h2 id="org4b751db"><span class="section-number-2">6</span> Conclusion</h2>
<div class="outline-text-2" id="text-6">
<p>
These changes part of our new GraalVM release (release candidate
12). If you use or maintain a Ruby C extension then now is great time
to start testing it with TruffleRuby. Get in touch with us via <a href="https://twitter.com/truffleruby?lang=en">twitter</a>
or <a href="https://gitter.im/graalvm/truffleruby">Gitter</a>, or report issues via our <a href="https://github.com/oracle/truffleruby">GitHub repo</a>
</p>
</div>
</div>

    </div>
</section>
]]></description>
      <pubDate>2019-02-04</pubDate>
      <guid>https://aardvark179.github.io/blog/capi.html</guid>
    </item>
    <item>
      <title>Lightweight fibres for TruffleRuby.</title>
      <link>https://aardvark179.github.io/blog/fibers.html</link>
      <description><![CDATA[<section id="content" role="main">
    <div id="outline-container-sec-" class="row" style="padding-top: 70px">
        <div class="col-md-2"></div>
            <h1>Lightweight fibres for TruffleRuby.</h1>
            <div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#orgef1229c">1. Disclaimer</a></li>
<li><a href="#orge13e4b0">2. Lightweight threads, continuations, and Project Loom.</a>
<ul>
<li><a href="#orgbd69f7d">2.1. Lightweight user mode threads</a></li>
<li><a href="#orgcf1fa7b">2.2. Continuations</a></li>
<li><a href="#org9ad9b94">2.3. What makes these features more light weight than <code>java.lang.Thread</code>?</a></li>
</ul>
</li>
<li><a href="#orgd49a654">3. Fibres and continuations in Ruby</a>
<ul>
<li><a href="#org8408e6a">3.1. Fibres</a></li>
<li><a href="#orgdffc626">3.2. Continuations</a></li>
</ul>
</li>
<li><a href="#org2115eb0">4. The differences between these two models</a></li>
<li><a href="#org4c0ab73">5. Implementing Ruby's model using Loom's</a>
<ul>
<li><a href="#org02127b5">5.1. Implementing Ruby fibres</a>
<ul>
<li><a href="#orgd676819">5.1.1. With Loom's Fibres</a></li>
<li><a href="#orgb1ab4aa">5.1.2. With Loom's Continuations</a></li>
</ul>
</li>
<li><a href="#orgeae422f">5.2. Implementing <code>callcc</code></a></li>
</ul>
</li>
<li><a href="#org6e189d6">6. The results</a>
<ul>
<li><a href="#orgff01ad2">6.1. How many fibres can we support?</a></li>
<li><a href="#orgb92b0df">6.2. Scheduling between fibres</a></li>
</ul>
</li>
<li><a href="#orgd090810">7. Conclusion</a></li>
</ul>
</div>
</div>
<p>
The MRI Ruby implementation supports heavy weight threads which are
pre-emptively scheduled, and lightweight fibres which may run on those
threads and are co-operatively scheduled. Fibres have been a problem
for JVM Ruby implementations because the VM itself did not have a
concept of a fibre, or a lightweight abstraction we could build upon,
but that's starting to change with <a href="http://openjdk.java.net/projects/loom/">Project loom</a>, let's take a look at
what it gives, and how we can use that to implement fibres that can
scale beyond even MRI's.
</p>
<div id="outline-container-orgef1229c" class="outline-2">
<h2 id="orgef1229c"><span class="section-number-2">1</span> Disclaimer</h2>
<div class="outline-text-2" id="text-1">
<p>
Everything I"m talking about here is based on early prototypes. The
APIs and features may change radically before anything nears being
production, and there is no fixed timetable for when these feature
will arrive in a production JVM except, "When it's done." I
contributed the support for Graal to Loom, and implemented the
prototype we'll talk about here for TruffleRuby.
</p>
</div>
</div>
<div id="outline-container-orge13e4b0" class="outline-2">
<h2 id="orge13e4b0"><span class="section-number-2">2</span> Lightweight threads, continuations, and Project Loom.</h2>
<div class="outline-text-2" id="text-2">
<p>
Project Loom is an OpenJDK project with the goal
</p>
<blockquote>
<p>
To explore and incubate Java VM features and APIs built on top of them
for the implementation of lightweight user-mode threads (fibres),
delimited continuations (of some form), and related features, such as
explicit tail-call.
</p>
</blockquote>
<p>
That may sound complex, but let's look at it bit by bit and see what
it gives us in Java, and how we can use that in TruffleRuby.
</p>
</div>
<div id="outline-container-orgbd69f7d" class="outline-3">
<h3 id="orgbd69f7d"><span class="section-number-3">2.1</span> Lightweight user mode threads</h3>
<div class="outline-text-3" id="text-2-1">
<p>
These are currently called <code>java.lang.Fiber</code> in the project loom
prototype, and they provide a nice simple model. Fibres are started
with a scheduler, and whenever the has nothing to do it parks
itself. When the scheduler decided that that fibre has more work it
can do then it will schedule it next. The scheduler isn't special,
it's a standard <code>java.util.concurrent.Executor</code> and it can use
whatever scheduling strategy it likes, across as many threads as it
likes. If your fibres are mostly doing IO, or waiting for locks then
you probably won't even have to park them explicitly as the Java class
library will do that for you.
</p>
</div>
</div>
<div id="outline-container-orgcf1fa7b" class="outline-3">
<h3 id="orgcf1fa7b"><span class="section-number-3">2.2</span> Continuations</h3>
<div class="outline-text-3" id="text-2-2">
<p>
Continuations are a lower level concept upon which those lightweight
threads are built. A continuation can be run, and can yield control
back to the function which ran it. If it is run again then it will
continue from the point at which it yielded with the same call stack,
local variables, and so forth.
</p>
</div>
</div>
<div id="outline-container-org9ad9b94" class="outline-3">
<h3 id="org9ad9b94"><span class="section-number-3">2.3</span> What makes these features more light weight than <code>java.lang.Thread</code>?</h3>
<div class="outline-text-3" id="text-2-3">
<p>
A normal <code>java.lang.Thread</code> in Hotspot is a normal operating system
thread, it has a stack, and will be scheduled by the operating system,
and in the JVM it will include data structures for things like thread
local variables. Scheduling threads is heavyweight because it has to
happen via kernel space and because the kernel only has a very limited
idea of which thread it would be sensible to schedule
next. Lightweight user mode threads and continuations can involve much
less data since we only need to preserve the part of the stack between
the call to run the continuation and the point at which it yielded,
and can often be very effectively scheduled since the application may
often know precisely what to run next.
</p>
</div>
</div>
</div>
<div id="outline-container-orgd49a654" class="outline-2">
<h2 id="orgd49a654"><span class="section-number-2">3</span> Fibres and continuations in Ruby</h2>
<div class="outline-text-2" id="text-3">
<p>
Ruby has its own model of fibres and continuations, and they aren't
quite the same as Loom's
</p>
</div>
<div id="outline-container-org8408e6a" class="outline-3">
<h3 id="org8408e6a"><span class="section-number-3">3.1</span> Fibres</h3>
<div class="outline-text-3" id="text-3-1">
<p>
Ruby has the <code>Fiber</code> class. You create a <code>Fiber</code> with a block like this
</p>
<div class="org-src-container">
<pre class="src src-ruby">f = Fiber.new do |x|
  puts "Fiber called with #{x}"
end
</pre>
</div>
<p>
and call it like this
</p>
<div class="org-src-container">
<pre class="src src-ruby">f.resume(1)
</pre>
</div>
<p>
which should produce <code>Fiber called with 1</code> as output. Fibres can
explicitly transfer control to another fibre, or yield to the fibre
which transferred control to them.
</p>
</div>
</div>
<div id="outline-container-orgdffc626" class="outline-3">
<h3 id="orgdffc626"><span class="section-number-3">3.2</span> Continuations</h3>
<div class="outline-text-3" id="text-3-2">
<p>
Continuations in Ruby are created using the <code>Kernel#callcc</code>
method. This takes a block which will be executed immediately, and
create a continuation object which when called will resume execution
from the end of the block. They are quite hard to get your head round,
so an example is probably helpful. I've adapted this one from the Ruby
documentation on continuations.
</p>
<div class="org-src-container">
<pre class="src src-ruby">require "continuation"
def f
  arr = [ "Freddie", "Herbie", "Ron", "Max", "Ringo" ]
  cont = nil
  callcc do |cc| 
    puts 'Creating continuation'
    cont = cc
  end
  puts(message = arr.shift)
  cont.call unless message =~ /Max/
end
</pre>
</div>
<p>
If you run this Ruby method it will output the following
</p>
<pre class="example">
Creating continuation
Freddie
Herbie
Ron
Max

</pre>
<p>
What's going on here? Well the method creates an array <code>arr</code>, and a
variable <code>cont</code>. It then calls <code>callcc</code> which executes the block,
passing in the continuation object as an argument. The block prints
out that we're creating a continuation, and assigns it to <code>cont</code> so we
can use it later. We then remove the first element of <code>arr</code> and assign
it to <code>message</code> and output it. Finally we call the continuation again
unless <code>message</code> matches "Max", and this causes the latter half of the
method to be run again.
</p>

<p>
There's quite a lot of debate over whether continuations like this are
a good idea, and they certainly aren't widely used in production code,
but they are still part of the standard Ruby implementation and we do
support them in TruffleRuby.
</p>
</div>
</div>
</div>
<div id="outline-container-org2115eb0" class="outline-2">
<h2 id="org2115eb0"><span class="section-number-2">4</span> The differences between these two models</h2>
<div class="outline-text-2" id="text-4">
<p>
As you can see these models differ in a couple of important ways.
</p>

<p>
In the case of fibres Loom places the responsibility for scheduling on
an object outside of the fibres themselves, while Ruby allows each
fibre to either explicitly transfer control to another or yield to
whichever fibre transferred control to it.
</p>

<p>
Continuations are even more marked in their differences. Loom's behave
in many ways more like Ruby's fibres, with each call and yield
advancing the execution state.
</p>
</div>
</div>
<div id="outline-container-org4c0ab73" class="outline-2">
<h2 id="org4c0ab73"><span class="section-number-2">5</span> Implementing Ruby's model using Loom's</h2>
<div class="outline-text-2" id="text-5">
<p>
So given the quite different nature of these two models can we
implement Ruby's fibres and continuations using what Loom provides?
</p>
</div>
<div id="outline-container-org02127b5" class="outline-3">
<h3 id="org02127b5"><span class="section-number-3">5.1</span> Implementing Ruby fibres</h3>
<div class="outline-text-3" id="text-5-1">
<p>
Most of the code in TruffleRuby concerned with fibres is in a small
set of classes, mainly <code>FiberManager</code> which handles the creation of
fibres and how control is passed between them. This makes it quite
easy for us to prototype new implementations.
</p>
</div>
<div id="outline-container-orgd676819" class="outline-4">
<h4 id="orgd676819"><span class="section-number-4">5.1.1</span> With Loom's Fibres</h4>
<div class="outline-text-4" id="text-5-1-1">
<p>
Could we simply replace the use of threads in <code>FiberManager</code> with
fibres? Well, we'd need a scheduler that could cope with both the
explicit yielding from a Ruby <code>Fiber</code> and other parts of the Java
class library park fibres for their own reasons. This is certainly an
approach worth investigating, but Loom's fibre API is one area most
likely to change, so we'll leave that attempt till later.
</p>
</div>
</div>
<div id="outline-container-orgb1ab4aa" class="outline-4">
<h4 id="orgb1ab4aa"><span class="section-number-4">5.1.2</span> With Loom's Continuations</h4>
<div class="outline-text-4" id="text-5-1-2">
<p>
Loom's continuations seem like a much better fit. The <code>FiberManager</code>
can represent Ruby <code>Fiber</code> with a <code>java.lang.Continution</code>, and
orchestrate the transfer of control between them. Execution can
process roughly as follows:
</p>
</div>
<ol class="org-ol">
<li><a id="orgaea653b"></a>Creating a <code>Fiber</code><br />
<div class="outline-text-5" id="text-5-1-2-1">
<p>
This should create a <code>java.lang.Continuation</code> which will do the following:
</p>
<ol class="org-ol">
<li>Fetch any arguments given to the fibre.</li>
<li>Set itself as the current fibre.</li>
<li>Run the provided code block.</li>
<li>Clean up and tell the <code>FiberManager</code> what should be run next.</li>
</ol>
<p>
The fibre won't be run immediately, that can happen when
<code>Fiber#resume</code> is called.
</p>
</div>
</li>
<li><a id="orge6c65d6"></a>Transferring control from the thread into a <code>Fiber</code><br />
<div class="outline-text-5" id="text-5-1-2-2">
<p>
The <code>FiberManager</code> needs to set any arguments to the fibre, and then
run the continuation representing it. When control returns back to the
<code>FiberManager</code> it should check what to schedule next.
</p>
</div>
</li>
<li><a id="org83761ea"></a>Transferring control from one <code>FIber</code> to another.<br />
<div class="outline-text-5" id="text-5-1-2-3">
<p>
The running fibre should set the next fibre to be run on the
<code>FiberManager</code>, set any arguments for it, and then yield. Control will
pass back to the <code>FiberManager</code> which should run the continuation for
the scheduled fibre. That fibre will then set itself as the current
fibre on the <code>FiberManager</code>, and get any arguments.
</p>
</div>
</li>
<li><a id="orgb5779f8"></a>Transferring control from a <code>Fiber</code> to the thread.<br />
<div class="outline-text-5" id="text-5-1-2-4">
<p>
This is much like the above case, except the <code>FiberManager</code> will not
run another continuation, instead it will simply return the results to
its original caller.
</p>
</div>
</li>
</ol>
</div>
</div>
<div id="outline-container-orgeae422f" class="outline-3">
<h3 id="orgeae422f"><span class="section-number-3">5.2</span> Implementing <code>callcc</code></h3>
<div class="outline-text-3" id="text-5-2">
<p>
<code>callcc</code> is harder to map to Loom's fibres or continuations, at least
at the moment. If the continuations can be cloned however then it
could be done roughly as follows. Each heavyweight thread would need
to be run almost entirely inside a <code>java.lang.Continuation</code>, and the
control flow would go like this:
</p>
<ol class="org-ol">
<li>Creating the continuation.
<ol class="org-ol">
<li>Create a Ruby <code>Continuation</code> object.</li>
<li>Execute the block passed to <code>callcc</code> passing the Ruby continuation in.</li>
<li>Place the Ruby continuation somewhere the thread can access it.</li>
<li>Yield control to the thread.</li>
<li>The thread clones the main <code>java.lang.Continuation</code> and sets it
on the Ruby continuation object.</li>
<li>The thread returns control to the original <code>java.lang.Continuation</code>.</li>
</ol></li>
<li>Calling the continuation.
<ol class="org-ol">
<li>Place the Ruby continuation somewhere the thread can access it.</li>
<li>Yield control to the thread.</li>
<li>The thread clones the <code>java.lang.Continuation</code> on the Ruby
continuation.</li>
<li>The thread returns control to this new cloned continuation.</li>
</ol></li>
</ol>
</div>
</div>
</div>
<div id="outline-container-org6e189d6" class="outline-2">
<h2 id="org6e189d6"><span class="section-number-2">6</span> The results</h2>
<div class="outline-text-2" id="text-6">
<p>
So, having implemented a prototype for TruffleRuby the question is,
how well does it work? The tests we can run at the moment are a little
limited as the Loom prototype has some limitations, but it's enough to
test a few things.
</p>
</div>
<div id="outline-container-orgff01ad2" class="outline-3">
<h3 id="orgff01ad2"><span class="section-number-3">6.1</span> How many fibres can we support?</h3>
<div class="outline-text-3" id="text-6-1">
<p>
It's very easy to support a large number of fibres if they are simply
initialised but never used, so we want a test that will create a large
set of fibres which are active. Something like this should do.
</p>
<div class="org-src-container">
<pre class="src src-ruby">def test_fiber_1(t)
  Fiber.new do |x|
    if x &gt; t
      Fiber.yield x
    else
      f = test_fiber_1(t)
      f.transfer( x + 1 )
    end
  end
end
</pre>
</div>
<p>
Using this code I ran some tests to see how many fibres could be
supported by different implementations. This was done by making an
initial guess and then performing a binary search to find the
approximate number of fibres that could be supported without raising
an error, or causing the VM to crash completely. MRI 2.5 has made
changes to its fibre code and can now support a larger number of
fibres. I'll update these results with figures for MRI 2.5 soon.
</p>
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="org-left" />

<col  class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Ruby implementation</th>
<th scope="col" class="org-right">Fibres</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">MRI 2.4</td>
<td class="org-right">30000</td>
</tr>

<tr>
<td class="org-left">JRuby 9.1.11.0</td>
<td class="org-right">3000</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (threads)</td>
<td class="org-right">3000</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (continuations)</td>
<td class="org-right">1000000</td>
</tr>
</tbody>
</table>
<p>
These results are very encouraging. The only real limit to the number
of fibres we can support appears to be memory.
</p>

<p>
Testing the performance of this is not easy with JRuby or TruffleRuby
using threads as neither can clean up threads fast enough to run the
test multiple times with large numbers of threads. We can however
compare TruffleRuby with continuations to MRI 2.4. The test was run
with 10000 fibres 20 times fibres to allow for warm up, and then
another 20 times to collect timings.
</p>
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="org-left" />

<col  class="org-right" />

<col  class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Ruby implementation</th>
<th scope="col" class="org-right">Average time (s)</th>
<th scope="col" class="org-right">Standard deviation</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">MRI 2.4</td>
<td class="org-right">0.49</td>
<td class="org-right">0.032</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (continuations)</td>
<td class="org-right">0.20</td>
<td class="org-right">0.064</td>
</tr>
</tbody>
</table>
</div>
</div>
<div id="outline-container-orgb92b0df" class="outline-3">
<h3 id="orgb92b0df"><span class="section-number-3">6.2</span> Scheduling between fibres</h3>
<div class="outline-text-3" id="text-6-2">
<p>
Another thing to test is how fast it is to transfer control from one
fibre to another. We'll use a very small test which will be completely
dominated by the transfer time. Here is the source for this test.
</p>
<div class="org-src-container">
<pre class="src src-ruby">def create_fibers(t)
  fibers = []
  0...t.times do
    fibers &lt;&lt; Fiber.new do |x|
      while (true)
        f = fibers[rand(t)]
        Fiber.yield(f)
      end
    end
  end
  fibers
end

def schedule_fibers(t, n)
  fibers = create_fibers(t)
  next_fiber = fibers.first
  0...(n * t).times do
    next_fiber = next_fiber.resume
  end
end
</pre>
</div>
<p>
This will create <code>t</code> fibres and transfer control among them <code>t * n</code>
times. I had hoped to test this with large numbers of fibres, but
again encountered issues with running the test using JRuby or
TruffleRuby with threads, so tested with <code>t = 100</code> and <code>n = 1000</code>
</p>
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="org-left" />

<col  class="org-right" />

<col  class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Ruby implementation</th>
<th scope="col" class="org-right">Average time (s)</th>
<th scope="col" class="org-right">Standard deviation</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">MRI 2.4</td>
<td class="org-right">0.26</td>
<td class="org-right">0.010</td>
</tr>

<tr>
<td class="org-left">JRuby 9.1.11.0</td>
<td class="org-right">1.26</td>
<td class="org-right">0.046</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (threads)</td>
<td class="org-right">1.32</td>
<td class="org-right">0.082</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (continuations)</td>
<td class="org-right">2.01</td>
<td class="org-right">0.31</td>
</tr>
</tbody>
</table>
<p>
As you can see, MRI beats all the other implementations easily, and
Loom's continuations still have a lot of work to be done on them
performance wise. Some experiments with varying the values of <code>t</code> and
<code>n</code> suggests that MRI's time is mostly spent creating the fibres, and
it's very quick to transfer control between them. All other
implementations are dominated by the time to transfer control between
fibres and have run times mostly unaffected by the total number of
fibres created (at least within the range I could test).
</p>
</div>
</div>
</div>
<div id="outline-container-orgd090810" class="outline-2">
<h2 id="orgd090810"><span class="section-number-2">7</span> Conclusion</h2>
<div class="outline-text-2" id="text-7">
<p>
Project Loom allows TruffleRuby to support a large number of fibres in
a lightweight way, There's still a lot of performance work to do, but
these techniques look promising could also be used by JRuby and other
JVM languages to support fibres, continuations, and other lightweight
concurrency models.
</p>
</div>
</div>

    </div>
</section>
]]></description>
      <pubDate>2018-08-02</pubDate>
      <guid>https://aardvark179.github.io/blog/fibers.html</guid>
    </item>
    <item>
      <title>Making special variables less special in TruffleRuby</title>
      <link>https://aardvark179.github.io/blog/special_variables.html</link>
      <description><![CDATA[<section id="content" role="main">
    <div id="outline-container-sec-" class="row" style="padding-top: 70px">
        <div class="col-md-2"></div>
            <h1>Making special variables less special in TruffleRuby</h1>
            <div id="outline-container-orgfc5003b" class="outline-2">
<h2 id="orgfc5003b"><span class="section-number-2">1</span> Making special variables less special in TruffleRuby</h2>
<div class="outline-text-2" id="text-1">
<p>
Ruby has quite a large set of <a href="http://ruby-doc.org/docs/ruby-doc-bundle/Manual/man-1.4/variable.html#variables">pre-defined global variables</a> which are
special in a variety of ways. Some are read only, some are only
defined when others are non-nil, some are local to a thread, and two
are really special. Understanding exactly how they behave is hard
because you can't normally look in just one place. In MRI for example
you may be able to find where the variables are defined in the C code,
but you'll also need to trace through many functions to track
down everything that happens, and you may find there is some special
code in the parser or compiler that changes that. In TruffleRuby we
like to implement as much as we can in Ruby, so let's see if we can do
that here and make it easier for everyone to understand how these
variables behave.
</p>

<p>
This sort of thing has been <a href="https://github.com/0x0dea/viva">done before</a> as a fun demo, but not for
actually implementing Ruby itself, even in implementations that try to
write as much of their standard library in Ruby as they can.
</p>
</div>
<div id="outline-container-org8b70bb3" class="outline-3">
<h3 id="org8b70bb3"><span class="section-number-3">1.1</span> Hooked variables</h3>
<div class="outline-text-3" id="text-1-1">
<p>
The most important thing we're going to need is a way to define
variables with custom behaviour from Ruby. There's a function in MRI's
C API which allows this, <code>rb_define_hooked_variable</code>, and several
macros and other variants that use that. Since we support those
function we already had the capability, but we do need to expose it in
a nice way. We don't want to add methods to existing classes that
might cause clashes so we tend to put things like this into modules
under the <code>Truffle</code> name space. We are also going to want to define a
lot of read only variables, so it's probably a good idea to make a
method for that as well. In TruffleRuby you'll find these methods in
<code>Truffle::KernelOperations</code>, but for brevity we'll just put them under
<code>Truffle</code> here.
</p>
<div class="org-src-container">
<pre class="src src-ruby">module Truffle
  def self.define_hooked_variable(name, getter, setter, defined = proc { 'global-variable' })
    # Work is done in here.
  end

  def self.define_read_only_global(name, getter)
    setter = -&gt; _ { raise NameError, "#{name} is a read-only variable." }
    define_hooked_variable(name, getter, setter)
  end
end
</pre>
</div>
<p>
This method is going to take the name of a variable to be defined,
procedures for getting and setting the value, and an optional
procedure to be used for <code>defined? $my_variable</code>. If you've never used
<code>defined?</code> in Ruby it's a little special. It returns the semantic
meaning of the expression that follows it. Simple expressions will
just return <code>'expression'</code>, method names will return <code>'method'</code>, and
so on. If you try <code>defined? $var</code> then you will get
<code>'global-variable'</code> if <code>$var</code> has been assigned to or <code>nil</code> if it
hasn't. Some of Ruby's special variables have more complex behaviour
so we need to be able to provide a procedure for that.
</p>
</div>
</div>
<div id="outline-container-org57e431c" class="outline-3">
<h3 id="org57e431c"><span class="section-number-3">1.2</span> Trivial example</h3>
<div class="outline-text-3" id="text-1-2">
<p>
Let's see this in action by defining our own hooked variable.
</p>
<div class="org-src-container">
<pre class="src src-ruby">x = nil
Truffle.define_hooked_variable(
  :$my_var,
  -&gt; { x },
  -&gt; v { puts "Setting $my_var to #{v}.",
         x = v })
</pre>
</div>
<p>
Now if you try doing <code>$my_var = "something"</code> you'll see a message
saying <code>Setting $my_var to something.</code> You should also be able to get
back the value you stored by doing <code>$my_var</code>. Now we know this works
let's see if we can define some of the simple special variables.
</p>
</div>
</div>
<div id="outline-container-orgfe6beb3" class="outline-3">
<h3 id="orgfe6beb3"><span class="section-number-3">1.3</span> Variables related to ARGF</h3>
<div class="outline-text-3" id="text-1-3">
<p>
Several special variables are connected to <code>ARGF</code>, they link to
properties on that object but can't be written to themselves.
</p>

<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_read_only_global :$&lt;, -&gt; { ARGF }
Truffle.define_read_only_global :$FILENAME, -&gt; { ARGF.filename }
</pre>
</div>

<p>
There's also <code>$*</code> which holds the arguments not consumed by the Ruby implementation itself.
</p>

<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_read_only_global :$*, -&gt; { ARGV }
</pre>
</div>

<p>
Finally we'll look at <code>$.</code>. This is set by various methods on <code>ARGF</code>
and file objects, but it's not actually <code>ARGF.lineno</code> since updating
it doesn't actually change that value. Instead we hold it on another
instance variable on <code>ARGF</code> like this:
</p>

<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_hooked_variable(
  :$.,
  -&gt; { ARGF.instance_variable_get(:@last_lineno) },
  -&gt; value { value = Truffle::Type.coerce_to value, Fixnum, :to_int
             ARGF.instance_variable_set(:@last_lineno, value) } )
</pre>
</div>
</div>
</div>

<div id="outline-container-org6b84598" class="outline-3">
<h3 id="org6b84598"><span class="section-number-3">1.4</span> Other simple cases</h3>
<div class="outline-text-3" id="text-1-4">
<p>
Quite a few variables allow writes to them, but include some extra
checks. At first glance, it appears we could simply represent these
constraints with a lambda. While this is a nice, clear solution in
Ruby, it unfortunately complicates parts of the TruffleRuby runtime
written in Java. To help keep things simple for both the Ruby and Java
parts of the runtime, we've added <code>Truffle.global_variable_get</code> and
<code>Truffle.global_variable_set</code>. and we can then use them like this:
</p>

<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_hooked_variable(
  :$stdout,
  -&gt; { Truffle.global_variable_get(:$stdout) },
  -&gt; v { raise TypeError, "$stdout must have a write method #{v.class} given." unless v.respond_to?(:write)
         Truffle.global_variable_set(:$stdout, v) })

alias $&gt; $stdout
</pre>
</div>

<p>
There's a few more like this, and I won't go through them all, but
they can all be done as nice simple Ruby.
</p>
</div>
</div>
<div id="outline-container-org28a2c66" class="outline-3">
<h3 id="org28a2c66"><span class="section-number-3">1.5</span> But will it optimise?</h3>
<div class="outline-text-3" id="text-1-5">
<p>
All the variables I've mentioned so far have a few things in
common. They have relatively simple semantics, and they aren't used
that that often, or aren't likely to be a real performance
bottleneck. But later on we're going to look at some that are used
much more heavily and are more complex to implement, so let's talk
about what will optimise now.
</p>
</div>
<div id="outline-container-orgac136d0" class="outline-4">
<h4 id="orgac136d0"><span class="section-number-4">1.5.1</span> A normal global</h4>
<div class="outline-text-4" id="text-1-5-1">
<p>
What happens when we run a simple statement like <code>$foo</code> in
TruffleRuby? Well, that statement gets parsed into an AST (an Abstract
Syntax Tree). In this case the only node we need to think about in the
tree is a <code>ReadGlobalVariableNode</code>. When it is run it will look up the
storage for that variable and return the result. If it were used
inside a loop then it would only lookup the variable storage the first
time it was executed; subsequent executions would just return the
value from the storage. That should be retty fast, right?
</p>
</div>
</div>
<div id="outline-container-org28cddc9" class="outline-4">
<h4 id="org28cddc9"><span class="section-number-4">1.5.2</span> Optimising for constant values</h4>
<div class="outline-text-4" id="text-1-5-2">
<p>
Most global variables won't change their value, and we'd like be able
to assume those values really are constant if we can. So the storage
for each global includes a couple of extra bits of information. We
keep track of the number of times a global has had its value changed,
and we keep an <code>Assumption</code> to represent the value being
constant. When code is compiled with a JIT (just in time) compiler
assumptions are often used to track speculative optimisations, and
marking an assumption as invalidated will cause the JIT to invalidate
the compiled code. So, how do we use this for global variables?
</p>
</div>
</div>
<div id="outline-container-org43aab3a" class="outline-4">
<h4 id="org43aab3a"><span class="section-number-4">1.5.3</span> Specialising</h4>
<div class="outline-text-4" id="text-1-5-3">
<p>
<code>ReadGlobalVariableNode</code> is slightly more complex than I let on. It
actually has two specialisations which can be used.
</p>
<div class="org-src-container">
<pre class="src src-java">@Specialization(assumptions = "storage.getUnchangedAssumption()")
public Object readConstant(
        @Cached("storage.getValue()") Object value) {
    return value;
}

@Specialization
public Object read() {
    return storage.getValue();
}
</pre>
</div>
<p>
What this says is that if the assumption is true then we can cache the
value of the global, and return constant value without reading it from
storage every time. The JIT understands that the cached value is
constant, so can exploit that fact when making other optimisations. If
the variable is written to then that `Assumption` will be invalidated
and we'll fall back to getting the value from storage every time.
</p>
</div>
</div>
<div id="outline-container-org2b08509" class="outline-4">
<h4 id="org2b08509"><span class="section-number-4">1.5.4</span> But what about those hooked variables we just defined?</h4>
<div class="outline-text-4" id="text-1-5-4">
<p>
Once again <code>ReadGlobalVariableNode</code> is slightly more complex
than I let on. It also has cases for global variables with hooked
storage. It's not too bad though, because the hooks for a variable
must be constant, so we only really need to worry about how fast those
lambdas will run. Let's consider the lambda we defined
</p>
<div class="org-src-container">
<pre class="src src-ruby">-&gt; { Truffle.global_variable_get :$stdout }
</pre>
</div>
<p>
The <code>global_variable_get</code> method is defined in our Java runtime, and
it has two specialisations. Let's take a look at the first one.
</p>
<div class="org-src-container">
<pre class="src src-java">@Specialization(guards = "name == cachedName")
public Object read(DynamicObject name,
        @Cached("name") DynamicObject cachedName,
        @Cached("createReadNode(name)") ReadSimpleGlobalVariableNode readNode) {
    return readNode.execute();
}
</pre>
</div>
<p>
The first time the method is called we'll keep a reference to the name
of the variable we wanted to get, and we'll create a node to read the
value &mdash; it's a simple version of the node for reading globals that
doesn't care about any hooks. So as long as the symbol stays constant
all it will do is execute the read node. As long as the stored value
remains constant the read node will just return the cached value, and
the JIT can optimise away all the apparent extra work.
</p>
</div>
</div>
<div id="outline-container-org1464d4c" class="outline-4">
<h4 id="org1464d4c"><span class="section-number-4">1.5.5</span> Not so constant</h4>
<div class="outline-text-4" id="text-1-5-5">
<p>
All that would be great if we only had that single lambda that did
</p>
<div class="org-src-container">
<pre class="src src-ruby">-&gt; { Truffle.global_variable_get :$stdout }
</pre>
</div>
<p>
but we've also got
</p>
<div class="org-src-container">
<pre class="src src-ruby">-&gt; { Truffle.global_variable_get :$stderr }
</pre>
</div>
<p>
and many others, so that symbol won't be constant any more, will it?
Luckily we have another tool we can use to help with that
problem: we can use a fresh copy of the <code>global_variable_get</code> method
everywhere it is used in the source. As long as the symbol is constant
at each of these call sites things should still work nicely.
</p>
</div>
</div>
</div>
<div id="outline-container-org2456e9b" class="outline-3">
<h3 id="org2456e9b"><span class="section-number-3">1.6</span> There's special, and then there's special</h3>
<div class="outline-text-3" id="text-1-6">
<p>
Next up the difficulty ladder are variables which are local to a
thread. To implement <code>$SAFE</code> we'll need a way to return the value for
the current thread when it is read and written, as well as checking
any new value is valid. This value must not be visible in the normal
fiber local variables accessed using <code>Thread#[]</code> or the thread locals
accessed from <code>Thread#thread_variable_get</code>, so we'll need something on
<code>Truffle::ThreadOperations</code> to do that job.
</p>
<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_hooked_variable(
  :$SAFE,
  -&gt; { Truffle::ThreadOperations.get_thread_local(:$SAFE) },
  -&gt; value { value = Truffle::Type.check_safe_level(value)
             Truffle::ThreadOperations.set_thread_local(:$SAFE, value) }
)
</pre>
</div>

<p>
The only new thing we have here is the ability to get or set a value on
the current thread. You might assume those methods have to be written in Java,
but they're written in Ruby as well. The get method looks something like
</p>
<div class="org-src-container">
<pre class="src src-ruby">def self.get_thread_local(key)
  locals = thread_get_locals(Thread.current)
  object_ivar_get(locals, key)
end
</pre>
</div>
<p>
The values local to a thread are stored as a normal object with
instance variables, and we could have used
<code>Kernel#instance_variable_get</code> on locals, except <code>:$SAFE</code> isn't a
valid name for an instance variable in Ruby.
</p>

<p>
Everything here can be optimised in the same way I described
above. Accessing instance variables is extremely fast as long as the
owning object always has the same set of variables, and so as long as
the key stays constant it will just be a field access in an
object. <code>Thread.current</code> will be constant if you only use a single
thread, and getting the thread locals is just like getting an instance
variable. In reality you'll probably be using more than one thread,
but it should still optimise well if the method is copied for each
call site.
</p>

<p>
There's only a few other thread local variables, <code>$!</code> which holds the
last raised exception, <code>$?</code> which holds the return code of the last
child process, and <code>$@</code> which is just an alias for <code>$!.backtrace</code>. The
remaining ones I want to talk about are all connected with regular
expressions, and they are even more complex and subtle.
</p>
</div>
</div>
<div id="outline-container-orgb74db3a" class="outline-3">
<h3 id="orgb74db3a"><span class="section-number-3">1.7</span> &#x2026;and then there's really special</h3>
<div class="outline-text-3" id="text-1-7">
<p>
<code>$~</code> is more complex than you might realise. It holds the value of the
last regular expression match done in a variety of ways, and hence is
thread local. But more than that it is also frame local. What do I
mean by that? Well, try this code in <code>irb</code> and see what you get.
</p>
<div class="org-src-container">
<pre class="src src-ruby">def a(str)
  /foo/ =~ str
  $~
end

def b(str)
  a(str)
  $~
end

a("There is a foo in this string")
b("There is a foo in this string")
</pre>
</div>
<p>
The call to <code>a</code> will return a <code>MatchData</code> object, but the call to <code>b</code>
will return <code>nil</code>. Even setting <code>$~</code> in <code>a</code> won't affect the value we
see in <code>b</code>. It's pretty useful because no library call you make can
unexpectedly change the value of <code>$~</code> that you might be relying on, but
it is going to make our job implementing it harder.
</p>
</div>
<div id="outline-container-org54111fd" class="outline-4">
<h4 id="org54111fd"><span class="section-number-4">1.7.1</span> Getting and setting the last match</h4>
<div class="outline-text-4" id="text-1-7-1">
<p>
In our core library we need a way to reach up to the caller and set
the value of <code>$~</code> it sees in this thread, and we'll need to do
something similar for the variable hooks. What might a method for
accessing <code>$~</code> in a frame look like? Well we already have a way to
represent a frame in Ruby, <code>Binding</code>!
</p>
<div class="org-src-container">
<pre class="src src-ruby">module Truffle
  module RegexpOperations
    def self.last_match(a_binding)
      Truffle.frame_local_variable_get(:$~, a_binding)
    end
  end
end
</pre>
</div>
<p>
<code>frame_local_variable_get</code> will access a hidden local variable in the
binding, and then pull out the thread local value stored in
there. That thread local storage is implemented in Java, and optimised
for the common case that it will only hold a value for one
thread.. The same kind of specialisations we're described above hold
true however for all these parts.
</p>

<p>
The variable we want (<code>$~</code>) is constant, accessing a variable in
<code>a_binding</code> can be optimized just like access to an instance variable
on an object, so the hard part is going to be ensuring that
<code>a_binding</code> always come from the same method or block. How can we
arrange that, and how can we pass a binding into a variable hook?
</p>

<p>
Well, we'll change how we handle variable hooks a
little. <code>ReadGlobalVariableNode</code> actually has two specialisations for
calling a hook, based on the arity of the hook procedure. If it
requires an argument then we'll pass in the binding where it has been
called, and we'll do something similar for write hooks. We'll also
mark the check when declaring the variable, and tell the runtime to
split the hooks for each call site if they take a binding.
</p>
</div>
</div>
<div id="outline-container-org870718d" class="outline-4">
<h4 id="org870718d"><span class="section-number-4">1.7.2</span> Defining <code>$~</code> and setting the last match</h4>
<div class="outline-text-4" id="text-1-7-2">
<p>
With that in place <code>$~</code> can simply be defined as
</p>
<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_hooked_variable(
  :$~,
  -&gt; b { Truffle::RegexpOperations.last_match(b) },
  -&gt; v, b { Truffle::RegexpOperations.set_last_match(v, b) })
</pre>
</div>
<p>
The core library will need to set <code>$~</code> in callers, and it can do this
with <code>set_last_match</code>. It needs to get the caller's binding but we
already have a mechanism to do that (it's how we implement
<code>Kernel#binding</code>) and it needs to optimise so we spot when it is
happening and automatically mark methods to be split.
</p>
</div>
</div>
<div id="outline-container-org1dc7963" class="outline-4">
<h4 id="org1dc7963"><span class="section-number-4">1.7.3</span> The other regexp variables</h4>
<div class="outline-text-4" id="text-1-7-3">
<p>
Most of the other variables connected with regular expressions are
fairly simple. If the last match is not set then they will be <code>nil</code>,
and are not defined if you do <code>defined? $var</code>. Luckily this is quite
easy to represent using our <code>define_hooked_variable</code> method. For
example <code>$&amp;</code> is simply.
</p>
<div class="org-src-container">
<pre class="src src-ruby">Truffle.define_hooked_variable(
  :$&amp;,
  -&gt; b { match = Truffle::RegexpOperations.last_match(b)
         match[0] if match },
  -&gt; { raise SyntaxError, "Can't set variable $&amp;"},
  -&gt; b { 'global-variable' if Truffle::RegexpOperations.last_match(b) })
</pre>
</div>
<p>
Notice that we raise a <code>SyntaxError</code> when trying to set this variable
rather than the <code>NameError</code> other variables raise. It's just one of
the things that makes these variables extra special!
</p>
</div>
</div>
</div>
<div id="outline-container-orgdafd4d5" class="outline-3">
<h3 id="orgdafd4d5"><span class="section-number-3">1.8</span> Testing performance</h3>
<div class="outline-text-3" id="text-1-8">
<p>
Let's check global variable reads and hooked variable reads are still
good and fast. If you're wondering why I'm not testing writes it's
because they must introduce a full memory fence so the result can be
seen by other threads (see the <a href="https://docs.google.com/document/d/1pVzU8w_QF44YzUCCab990Q_WZOdhpKolCIHaiXG-sPw/edit#heading=h.bkpwfrblzkh">global variables section in the
proposed Ruby memory model</a> for details), and that really
dominates. Let's try a simple benchmark like
</p>
<div class="org-src-container">
<pre class="src src-ruby">$var = 1
def simple_count
  total = 0
  10000.times do
    total += $var
  end
  total
end
</pre>
</div>
<p>
We'll run the benchmark on MRI, JRuby, and TruffleRubby, and we'll
also run it on TruffleRuby with <code>$var</code> defined as a hooked
variable. We do see some noise in these benchmarks and it take a few
seconds for TruffleRuby and JRuby's JITs to kick in, so I allow the
benchmarks to run for a few seconds and then took the average
iterations per second of this peak performance. All numbers have been
rounded to two siginificant figures.
</p>
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="org-left" />

<col  class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Implementation</th>
<th scope="col" class="org-right">IPS</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">MRI</td>
<td class="org-right">2100</td>
</tr>

<tr>
<td class="org-left">JRuby</td>
<td class="org-right">2400</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (normal)</td>
<td class="org-right">3400000</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (hooked)</td>
<td class="org-right">3400000</td>
</tr>
</tbody>
</table>
<p>
What does this really tell us? Well, it tells us that we've worked out
<code>$var</code> is constant and we can still successfully do that when it's a
hooked variable, and maybe that has allowed the JIT to get really
aggressive with our test. Let's try making <code>$var</code> less constant and
see what happens.
</p>
<div class="org-src-container">
<pre class="src src-ruby">$r = Random.new

def simple_count
  $var = $r.rand(8)
  total = 0
  10000.times do
    total += $var
  end
  total
end
</pre>
</div>
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">


<colgroup>
<col  class="org-left" />

<col  class="org-right" />
</colgroup>
<thead>
<tr>
<th scope="col" class="org-left">Implementation</th>
<th scope="col" class="org-right">IPS</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-left">MRI</td>
<td class="org-right">2100</td>
</tr>

<tr>
<td class="org-left">JRuby</td>
<td class="org-right">2400</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (normal)</td>
<td class="org-right">68000</td>
</tr>

<tr>
<td class="org-left">TruffleRuby (hooked)</td>
<td class="org-right">19000</td>
</tr>
</tbody>
</table>
<p>
So we ar seeing some slowdown, but we're still faster than other
implementations. The slowdown we see is quite sensitive to the precise
benchmark design. Some showed very little slowdown while this case has
is 3 times slower with hooked variables.
</p>
</div>
</div>
<div id="outline-container-org119c122" class="outline-3">
<h3 id="org119c122"><span class="section-number-3">1.9</span> What's left?</h3>
<div class="outline-text-3" id="text-1-9">
<p>
After this work there's only two special bits of variable support left
in our parser. We still look for <code>$1...$N</code> for accessing captured
group in <code>$~</code>. They would be trivial to implement in Ruby, but how
high is N? If we want to be exactly like MRI then there should be as
many variables as there are capture groups in a the regexp last match,
but only the first nine will be listed by
<code>Kernel#global_variables</code>. We might handle this by introducing a
<code>variable_missing</code> method that would be called if the global variable
storage has not already been declared, this could then create hooked
variables for captured group variables and normal storage for anything
else.
</p>

<p>
The other special handling we still have is for named captures. If you
use <code>=~</code> on a regexp literal, and it has named capture groups, then
the equivalently named local variables will be set to the capture
groups. We could write most of that in Ruby, but we'd still need to
check for named captures in the parser, and making sure it optimised
well would probably require some extra work that we haven't done yet.
</p>

<p>
Since we saw some slow down from hooked variables in performance
testing we may want to look more deeply into that and see if it can be
reduced or eliminated, and we migth look at rewriting the storage for
<code>$~</code> in Ruby as well.
</p>
</div>
</div>
<div id="outline-container-orgfd1da4f" class="outline-3">
<h3 id="orgfd1da4f"><span class="section-number-3">1.10</span> Conclusion</h3>
<div class="outline-text-3" id="text-1-10">
<p>
TruffleRuby lets us implement more of Ruby in Ruby itself while still
allowing aggressive optimisation to be done. This can help make our
runtime smaller and hopefully make it easier for the community to
understand and contribute to our implementation.
</p>
</div>
</div>
</div>

    </div>
</section>
]]></description>
      <pubDate>2018-07-28</pubDate>
      <guid>https://aardvark179.github.io/blog/special_variables.html</guid>
    </item>
  </channel>
</rss>