Planet Ruby

module N
  def go
    puts "N#go"
    super
  end
end

class B
  def go
    puts "B#go"
  end
end

class A < B
  include N
end

A.new.go

Will print:

N#go
B#go

This is HIGHLY useful implementing rails plugins, where normally you’d use alias_method_chain to change a method directly inside ActiveRecord::Base. Instead, simply call super in the method that provides the new functionality, and when your module is included in your module class (which is a subclass of ActiveRecord::Base), and the main, ActiveRecord::Base implementation will be called.

NOTE: This trick only works if the method you wish to wrap is located in a superclass of the class you have defined the module in. IE, if N were included in B directly rather than A, N#go would never be called.

http://svn.apache.org/viewvc/webservices/xmlrpc/trunk/server/src/main/java/org/apache/xmlrpc/server/RequestProcessorFactoryFactory.java?view=markup

just look at that url for a second... (if it doesn't go off screen, that is)

the domain... sure. whatever.
viewvc. yeah. k.
webservices/xmlrpc/trunk/server/src/main/java -- damn... we haven't even hit code yet!
org/apache/xmlrpc/server -- WE STILL haven't hit code yet!?!

and then... the kicker:

wtf? ... wow. check it:

public interface RequestProcessorFactoryFactory { ... }

this thing declares an interface for a factory that produces other factories that produce things that process requests. processing requests... request processing. factory of factory of... I feel like the lawyer in Idiocracy. duuuh... errr. ummm.

How many freakin' levels of indirection does this world need? It's not even a class.

kill. me. now.

(Via Labnotes » Rounded Corners - 196.)

Rails 2.1 is not far off the horizon and we’ve been adding a ton of extra deliciously nice goodies in preparation of its release lately. As always, the good Ryan Daigle has been keeping a watchful eye on the changelog and has been documenting some of the new features. The latest stars are:

• Gem Dependencies
• Dirty tracking with partial updates
• has_finder in the form of named_scope
• Built-in timezone support
• Better caching infrastructure

The Rubyspecs, an extensive set of conformance tests for Ruby platform, is a great tool to enforce the compatibility of JRuby (and Rubinius, and IronRuby). But the specs are currently being mostly run on Linux/BSD/MacOS (against Ruby, JRuby, Rubinius), and are rarely (if ever) executed in Windows environment. Until now, that is.

I tried to run them on Windows some time ago and the mspec, the testing framework that powers Rubyspecs, had some issues with Windows paths. Since then, the issues have been resolved and the Rubyspecs are functional on Windows.

I’ve run them against JRuby, found 4-5 bugs right away (crashes in calls that are not supported on Windows even in Matz Ruby), fixed those, and was able to run the specs to the completion! There are couple of hang-ups here and there, and lots of failures due to fact that rubyspecs were never used on Windows and don’t expect Windows-specific behavior.

I guess, that’s the next big thing, to go over all the specs and adjust them to take Windows specifics into account when needed.

Watch out, Railsters. The next big thing is going to be Cobol on Cogs.

In my previous post, I asked for examples of great Ruby code. I got emails, comments, instant messages, and twitter messages with suggestions. I’d still like more.

But one really cool suggestion came from an idea Bil Kleb and Peter Provost had. They proposed a Hot or Not -like site for code.

I thought that would be fun, so Bruce and I whipped something up in a couple of hours over the weekend: Slick or Slack?.

It’s not done yet, but we deployed it anyway. My experience helping Ze Frank with Color Wars 2008 has convinced me that deploying things that aren’t done is more often a good idea than not. If for nothing else that for mental health reasons.

What we want to do now is to start collecting potentially “Slick” code snippets. This could mean beautiful code, clever code, code which does something awesome, or whatever. You can also post code that sucks. We just need code to start the tallies running.

When you visit the site, you’ll be presented with two code samples, probably doing totally different things. They might even be in different languages. Your job is to choose the better of the two. If you can’t make up your mind, just hit refresh.

I doubt this is going to unearth great code in the same way that I doubt Hot or Not unearths great people. But I hope it will be fun to play with. It will probably be broken and we know it’s lacking some necessary features to make it more useful and/or fun. Suggestions are welcome. Just don’t take it too seriously.

Amazon has a cool article on how to use Active Resource as a consumer for SimpleDB through the AWS SDB Proxy for Rails.

Last week, I used the Lwt cooperative lightweight thread library to implement a benchmark that measures context switch performance, determined that it was GC bound and timed it against comparable programs (i.e., the fastest implementations in the computer language benchmark games, which are all based on lightweight threads) and a C version that uses POSIX threads, obtaining these results:

Here are the figures I get for the C version I made with Protothreads:

(Read below for an explanation of the difference between optimum and pessimum scheduling policies.)

It is nearly 400 times faster than the C version with POSIX threads, and represents a one order of magnitude improvement over the other lightweight thread implementations. It also needs less memory. The performance is almost unbelievable. Where's the catch, ugly code maybe? Judge for yourself; here are the Protothreads and the POSIX threads C implementations head to head*1:

/* Protothreads, 0.076s*/                      /* POSIX threads, 28.7s */
#include <stdio.h>                             #include <stdio.h>
#include <stdlib.h>                            #include <stdlib.h>
#include "pt.h"                                #include <pthread.h>
#include "pt-sem.h"                            #include <limits.h>
                                               
#define NUM_THREADS 503                        #define NUM_THREADS 503
                                               
static struct pt thread_context[NUM_THREADS];  struct stack {
static int data[NUM_THREADS];                     char x[PTHREAD_STACK_MIN];
static struct pt_sem mutex[NUM_THREADS];       };
                                               
static                                         static pthread_mutex_t mutex[THREADS];
PT_THREAD(thread(struct pt *pt, int id))       static int data[THREADS];
{                                              static struct stack stacks[THREADS];
  static int token;                            
  static int r;                                static void* thread(void *num)
  PT_BEGIN(pt);                                {
                                                  int l = (int)num;
  while(1) {                                      int r = (l+1) % THREADS;
      PT_SEM_WAIT(pt, &mutex[id]);                int token;
      token = data[id];                        
      r = (id + 1) % NUM_THREADS;                 while(1) {
      if(token) {                                    pthread_mutex_lock(mutex + l);
          data[r] = token - 1;                       token = data[l];
          PT_SEM_SIGNAL(pt, &mutex[r]);              if (token) {
      } else {                                          data[r] = token - 1;
          printf("%d\n", id + 1);                       pthread_mutex_unlock(mutex + r);
          exit(0);                                   }
      }                                              else {
  }                                                     printf("%i\n", l+1);
  PT_END(pt);                                           exit(0);
}                                                    }
                                                  }
int                                            }
main(int argc, char *argv[])                   
{                                              int main(int argc, char **argv)
 int i;                                        {
                                                  int i;
 if(argc != 2) exit(255);                         pthread_t cthread;
 data[0] = atoi(argv[1]);                         pthread_attr_t stack_attr;
                                               
 for(i = 0; i < NUM_THREADS; i++) {               if (argc != 2) exit(255);
     PT_SEM_INIT(&mutex[i], 0);                   data[0] = atoi(argv[1]);
     PT_INIT(&thread_context[i]);              
 }                                                pthread_attr_init(&stack_attr);
                                               
 PT_SEM_INIT(&mutex[0], 1);                       for (i = 0; i < THREADS; i++) {
                                                     pthread_mutex_init(mutex + i, NULL);
 while(1) {                                          pthread_mutex_lock(mutex + i);
     for(i = 0; i < NUM_THREADS; i++)          
         thread(&thread_context[i], i);              pthread_attr_setstack(&stack_attr, &stacks[i], 
 }                                                                         sizeof(struct stack));
}                                                    pthread_create(&cthread, &stack_attr, thread, 
                                                                    (void*)i);
                                                  }
                                               
                                                  pthread_mutex_unlock(mutex + 0);
                                                  pthread_join(cthread, NULL);
                                               }
                                               

Even though the Protothreads code is similar to the one with pthreads, there are two important differences:

• there is no thread-local data in the Protothreads version. r is recomputed on each iteration
• the proto threads are scheduled manually by running all the corresponding functions in sequence

These dissimilarities give some insight into the nature of Protothreads. They are little more than small state machines whose state is stored in an opaque pt structure. Protothreads' implementation consists of a few preprocessor macros in headers, so the best way to see how they work is to examine the output of the preprocessor (gcc -E, slightly abridged and reformatted here):

By looking at my commit logs it will be clear that I haven't really done much with Bus Scheme recently. I was having a lot of fun with the project early on, but as of late my enthusiasm has abated. I could chalk it up to the fact that Scheme is such a simple language that I'm running out of things to do. But I think the real problem is that it's too easy. Where's the glory to be had in a Scheme interpreter? The entire spec only runs about fifty pages, and it's been done before! The urge for a challenge is sadly left unmet. Plus as is common knowledge, nobody actually uses Scheme in the real world. [*] It just wasn't working out for me in spite of myself.

With that in mind I am determined I shouldn't have these problem on my next project. I'd like to announce the launch of Unicycle Java:

Unicycle Java is an implementation of the Java Programming Language™, but implemented while riding a unicycle. Note that at all the implementation of Unicycle Java must be written while riding a unicycle. Documentation, tests, and administrivia may be accomplished elsewhere, but all actual implementation code is strictly unicycle-driven. Unicycle Java is an Enterprise-grade[**] piece of software, suitable for highly scalable, performant, demanding deployments.

There isn't any code that's ready to be made public yet, but we will be sure to post progress on here as it approaches a usable state. The first task will be to procure a unicycle, so please send a message if you have any leads in this direction.

FAQ

How will you write code while riding a unicycle?

We haven't figured out all the details of how this will work, but we are looking into getting a kind of laptop-harness. Luckily unlike bicycles, unicycles do not require use of the hands, so typing is actually somewhat feasible.

What about type safety?

Always wear a helmet.

Will Unicycle Java have closures?

Of course! Unicycle Java is a powerful modern programming language, not a toy like BASIC.

Is Unicycle Java going to make stability a priority?

Since unicycles are by nature not very stable, it would be virtually impossible to do this. We believe this is simply one of the costs of unicycle-driven development. However, Unicycle Java comes with a remarkable new technology called JavaLids that will keep things safe in the event of a systems crash.

Will it be possible to run Unicycle Java in the same environments as Bus Scheme?

Do not by any means attempt to try this. Riding a unicycle in the bus lane is foolhardy and dangerous.

Unicycle Java is an important part of a balanced breakfast.

[*] - It's been theorized that all the parentheses get clogged in the Tubes due to their elongated nature. XML's angle brackets are acceptable as long as they get oriented pointy-end first before transmission, but since parentheses are round and slippery this is much more difficult.

[**] - Not really. Get over yourself.

To my surprise, Git is becoming one of the most popular source control systems in Ruby community. New blog entries on how to use Git are popping up all over the place and the amount of excitement is just very unexpected. Who would have thought that source control tools might be so exciting! Well, here are my recipes on how to use Git with Ruby/JRuby subversion repositories.

Personally, I started using Git about 6 months ago, since Rubinius is under Git, and I really wanted to get access to “rubyspecs” which are part of Rubinius repository. At first, I really disliked Git just because the command line is rather complicated, and it takes some time to get used to it and to get familiar with basic Git concepts. After a while, I noticed that I started using Git more and more, and slowly but surely the full power of the tool just started to amaze me. Switches between branches are LESS THAN A SECOND, instant access to the entire history, powerful branch merging, git grep, and the list could go on and on.

So I started to think about switching to Git as my main SCM tool, even on projects that are under Subversion. Again, the git svn turned out to be very capable and complete tool, allowing bidirectional access to subversion repository from Git. Basically, git svn allows to clone the subversion repository with ENTIRE HISTORY into Git repository, and the changes to that Git repository can be pushed back to main Subversion one. On top of that, one can use Stacked Git to maintain a series of patches. This is very useful in case when there is no write access to the main repository. So the patches are handled by Stacked Git and are being sent out to the maintainers via email, or attached to the bug reports. And that’s how I worked on JRuby until I got the commit rights:

1. stg pull svn - get the latest sources from the main (svn) repository, this command will first pop all your currently applied patches, get the latest sources, and re-apply the patches on top of them.
2. stg edit - create new patch, provide description
3. hack, hack, hack
4. stg refresh - record the changes into the patch
5. stg export -p - export the patches into standalone files
6. send the patch to the maintainers

It’s sooo much easier to maintain the patches that way rather than just having the diff files flying around. First, the patches are always against the latest sources, and never outdated (well, at least when somebody changes something in the main repository that conflicts with the patch, you’ll see the issue immediately during the step #1 above, and can correct it right away). Second, it’s trivial to push/pop patches, reorganize them, test with and without them, easy to see which ones applied and which ones are not, etc.

For those who are serious on working on some open source project which is currently under subversion, I recommend to investigate git-svn and stgit in more detail.

Now, here are a couple of examples on how to create a Git repository out of public subversion one.

Let’s start with Ruby itself:

git svn clone -Ttrunk -ttags -bbranches http://svn.ruby-lang.org/repos/ruby ruby.git

Warning: This would take about 12 hours to complete! For those who don’t want to wait that long, here’s the repository I’ve already created: ruby.git.tgz (md5sum: 576b5667affe040fd33478ea074c13b8). Think about it, these 60mb contain entire Ruby history! This is LESS than size of subversion’s working copy! I measured the time to switch between Ruby 1.8 and Ruby 1.9 branches, that’s 500.000 lines changed in 2000 files, and it takes on my Ubuntu Linux about  1.5 SECONDS. git-grep of entire 1.9 branch is 0.2 seconds. git-log of entire 1.9 branch is 0.3 seconds, from the very last revision to the revision #1, is 0.3 seconds. What else can I say?

Now, the JRuby repository:

git-svn clone -Ttrunk/jruby -ttags -bbranches http://svn.codehaus.org/jruby jruby.git

If you have commit rights to JRuby repository, just change HTTP to HTTPS. The process would take about 2.5 hours (and I saw reports that it can be interrupted and then resumed too).

Once you have the local Git repository, you’ll probably need to do the following (when your are in the root of the repository) :

   1: git gc –aggressive –prune

   2: (echo; git-svn show-ignore) >> .git/info/exclude

This would compact the repository further, and will set the proper excludes.

This is just a very basic outline of the process, so if you’re interested, do check out the official documentation and feel free to ask questions.

gibak is a backup tool based on git. Since gibak builds upon the infrastructure offered by Git, it shares its main strengths:

• speed: recovering your data is faster that cp -a...
• full revision history
• space-efficient data store, with file compression and textual/binary deltas
• efficient transport protocol to replicate the backup (faster than rsync)

gibak uses Git's hook system to save and restore the information Git doesn't track itself such as permissions, empty directories and optionally extended attributes and mtime fields.

You can read more about gibak here.

What's new in 0.3.0

• OSX support
• support for extended attributes both on Linux and OSX (it might work on other systems if their getxattr(2) interface matches Linux or OSX's).
• a few bugfixes

Read README.upgrade if you used earlier versions of gibak and want to use extended attributes.

Getting it

The latest tarball as of 2008-03-31 is gibak-0.3.0.tar.gz.

You can always get the latest code with

git clone http://eigenclass.org/repos/git/gibak/.git/

The repository can be browsed at http://eigenclass.org/repos/gitweb

Thanks

• Lee Marlow solved problems with submodule paths including spaces
• sean provided most of the fixes required for OSX support

Some FAQs

(This will eventually be moved to a new node.)

Will the repository grow monotonically? Can I delete older history? What about large files?

def fib(n)
  {n => :fixnum}
  if n < 2
    n
  else
    fib(n - 1) + fib(n - 2)
  end
end

def go
  fib(35)
end

jruby -rduby/plugin/math lib/ruby/site_ruby/1.8/duby/c_compiler.rb fib.rb

int fib(int n) {;
if (n < 2) {n} else {fib(n - 1) + fib(n - 2)};
}

;
int go() {fib(35)}

;

@JRubyMethod(name = "copy_stream", meta = true, compat = RUBY1_9)
public static IRubyObject copy_stream(
        IRubyObject recv, IRubyObject stream1, IRubyObject stream2)
        throws IOException {
    RubyIO io1 = (RubyIO)stream1;
    RubyIO io2 = (RubyIO)stream2;

    ChannelDescriptor d1 = io1.openFile.getMainStream().getDescriptor();
    if (!d1.isSeekable()) {
        throw recv.getRuntime().newTypeError("only supports file-to-file copy");
    }
    ChannelDescriptor d2 = io2.openFile.getMainStream().getDescriptor();
    if (!d2.isSeekable()) {
        throw recv.getRuntime().newTypeError("only supports file-to-file copy");
    }

    FileChannel f1 = (FileChannel)d1.getChannel();
    FileChannel f2 = (FileChannel)d2.getChannel();

    long size = f1.size();

    f1.transferTo(f2.position(), size, f2);

    return recv.getRuntime().newFixnum(size);
}

require 'benchmark'

Benchmark.bmbm do |bm|
  bm.report("Control") do
    10000.times do
      File.open(__FILE__) do |file|
        File.open(__FILE__ + ".tmp", 'w') do |file2|
        end
      end
    end
  end
  bm.report("10k file copies") do
    10000.times do
      File.open(__FILE__) do |file|
        File.open(__FILE__ + ".tmp", 'w') do |file2|
          IO.copy_stream(file, file2)
        end
      end
    end
  end
end

                      user     system      total        real
Control           2.191000   0.000000   2.191000 (  2.191000)
10k file copies   5.190000   0.000000   5.190000 (  5.190000)

                      user     system      total        real
Control           0.360000   0.550000   0.910000 (  0.903211)
10k file copies   0.850000   2.450000   3.300000 (  5.363433)

• Learn - and understand - Lisp
• More generally, learn a new language that is sufficiently different from your current knowledge
• Work with domain specific languages
• Understand metaprogramming
• Read up on the available data structures (Knuth anyone)?
• Implement a compiler and runtime system for a language