Ruby to C, a couple snippets

As you might already know, the Ruby interpreter you’re (probably) using (the one at ruby-lang.org) know as MRI, for Matz Ruby Interpreter, is written in C. The VM is entirely written in C and most of the standard libraries too. So it seems that the way to make code run faster is really to dive into C.

Now I hear you yell with your ears all red, eyes full of blood and a big vein pumping on your forehead: But it will be unmaintainable!

If everything seems under control, you’re not going fast enough

- Mario Andretti

The hard part is that the C API of MRI lacks documentation. You either have to search through the mailinglist, look at other people’s code or try to guess from Ruby’s code. I recently ported a bit of code from Ruby to C and here are a couple of Ruby to C mapping that might help you.

If you’ve never written or looked at a Ruby extension before, I suggestion reading Peter Cooper excellent and very simple tutorial on how to set things up.

Objects are VALUEs

my_var = nil this_is_true = true some_fixnum = 1

VALUE my_var = Qnil; VALUE this_is_true = Qtrue; VALUE some_fixnum = INT2FIX(1);

Strings

string = "hi"

VALUE string = rb_str_new2("hi"); /* Ruby string to C string */ char *s = RSTRING_PTR(str); int len = RSTRING_LEN(str);

Object to C struct

In C you'll probably want to store your data in a struct. Ruby provide some things to wrap a struct inside a Ruby object. Also, since Ruby is not aware of the stuff created in the C world we have to be pretty explicit about everything.

module MyModule
  class MyClass
    def close
      @closed = true
    end
  end
end


In C
/* This is called by Ruby GC when the object is freed
 * so free all resources used here. We hook it up
 * in the Data_Wrap_Struct call down there.  */
void my_obj_free(my_struct_t *s)
{
  free(s);
}

/* Called by Ruby then and instance of your class is created
 * hooked by rb_define_alloc_func. */
VALUE my_obj_alloc(VALUE klass)
{
  my_struct_t *s = ALLOC_N(my_struct_t, 1);
  /* This stores the struct inside the Ruby object, so you
   * can get the struct back on each method call. */
  return Data_Wrap_Struct(klass, NULL, my_obj_free, s);
}

/* The actual MyClass#close method, first argument is the
 * instance object. It's hook into our class by the rb_define_method
 * call in Init_ */
VALUE my_obj_close(VALUE self)
{
  my_struct_t *s;
  /* This is where we get the struct back from the Ruby object */
  Data_Get_Struct(self, my_struct_t, s);
  
  s->closed = 1;
}

/* Init_* is magically called by Ruby to bootstrap your extension,
 * it's like a main function if you will. */
void Init_name_of_your_extension()
{
  VALUE mMyModule = rb_define_module("MyModule");
  VALUE cMyClass = rb_define_class_under(mMyModule, "MyClass", rb_cObject);  

  rb_define_alloc_func(cMyClass, my_obj_alloc);
  rb_define_method(cMyClass, "close", my_obj_close, 0);
}


Calling methods



obj.method_name

result = begin
  obj.method_name(arg)
rescue
  # …
end




rb_funcall(obj, rb_intern("method_name"), 0);
/* if an error is raised, result will be set to Qundef */
VALUE result = rb_funcall_rescue(obj, rb_intern("method_name"), 1, arg);





Blocks



my_ary.each do |i|
  puts i
end




VALUE i_each(VALUE elem, VALUE *obj)
{
  /* elem is the yielded object and obj is a pointer to obj down there v */
  printf("%d\n", FIX2INT(elem));
  return Qnil;
}
rb_iterate(rb_each, my_ary, i_each, (VALUE) obj);





Error handling



raise ArgumentError, "..."




rb_raise(rb_eArgError, "...");
/* To raise an error corresponding to the one in errno */
rb_sys_fail("oops");





Garbage Collection
When creating Ruby objects in the C world, Ruby is not aware of the scope of each, it's impossible to track when a variable is used and when it's not, so it's impossible for the garbage collector to know when do leave or sweep objects. If you create a long live object, that will persist for several method calls you have to tell Ruby's garbage collector when it's in use and when you're finished with it. If you don't, Ruby GC will free your variable in the worst possible time, ending up with unthinkable tragedies around the world, leaving you homeless and poor for all eternity.
VALUE var = rb_str_new2("prout");

/* Tell Ruby GC this object is in use */
rb_gc_register_address(&var);
/* when you're finished you can unregister it and let the GC free it */
rb_gc_unregister_address(&var);


Note that you don't need to do this if you only use the variable in one method since the control is not given back to Ruby during the method call. Also if the variable is still referenced in the Ruby world it's already registered.
More, more, more!
This is far from a complete guide to MRI, you'll need to dig the Ruby doxygen doc if you need more info.
Now lets port Rails to C!
							

Moron no more : I can Git !

Since Linus said we’re all morons ’cause we use Subversion and the like, lots of people started to migrate to his new faster, better, stronger, cooler and geekier revision control system Git.

Installation for Subversion morons like me

Get the latest stable source from
http://git.or.cz/

wget http://kernel.org/pub/software/scm/git/git-1.5.3.7.tar.gz
tar xzf git-1.5.3.7.tar.gz
cd git-1.5.3.7.tar.gz
make prefix=/usr/local all
sudo make prefix=/usr/local install

Create your own little repo

mkdir moron
cd moron
git init

Congrats, you just created your first Git repo!

Most common commands looks like Subversion ones. add, rm, mv and commit. But you gotta remember that there’s no such thing as a working copy in Git. It’s a distributed revision control system. Which means that your local copy is a repository. In fact, with Git you do not checkout (in the Subversion sense) a copy of the repo, you clone it. That’s why Git is so much faster. Most of the commands: history browsing, local commits, reverting, etc, are done locally, without talking to the server. Check the official guide for Subversion morons.

Try adding a file:

echo "I'm no longer a moron Linus" > moron-no-more.txt
git commit -a

The -a ensures that all new files are added, you can do this manually with the git add command.
You then commit you change as you would do in Subversion, with the difference that it’s all done locally:

git commit -m "Take that Linus!"

All is not lost for us poor Subversion mortals

The cool thing about Git is it has builtin support for playing nicely with Subversion. Which ease the migration and can even let you use Git side by side with Subversion. So you can import a Subversion branch like this:

git svn init http://repo/url/trunk
git svn fetch

You Git repo now has all the revisions of you Subversion branch. Try running git log.
You can even send you changes back to Subversion:

git svn dcommit

Create your central repo

mkdir mah-repow.git
cd mah-repow.git
git --bare init-db
git-repo-config remotes.origin.url http://yourhostname.com/git/mah-repow.git/; # notice trailing slash!
git-update-server-info

Upload that directory to your public server. Like Subversion, Git supports various ways of serving a remote repo, I chose WebDAV because it’s the simpler one on DreamHost.

To send your changes to the server with Git you push your changes to the server and pull changes from others. With our previous examples, to push a series of commits to the server you’d do:

git repo-config remotes.origin.url http://yourhostname.com/git/mah-repow.git/; # once to setup the default server url
git push

Happy Gitting!

Getting TextMate ready for Rails 2.0

After updating a couple of existing Rails app to 2.0, I found out some of TextMate sugar was out of date. Here’s 2 things I did to put the hot and sweet sugar back into TextMate.

Update the bundle

TextMate bundles are under heavy development all the time, get the latest version from their repo:

svn export
http://macromates.com/svn/Bundles/trunk/Bundles/Ruby%20on%20Rails.tmbundle


Then double-click on the downloaded Ruby on Rails.tmbundle.
Fix the test commands
I'm a heavy user of Run (command+R), Run Focused Unit Test (shift+command+R) and the Rails menu (shift+ctrl+\) but that do not work with Rails 2 because of a conflict with the builder module.
Rob Sanheim explain how to solve this very easily, just rename the Builder.rb in the Rails bundle.
Hope this helps!
							

Moving to Rails 2.0

I know … it’s like exercising, you know you’ll be better afterwards but you don’t feel like going trough the pain right now. Updating a framework can be a pain, here’s a short list summarizing what I had to do to migrate RefactorMyCode from Rails 1.2.3 to 2.0 (Preview). I hope this makes it easier for you!

0. Pistonize it!

Instead of using the classic way of freezing Rails in you app tree (rake rails:freeze:edge TAG=rel_2-0-0_PR), I’d recommend using Piston:

piston import http://svn.rubyonrails.org/rails/tags/rel_2-0-0_PR/ vendor/rails


This way, you'll be able to make changes to Rails code and keep them while updating your copy!
1. Update your stuff
Update configs, scripts and javascripts:
rake rails:update


2. Move your session config to environment.rb
Add this in your environment.rb and remove the session ... line in application.rb:
# Your secret key for verifying cookie session data integrity.
# If you change this key, all old sessions will become invalid!
config.action_controller.session = {
  :session_key => '_app_name_session_id',
  :secret      => 'your secret key'
}

To compute your secret key:
require 'digest/md5'
puts [now = Time.now, now.usec, rand(0), $$, 'you_app_name'].inject(Digest::MD5.new) { |md5, e| md5 << e.to_s }

3. Split environment.rb
Rails now recommend putting your specific config code in config/initializers. Move everything outside Rails::Initializer.run do |config| ... end into separate files. For starter, create config/initializers/inflections.rb and config/initializers/mime_types.rb and move what was in environment.rb. Split as much as you can in files with descriptive names.
4. Singleton resource, plural controller name
If you were using singleton resource: map.resouce :account, you'll need to pluralize your controller name. In this case AccountsController. Not sure if it makes more sense, but follows the convention to pluralize controller names I guess...
5. Rename your views
Rails 2.0 brings a beautiful view naming convention:
[name].[content type].[template engine]
eg.: show.html.erb or create.js.rjs
Renaming manually can be pretty painful. Jamie created a Rake task for this.
6. Fix the plugins
Expects some plugins to break. In my case, asset_packager broke, but fixing it was simple: synching method signature for compute_public_path.
I'd recommend checking plugins compatibility before doing the jump. Read the doc. Or if you're feeling adventurous, give it a try.
7. Add Request Forgery Protection
Cross-site request forgery can be hard to protect from. Not with Rails 2.0.
Add this to your application.rb
# See ActionController::RequestForgeryProtection for details
# If you're using the Cookie Session Store you can leave out the :secret
protect_from_forgery :secret => 'same_as_the_session_one'


And this to your config/environments/test.rb:
# Disable request forgery protection in test environment
config.action_controller.allow_forgery_protection    = false


Good luck!
Update: Paolo sent me an article he wrote for upgrading routes to 2.0 w/ a script, nice!
							

Blogging from TextMate

It occurred to me that some people are still writing their blog post from WordPress, that is so last year. I also realized, while reading Hugh McGuire question, that TextMate is really powerful for blogging. I’ve been using it for quite some time now but didn’t realize until now all the tiny features I use while blogging. Here’s a short screencast of how I use TextMate to post to my blog.

TextMate blogger useful shortcuts

• ^⌥⇧B – Switch to Blog bundle
• ⇧⌘: – Spell check
• ^⌥⌘ – Preview
• ^⌘ – Publish
• title[tab] – Snippet for post title
• cat[tab] – Fetch the list of categories
• ^⇧L – Turn highlighted text into a link (also fetch the page title!)
• ^⇧⌘L – Search current text in Google and turn into a link
• ⌘B – Make current text bold
• ⌘I – Make current text italic
• ^⌘T – Search through Bundle items (and shortcuts)
• ^⇧⌥⇧⌘⌥X – Do absolutely nothing

Gary got another list of useful shortcuts in TextMate.

How do you write your blog posts ?

Blitzweekend : Don’t do it!

This post is cross-posted on Blitzweekend’s blog

When Heri asked me to write a post on Blitzweekend’s blog to share some of my tips on building an app in one weekend I thought only of this one:

Don’t do it!

Truth is, if your manager and boss finds out you can do in 2 days what takes us 4 months at work, we’d all be screwed! So keep it quiet and pretend it was an old project you’ve been working on for a long time (months, even years if your boss is not that bright).

See the true story bellow:

If you’re still reading, maybe you don’t care about being labelled as a fast worker. Perhaps you just like creating new things so much, you can help yourself but build stuff again and again! But for most people the problem is: how to turn this into something other people can use. Not only you. Here are 3 tips I’d like to share with you on how to get that damn weekend project out the door, fast, without any pain (until you get labelled a fast worker) while having fun (for now! wait ’till you get labelled!).

1. Focus

Keep everything simple, as simple as it can get. Don’t try to solve all the problems in the world all at once. Find one problem, for example: greasy door handles, and solve that single problem: a web site for locating clean door handles in a specific building.
But hey, it would be cool to also share pictures of the greasy door handle with your friends! That’s when you start loosing focus. Choose only one idea and keep it simple and atomic. It should be impossible to split your project into smaller ones.

2. Remove, don’t add stuff

Here’s a quote I quote way to often:

Perfection is reached not when there’s nothing left to add, but when there’s nothing left to remove
- Antoine de St. Exupery

Don’t think of new features you could add to make it more cool. Think of which one are just too much or don’t add that much to your project. If you need to add features to make your project interesting it’s because your original idea sucks.

It’s like a 3 legs chicken, if you pull on the features leg, you’re gonna need more resources and time. But if your time and resources are restricted, your only option is to cut off the features leg (who wants a 3 legs chicken anyway?).

If you think you’re going the wrong direction, start over. That’s why it’s important to put your code under source control from the beginning, revert is your friend when everything goes wrong!

3. Make it work already!

Don’t go big bang approach! Build sometime small, simple (even stupid) and release, rinse and repeat.
Most people don’t take the time to automate parts of their development because they think: it’s just a small project. Bullshit! Because it’s a small project, you’ll be making lots of small changes fast. So you wanna make sure you can deploy in one step, test in one step and restart, stop, cleanup, backup, eat, sleep in one step.

In the case of a web application, deployment can be time consuming if your not used to it. The Internet is full of guides on how to do it. Make sure you deploy a small version before your project gets too much dependencies. If something fails, it will be harder to find out which of the 10 libs you depend on is bogus rather the just making a bare-bone app work. I would also recommend automating your deployment with tools such as Capistrano, Vlad or Paste deployment.

Now that we’re all labelled as fast workers I think blitzweekend should be shrunk to 2 hours.

Have you built an app in 2 days yourself? Share some tips with us! (but don’t tell our bosses)

3 simple tips to make your Rails app 100 times faster

After the initial meltdown of the RefactorMyCode server I had to find a solution to put it back online fast!

Here are 3 things I did that helped me put the app back online and made it amazingly fast serving more then 50 000 page load in a day, even if it was hosted on a shared hosting plan on DreamHost. Not the fastest provider but the cheapest and most supported for sure. I since then switched to SliceHost on an account graciously offered by StandoutJobs (that’s right, for my personal project, how cool is that?).

I know lots of people have wrote guides to make your web app faster, but I tried, with this one, to focus on the simplest, yet most effective, things you can do to dramatically speed up your app without loosing any of Rails and Ruby tasty sugar. I think even people with little experience with Rails and web deployment can follow this guide with no problem.

1. Pack that asset up

First tip is the easiest to put in place. Downloading your 3 stylesheets and 6 javascript files slows down page loading because the web browser has to open a new connection for each one. Plus some browser limits the number of files that can be downloaded from a same host at the same time.
The solution is to reduce the number of external resources your page depend on by merging all of them into one file. AssetPackager is great for doing this in your Rails app, plus it minifies your javascript for free, check out RefactorMyCode stylesheet and javascript file. All that by changing one line in your view layout!

Now to make it load even faster you want to move all your static files to another host (see why here). For example on RefactorMyCode, images, the stylesheet and javascript are loaded from assets.refactormycode.com.
With Rails, this is stupidly easy. In your config/environments/production.rb, set config.action_controller.asset_host to your assets host. Now all the image_tag, javascript_include_tag, etc will point to this host. On your DNS server, all you need to do is setup an alias (A record) pointing to the same IP as the main domain.

2. Remove unused stuff

Something to keep in mind while trying to improve performance is that, for each request, everything will be loaded, checked and executed. Authentication, session, database queries. Do you really need all this stuff on every request?

You can turn off session on specific actions with:

session ff, nly => :index

This boost performance like crazy. So it’s a great alternative to caching if you really need to go to the database but not to login the current user.

If you’re using RestfulAuthentication or ActsAsAuthenticated plugin, you can turn off loading the user form the cookie with:

skip_filter :login_from_cookie

which will save you one query.

Use fragment caching for partial views with heavy data from the db and skip your queries on the controller side with:

@users = User.find('all) unless read_fragment('unique_cache_key')

Also with queries, make sure you know how to use the :include option for eager loading of associations.

Post.find(:all, :include => :user)

will cut the number of queries in half.

3. Cache the whole page

This last tip is by far the most effective. Cache everything you can to reduce queries to the database, or even, requests to the dispatcher.
The best solution is caching the whole page into and HTML file so the web server serves the static file rather then sending the request to the dispatcher. Most web server are serving static files faster then Lance Armstrong on drugs. And can serve lots and lots of them, letting your dispatchers breath a bit between requests. Rails Envy as a great guide on how to set this up.

Now the hard part is caching the layout. To cache a full page, the page needs to be the same for every users, logged in or not. To do that, you have to remove all those <%= ... if logged_in? %> and show everything, at least in the HTML. You still can use javascript to show and hide links that are for logged in users. For example here’s a code sample from RefactorMyCode:

var CurrentUser = {
  loggedIn: false,
  author: false,
  admin: false,

  init: function() {
    this.loggedIn = Cookie.get('token') != null;
    this.admin    = Cookie.get('admin') != null;
  }
};

var Application = {
  init: function() {
    CurrentUser.init();
  },

  onBodyLoaded: function() {
    if (CurrentUser.loggedIn) {
      $$('.if_logged_in').invoke('show');
      $$('.unless_logged_in').invoke('hide');
    }
    if (CurrentUser.admin) {
      $$('.if_admin').invoke('show');
    }
  }
};


Another thing you wont be able to use anymore on your layout is <%= flash[:notice] %>. The flash changes on each request, not cacheable. The solution again is to transfert everything to the client with cookies and javascript. Cacheable Flash is a nice Rails plugin to do just this. Cool thing is, you don't need to change anything on your controller. Just replace your previous flash ERB block with something like this:
<div id="error_div_id" class="flash flash_error"></div>
<div id="notice_div_id" class="flash flash_notice"></div>
<script type="text/javascript">
  Flash.transferFromCookies();
  Flash.writeDataTo('error', $('error_div_id'));
  Flash.writeDataTo('notice', $('notice_div_id'));
</script>


I hope this was helpful to you. Let me know if you got more tips or questions on how I've setup my server.
							

Incremental Perfectionism : 5 ways to write perfect code

If you’ve ever done a couple of interviews, you’ve probably got the “I’m a perfectionist” candidate. When asked about its flaws he says with a fake-hidden smile: “I’m a lil’ bit of a perfectionist”, thinking deep in their mind this is more of a good thing then a bad one. Perfectionist, those people that only craft perfect thing, only the best (by the best), free of any flaw (except being too perfect), a piece of art left to be admired, not touch by anyone even it’s creator!

Well, that might be good for some poor artist on crack but we’re not! We’re developer driven by business rules, deadlines, changing requirements, technology and coffee. Unhealthy perfectionism is like the big bang approach to integration. It’s all or nothing, but most of the time, it’s nothing.

Here are 5 ways to bring your software closer to perfection, bits by bits, while still shipping your damn code (no crack required).

1. Small changes

Making 1000 lines of code perfect in a respectable amount of time is impossible. But tuning 10 to perfection is easy and fun. It’s also easier to wrap your head around the tiny problem you’re trying to fix.
It’s even easier to do this when writing tests, they help you split a complex problem into smaller and simpler ones and focus on one, one at the time. Hack some code, make the tests pass, commit, update and repeat.

2. Every single change should make the whole better

Incremental Perfectionism, is like the poor-artist-on-crack perfectionism on a smaller, healthier scale. Don’t try to make the whole thing perfect. But never ever settle for something that is not the best possible solution to the small problem you’re solving. Don’t copy paste some code because it’s easier, don’t comment out a block of code because it’s faster, don’t live with ugly code. But don’t fix it all at the same time!

3. Refactor, refactor and refactor

Perhaps your understanding of the problem has evolve or you’ve learn a new and better way of doing something. Backed by a bunch of solid tests, listen to the code artist in you and refactor! Make the code beautiful, gorgeous, a piece of art, more perfect! Make the tests pass, commit, update and repeat.

4. svn -R revert .

If you feel you’re breaking any of the previous rules, start over. Maybe your changeset is getting too big. You’re trying to change too much things at the same time. If you’re afraid to loose your precious code, commit your working copy to a branch:

svn cp . http://svn.icanhazcode.com/svn/branches/my_precious

or create a patch:

svn diff > my_precious.diff

then, step on your ego and

svn -R revert .

But since we’re working with small changes, you haven’t lost that much right ?

5. Read lots of code

A good way to trigger your imagination into moving your software closer to perfection is too look at how other people are trying to do it. Open source projects are open books into some of the best programmers mind. They’ve spent time trying several ways to solve a problem, don’t spent your time doing the same thing!

I hope you find this helpful. Let me know how you’re making your code more perfect!

DRY-ing your Rails controller with filters

Ever noticed how all CRUD methods on each controller starts with the same line: @item = Model.find(params[:id]) ?

Hey! why not use filter so we don’t repeat ourselves ?

class PostsController < ApplicationController
  before_filter :find_post,   nly => [:show, :edit, :update, :destroy]

  def index
    @posts = Post.find(:all)
  end

  def show
  end

  def new
    @post = Post.new
  end

  def edit
  end

  def create
    @post = Post.new(params[:post])

    if @post.save
      flash[:notice] = 'Post was successfully created.'
      redirect_to post_url(@post)
    else
      render :action => "new"
    end
  end

  def update
    if @post.update_attributes(params[:post])
      flash[:notice] = 'Post was successfully updated.'
      redirect_to post_url(@post)
    else
      render :action => "edit"
    end
  end

  def destroy
    @post.destroy

    redirect_to posts_url
  end
  
  private
    def find_post
      @post = Post.find(params[:id])
    end
end


The cool thing about this is, if you decide to switch to using model name in the url, you'll only have one line to change!
It's a small step for Rails, but a great step towards honouring the DRY principle:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

- The Programatic Programmer

DRY-fulness we salute you!