Running a SproutCore Development Server with Rack (and Passenger)

UPDATE: Please understand this is only for development mode. It should not be used in Production. SproutCore provides sc-build for deployment.

Here at Centro we maintain a SproutCore application that sits in front of a number of Ruby applications on the backend. Our project has evolved heavily over the last year or so: we started with the standard sc-server, eventually moved to Unicorn, and finally to Passenger. There doesn't seem to be a lot of discussion around the use of the SproutCore build tools and I hope this writeup proves useful to others looking to do something similar.

In the early stages of development we designed our Ruby services to make requests to http://localhost:4020, as sc-server generously provides a Buildfile that allows for easy proxy configuration. At the time it seemed like a great way to DRY up our configuration in development mode. Why bother to track everybody's port number in each project?

# ===========================================================================
# Project:   Transis
# Copyright: ©2009 Centro
# ===========================================================================

config :all, :required => [:sproutcore, :blueprint, :transis],
  :test_required => [:test_runner_extension]

config :examples, :load_fixtures => true
config :main, :theme => 'transis', :layout => 'lib/index.rhtml'

proxy "/id",        :to => "localhost:3002"
proxy "/planning",  :to => "localhost:3003"
proxy "/templates", :to => "localhost:3004"
proxy "/research",  :to => "localhost:3005"
...


We started running into issues pretty quickly with sc-server being single threaded. For example, when a user's browser makes a request to /research that app will make a request to /id to verify their authenticity. Since everything is running through a single threaded server timeouts quickly start piling up and the app ground to a halt.

We experimented with a teeny Rack based proxy server but quickly bailed. There had to be a better way. All of our Ruby services play nice with Rack (thanks to Sinatra and Rails), and we know the SproutCore development server is written in Ruby. We wanted to unify our development environment as the collection of scripts required was getting out of hand. The next approach was to replace the default Thin server used in sc-server with a cluster of Unicorns. This meant we had to dive into the SproutCore source and figure out how sc-server really works.

After a brief look around, it became clear the place to start is lib/sproutcore/tools/server.rb. Check out the code snippet below.

# get project and start service.
project = requires_project!

...

SC.logger << "SproutCore v#{SC::VERSION} Development Server\n"
SC::Rack::Service.start(options.merge(:project => project))

Since SproutCore is built around Rack, this should be pretty easy. The last line is pointing to SC::Rack::Service.start, so our next step was to see what's going on in there. Below are a few crucial excerpts from the start method (a lot of configuration has been removed).

# Guess.
if ENV.include?("PHP_FCGI_CHILDREN")
  server = ::Rack::Handler::FastCGI

  # We already speak FastCGI
  options.delete :File
  options.delete :Port
elsif ENV.include?("REQUEST_METHOD")
  server = ::Rack::Handler::CGI
else
  begin
    server = ::Rack::Handler::Thin
  rescue LoadError => e
    begin
      server = ::Rack::Handler::Mongrel
    rescue LoadError => e
      server = ::Rack::Handler::WEBrick
    end
  end
end

...

app = self.new(*projects)

...

SC.logger << "Starting server at http://#{opts[:Host] || '0.0.0.0'}:#{opts[:Port]} in #{SC.build_mode} mode\n"
SC.logger << "To quit sc-server, press Control-C\n"
server.run app, opts

Now that we've seen sc-server is nothing more than a simple Rack app, the only piece of this puzzle still unsolved was the projects. How does sc-server generate that array? Back in lib/sproutcore/tool/server.rb (see the snippet above) it seems SC::Tools implements require_project! which eventually relies on SC::Project to do all the dirty work.

Putting all this together we managed to to design a very concise config.ru file. Check it out.

require 'rubygems'
require 'sproutcore'

root = Pathname(Dir.pwd).expand_path
project = SC::Project.load_nearest_project(root, :parent => SC.builtin_project)
service = SC::Rack::Service.new([project])
run service

We ran Unicorn with 6 workers for a while and it was going pretty well, but as our dev tools matured we decided to start running our backends in Passenger locally. Converting from Unicorn to Passenger was easy, here's where the awesome nature of Rack really shines. The above config.ru file just works with Passenger, nothing had to change. All you need is a vhost similar to the example below.

<VirtualHost *:80>
  ServerName transis.local
  DocumentRoot "/Users/abloom/Sites/centro/transis-ui/public"

  ErrorLog /var/log/apache2/transis-error.log
  CustomLog /var/log/apache2/transis-access.log combined

  # Rails application URIs
  RailsBaseURI /planning
  RailsBaseURI /research
  RailsEnv development

  # Rack application URIs
  RackBaseURI /id
  RackBaseURI /files
  RackBaseURI /templates
  RackEnv development
</VirtualHost>

By moving our entire development system to Passenger we were able to more closely replicate our production environment as well as simplify our startup and shutdown scripts. No longer must we rely on PID files or the output of lsof. Simply touch tmp/restart.txt in any of the projects to get Apache to reload your code. Life is good when things are easy.

Our journey with SC.GridView and insertionOrientation

We're building an internal tool here to manage publisher organizations and their properties. We want to be able to choose which channels are associated with each property. I followed the instructions at http://wiki.sproutcore.com/w/page/28719873/Show-a-relation-as-a-checkbox-on-a... with some modifications to tie the selection to a SC.GridView and ended up with the code:

channelsEditView: SC.GridView.design({
  rowHeight: 22,
  columnWidth: 300,
  contentBinding: 'Gut.propertyChannelsController.channelsForSelection',
  contentValueKey: 'name',
  contentCheckboxKey: 'checkBoxValue',
  isSelectable: NO,
  exampleView: SC.ListItemView
})

Which produced:

Gridviewwrongorder
That's a great start but we really want the items to sort alphabetically down the page instead of across. Let's take a look at the source code of SC.GridView:

SC.GridView = SC.ListView.extend(
/** @scope SC.GridView.prototype */ {
  classNames: ['sc-grid-view'],
 
  layout: { left:0, right:0, top:0, bottom:0 },

  /**
    The common row height for grid items.
   
    The value should be an integer expressed in pixels.
  */
  rowHeight: 48,
 
  /**
    The minimum column width for grid items.  Items will actually
    be laid out as needed to completely fill the space, but the minimum
    width of each item will be this value.
  */
  columnWidth: 64,

  /**
    The default example item view will render text-based items.
   
    You can override this as you wish.
  */
  exampleView: SC.LabelView,
 
  insertionOrientation: SC.HORIZONTAL_ORIENTATION,

Wow, insertionOrientation. That's pretty much exactly what we want. So after changing channelsEditView to:

channelsEditView: SC.GridView.design({
  rowHeight: 22,
  columnWidth: 300,
  contentBinding: 'Gut.propertyChannelsController.channelsForSelection',
  contentValueKey: 'name',
  contentCheckboxKey: 'checkBoxValue',
  isSelectable: NO,
  insertionOrientation: SC.VERTICAL_ORIENTATION,
  exampleView: SC.ListItemView
})

Nothing changes. Lets take a look at what insertionOrientation does:

~/.rvm/gems/ree-1.8.7-2010.02/gems/sproutcore-1.4.1$ grep -r insertionOrientation *
lib/frameworks/sproutcore/frameworks/desktop/views/grid.js:  insertionOrientation: SC.HORIZONTAL_ORIENTATION,

The original author knew that we would need that configuration later on, but didn't get a chance to implement it. Here's where the pulling up your sleeves aspect of using an open source framework comes in. Time to implement it. Let's take a look at SC.GridView to see how the child views are laid out. The interesting code is in layoutForContentIndex:

/** @private */
layoutForContentIndex: function(contentIndex) {
  var rowHeight = this.get('rowHeight') || 48,
      frameWidth = this.get('clippingFrame').width,
      itemsPerRow = this.get('itemsPerRow'),
      columnWidth = Math.floor(frameWidth/itemsPerRow),
      row = Math.floor(contentIndex / itemsPerRow),
      col = contentIndex - (itemsPerRow*row) ;
  return {
    left: col * columnWidth,
    top: row * rowHeight,
    height: rowHeight,
    width: columnWidth
  };
},

When the view is rendered, SproutCore loops over all the items in the content collection and runs this method to determine where to lay out all the child views. As we can see, this completely assumes that the child views are laid out in rows down the page. So we need to add the case for laying them out in columns:

layoutForContentIndex: function(contentIndex) {
  var rowHeight = this.get('rowHeight') || 48,
      content = this.get('content'),
      count = (content) ? content.get('length') : 0,
      frameWidth = this.get('clippingFrame').width,
      itemsPerRow = this.get('itemsPerRow'),
      rows = Math.ceil(count / itemsPerRow ),
      columnWidth = Math.floor(frameWidth/itemsPerRow),
      isHorizontal = this.get('insertionOrientation') === SC.HORIZONTAL_ORIENTATION,
      row = isHorizontal ? Math.floor(contentIndex / itemsPerRow) : contentIndex%rows,
      col = isHorizontal ? contentIndex - (itemsPerRow*row) : Math.floor(contentIndex/rows);
  return { 
    left: col * columnWidth,
    top: row * rowHeight,
    height: rowHeight,
    width: columnWidth
  };
}

That works perfectly when all the items are on the screen, but as we scroll and resize, items start disappearing off of the end of the last column. After a bit of research we find that SC.GridView has a method contentIndexesInRect that determines which child views are inside the clipping frame at any time so SproutCore only needs to render items that are actually being shown. Let's take a look at it:

contentIndexesInRect: function(rect) {
  var rowHeight = this.get('rowHeight') || 48 ,
      itemsPerRow = this.get('itemsPerRow'),
      min = Math.floor(SC.minY(rect) / rowHeight) * itemsPerRow,
      max = Math.ceil(SC.maxY(rect) / rowHeight) * itemsPerRow ;
  return SC.IndexSet.create(min, max-min);
},

After we change that to incorporate the horizontal and vertical orientations we have:

contentIndexesInRect: function(rect) {
  var rowHeight = this.get('rowHeight') || 48 ,
      content = this.get('content'),
      count = (content) ? content.get('length') : 0,
      frameWidth = this.get('clippingFrame').width,
      itemsPerRow = this.get('itemsPerRow'),
      rows = Math.ceil(count / itemsPerRow ),
      columnWidth = Math.floor(frameWidth/itemsPerRow);
  if(this.get('insertionOrientation') === SC.HORIZONTAL_ORIENTATION){
    var min = Math.floor(SC.minY(rect) / rowHeight) * itemsPerRow,
        max = Math.ceil(SC.maxY(rect) / rowHeight) * itemsPerRow;
    return SC.IndexSet.create(min, max-min);
  }else{
    var indexSet = SC.IndexSet.create();
    for(var colIndex=0;colIndex<itemsPerRow;++colIndex){
      var colMinX = colIndex*columnWidth,
          colMaxX = colMinX + columnWidth;
      if( colMinX > SC.minX(rect) || colMaxX < SC.maxX(rect) ){
        var min = Math.floor(SC.minY(rect) / rowHeight) + (colIndex * rows),
            max = Math.min(Math.ceil(SC.maxY(rect) / rowHeight) + (colIndex * rows), (colIndex * rows) + rows);
        indexSet.add(min,max-min);
      }
    }
    return indexSet;
  }
},

And that works even without all items on the screen:

Gridviewcorrectorder
But when the view is resized diagonally, the left/right alignment gets screwed up:
Gridviewoutofsnc
After a TON of time trying to debug that, we see in the comments of the original contentIndexesInRect:

/** @private
  Find the contentIndexes to display in the passed rect. Note that we
  ignore the width of the rect passed since we need to have a single
  contiguous range.
*/
contentIndexesInRect: function(rect) {

'Note ... we need to have a single contiguous range'. I really wish I could tell you I have some insight into WHY we need the contiguous range, but I wasn't able to find it. So we change our new contentIndexesInRect to:

contentIndexesInRect: function(rect) {
  var rowHeight = this.get('rowHeight') || 48 ,
      content = this.get('content'),
      count = (content) ? content.get('length') : 0,
      frameWidth = this.get('clippingFrame').width,
      itemsPerRow = this.get('itemsPerRow'),
      rows = Math.ceil(count / itemsPerRow ),
      columnWidth = Math.floor(frameWidth/itemsPerRow);
  if(this.get('insertionOrientation') === SC.HORIZONTAL_ORIENTATION){
    var min = Math.floor(SC.minY(rect) / rowHeight) * itemsPerRow,
        max = Math.ceil(SC.maxY(rect) / rowHeight) * itemsPerRow;
    return SC.IndexSet.create(min, max-min);
  }else{
    var max = 0,
        min = count;
    for(var colIndex=0;colIndex<itemsPerRow;++colIndex){
      var colMinX = colIndex*columnWidth,
          colMaxX = colMinX + columnWidth;
      if( colMaxX > SC.minX(rect) || colMinX < SC.maxX(rect) ){
        min = Math.min(min,Math.floor(SC.minY(rect) / rowHeight) + (colIndex * rows));
        max = Math.max(max,Math.min(Math.ceil(SC.maxY(rect) / rowHeight) + (colIndex * rows), (colIndex * rows) + rows));
      }
    }
    return SC.IndexSet.create(min,max-min);
  }
},

What is basically going on here is we iterate over all of the columns and record the highest and lowest index of the items to display and return that in an IndexSet.

So the final code that monkey patches SC.GridView to make insertionOrientation work is:

SC.GridView.prototype.mixin({

  contentIndexesInRect: function(rect) {
    var rowHeight = this.get('rowHeight') || 48 ,
        content = this.get('content'),
        count = (content) ? content.get('length') : 0,
        frameWidth = this.get('clippingFrame').width,
        itemsPerRow = this.get('itemsPerRow'),
        rows = Math.ceil(count / itemsPerRow ),
        columnWidth = Math.floor(frameWidth/itemsPerRow);
    if(this.get('insertionOrientation') === SC.HORIZONTAL_ORIENTATION){
      var min = Math.floor(SC.minY(rect) / rowHeight) * itemsPerRow,
          max = Math.ceil(SC.maxY(rect) / rowHeight) * itemsPerRow;
      return SC.IndexSet.create(min, max-min);
    }else{
      var max = 0,
          min = count;
      for(var colIndex=0;colIndex<itemsPerRow;++colIndex){
        var colMinX = colIndex*columnWidth,
            colMaxX = colMinX + columnWidth;
        if( colMaxX > SC.minX(rect) || colMinX < SC.maxX(rect) ){
          min = Math.min(min,Math.floor(SC.minY(rect) / rowHeight) + (colIndex * rows));
          max = Math.max(max,Math.min(Math.ceil(SC.maxY(rect) / rowHeight) + (colIndex * rows), (colIndex * rows) + rows));
        }
      }
      return SC.IndexSet.create(min,max-min);
    }
  },
  
  layoutForContentIndex: function(contentIndex) {
    var rowHeight = this.get('rowHeight') || 48,
        content = this.get('content'),
        count = (content) ? content.get('length') : 0,
        frameWidth = this.get('clippingFrame').width,
        itemsPerRow = this.get('itemsPerRow'),
        rows = Math.ceil(count / itemsPerRow ),
        columnWidth = Math.floor(frameWidth/itemsPerRow),
        isHorizontal = this.get('insertionOrientation') === SC.HORIZONTAL_ORIENTATION,
        row = isHorizontal ? Math.floor(contentIndex / itemsPerRow) : contentIndex%rows,
        col = isHorizontal ? contentIndex - (itemsPerRow*row) : Math.floor(contentIndex/rows);
    return { 
      left: col * columnWidth,
      top: row * rowHeight,
      height: rowHeight,
      width: columnWidth
    };
  }
});

And since a lot of what we fixed is how the scrolling and resizing movement works, you can see a quick demo here:

(download)