Upgrading to Rails 4.2

I’ve never experienced a painless upgrade of a Rails application to a newer major release of Rails.  This could be for many reasons, and I’ll list a few of them here.

  • The application makes use of Rails internals not considered part of a public API, or worse yet has applied some freedom patches (ha!) to alter some behavior.
  • The application has a very long list of Gem dependencies, some of which have been abandoned or lack updates and do not function when used with newer versions of Rails.
  • Rails core developers tend to think little of making changes to public APIs, often renaming things for what looks like little reason.  Just look at the *_filter name changes to *_action.  What a waste of time.

But the point of this post is to highlight a few issues I ran into recently and did not find a lot of information about when searching for people with similar problems.

Minitest, ci_reporter, and parallel_tests

Well, isn’t this a fine mess.  For Ruby testing there are generally three options, test-unit, minitest, and rspec.  This application happens to have a very large test-unit style set of tests, and some of those take much too long to execute so parallel_tests was introduced in the past to help speed up CI builds on Jenkins.  I still prefer rspec style test declarations and for this and other reasons wanted to make use of minitest.  And finally, our CI environment wants to have nice reports so producing junit style output is helpful and ci_reporter helps with that.

When the application was using Rails 3.2, making use of the necessary updated gems was impossible due to dependencies declared in Rails itself on lower versions of various things, minitest in particular here if I recall.  But on Rails 4.2 finally this could be addressed.  Now parallel_tests creates many processes for you using either system() or open(“|#{cmd}”), but what it doesn’t do well is pass along environment variables with this second form.  If you study the source for parallel_tests you see it should be possible to pass along a hash in :env to the runner, but unfortunately it exposes no means of doing so, and doesn’t handle it for you automatically either, which is well, probably a bug.

Here’s the issue, ci_reporter wants to add “–ci-reporter” to TESTOPTS in your environment.  You’re instructed to use ci:setup:minitest for this purpose as a pre-requisite of the rake task that will run your test suite.  Since parallel_tests isn’t going to pass this along to the processes created with the open() call, what to do?  After spending a day debugging the process and deciding to patch parallel_tests to properly set the :env feature they have, I accidentally discovered a solution involving no patching.

namespace :ci do
  task :test do
    Rake::Task['ci:setup:minitest'].invoke
    Rake::Task['parallel:test'].invoke('^test/(unit|helpers|functional|integration)', '--ci-reporter')
  end
end

Passing –ci-reporter as an argument like this passes it down through the layers all the way to minitest and finally, junit style results appear! Problem solved.  And in case you didn’t catch it in release notes, Rails core decided all the test directory structure should be different too.  Ok, I’ll deal with that later.

Arel changes

Okay I admit it, I bought hard on making use of Arel when working with ActiveRecord.  Some of the code in this application refers to Arel::SqlLiteral however, and that appears to have been renamed to Arel::Node::SqlLiteral.  Simple fix.  No idea why the API was changed.

More obscure finder changes

Some effort was taken to prepare for upgrading this application by changing finders ahead of time.  But a couple of things were missed.

  # Original Rails 3.2
  comments = post.comments.find(:all, :order => 'position ASC')
  # Updated Rails 4.2
  comments = post.comments.all.order('position ASC')

  # Original Rails 3.2
  Post.all(:select => "id, name")
  # Updated Rails 4.2
  Post.all.select('id', 'name')

As the Rails release notes say, all() now returns a full ActiveRecord::Relation object instead of an Array.

NoMethodError: undefined method `call’ for :set_dirty_by_association:Symbol

That’s weird, a relatively trivial test that adds a new record to a has_and_belongs_to_many relationship with a new record is blowing up.  It turns out this application was making use of after_add_for_#{association} and after_remove_for_#{association}, which were not public parts of the ActiveRecord API, and thus the upgrade requires making a change here.

The correct approach to solving that problem would be to make use of the association callbacks as documented here, http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html.

Closing Thoughts

If you want a more painless experience upgrading a Rails application to a newer version of the framework.  Evaluate your dependencies to determine if you really need them, and their viability for newer versions of Ruby or Rails.  Reduce dependencies where you can.  If you’ve dipped down into private parts of anyone else’s code via freedom patching, or simply making use of non-public APIs then try very hard to stop doing that.  In some cases you may need to try submitting a patch upstream to have your worthwhile changes incorporated for the greater good into a newer version of your dependency.

 

Advertisements
Posted in rails | Leave a comment

Monitoring Rails with statsd and graphite

Assuming you’ve googled this topic at all, you are bound to have come across this article from 37signals, pssst-your-rails-application-has-a-secret-to-tell-you. You can basically build an initializer doing all the things in that post to get started sending data to statsd.

Statsd is a fairly small node.js application that listens for UDP requests on a configured port and collects them for a short period of time before then sending them over TCP to a graphite server. The upshot is if your statsd daemon goes down, your application continues to operate, thanks to UDP’s fire and forget nature.

If you want to save a little time, try checking out Nunes, a gem to automatically cause Rail’s builtin instrumentation to send data to statsd, see https://github.com/jnunemaker/nunes.

Graphite is an interesting beast worthy of some time reading the documentation to become familiar with the whisper database structure and to learn how you might tweak the configuration. A busy graphite server can be as taxing as a database server, making use of SSD drives, etc. Graphite is horizontally scalable, but documentation on scaling graphite in this manner is particularly lacking. This post by Richard Crowley on federated graphite is all I’ve found at the time of this writing.

Now go and instrument all the things.

Posted in rails | Tagged , , , | Leave a comment

Ruby connection pooling made easy

If you ever want to have a connection pool, you need not reinvent the wheel because Mike Perham has done all the work for you already. Browse on over to connection_pool on github.

Posted in Uncategorized | Tagged , | Leave a comment

Write your own custom RSpec formatter

Earlier this year I spent some time writing a custom RSpec formatter. The purpose in my case was to be able to automatically update a remote service with the results of testing some code. You can find that project here, jaribio_formatter.

The code for a basic no-op formatter is below. If you want to output a message when a group is started or finished, or an example starts or ends, or simply record failures to summarize at the end, all the hooks are well defined by RSpec though not well documented.

class MyRSpecFormatter

    attr_reader :output, :results, :example_group

    def initialize(output)
      @output = output || StringIO.new
      @results = {}
    end

    def start(example_count)
    end

    def example_group_started(example_group)
    end

    def example_group_finished(example_group)
    end

    def example_started(example)
    end

    def example_passed(example)
    end

    def example_pending(example)
    end

    def example_failed(example)
    end

    def message(message)
    end

    def stop
    end

    def start_dump()
    end

    def dump_pending()
    end

    def dump_failures()
    end

    def dump_summary(duration, example_count, failure_count, pending_count)
    end

    # no-op
    def seed(seed)
    end

    def close()
    end

end

You can also add your own custom configuration to RSpec. To do that, you need to call add_setting. An example from jaribio_formatter below. One more line of code calls this method when the formatter is required. The relevant source code to the jaribio_formatter is here if needed for reference.

    def self.configure()
      RSpec.configure do |c|
        c.add_setting :jaribio_url
        c.add_setting :jaribio_api_key
        c.add_setting :jaribio_plans, :default => []
        c.add_setting :jaribio_auto_create, :default => false
        c.add_setting :jaribio_timeout, :default => 5
      end
    end
Posted in Uncategorized | Tagged , , | 2 Comments

Ruby’s Singleton and Custom Rails Application Configuration

A relatively simple way to load some configuration once and safely, is through the use of the Singleton module provided with Ruby.

class FooConfig
  include Singleton  
  attr_reader :foo

  def initialize()
    @foo = true
  end
end

Accessing the instance is easy, call FooConfig.instance. This method is also thread-safe so no two threads are going to create/access a different instance. The example is trite, but imagine instead loading data from a remote service. Now combine this with Mike Perham’s Configuration for Rails, the Right Way.

MyApp::Application.config.foo = FooConfig.instance

There is really little reason to access the configuration via Rails.application.config.foo, except that this way you can change it for things like tests more easily.

Speaking of testing, if you find you need to reset a singleton for some reason, the following seems to do the trick, which comes from http://blog.ardes.com/2006/12/11/testing-singletons-with-ruby. Just paste into a file like spec/support/singleton_reset.rb (if using rspec) and require it in spec/spec_helper.rb

require 'singleton'

class <<Singleton
  def included_with_reset(klass)
    included_without_reset(klass)
    class <<klass
      def reset_instance
        Singleton.send :__init__, self
        self
      end
    end
  end
  alias_method :included_without_reset, :included
  alias_method :included, :included_with_reset
end
Posted in rails | Tagged , , , | 1 Comment

Require a Gemfile from a Gemfile

It is probably best I don’t bother explaining how or why I ended up wanting to reference an existing Gemfile from another Gemfile, but suffice to say it is something I sort of needed to do. After a bit of searching I came across the following post, http://madebynathan.com/2010/10/19/how-to-use-bundler-with-plugins-extensions/.

The most relevant bit is the use of eval like below in the Gemfile you want to include another Gemfile in.

filename="..."
eval(IO.read(filename), binding)
Posted in Uncategorized | Tagged , | Leave a comment

Ruby segfault with net/http and openssl

I was attempting to push a new gem this evening and ruby 1.9.3 segfaulted. The dump indicated openssl and net/http as the issue.

Pushing gem to https://rubygems.org...
/Users/bjones/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/net/http.rb:799: [BUG] Segmentation fault

After a bit of googling I found a nice post describing how to resolve the issue.

http://www.christopherirish.com/2011/09/02/ruby-1-9-2-segmentation-fault-and-openssl/

The key part for me was adding –with-openssl-dir=/opt/local to the end of the command when running `rvm install ruby-1.9.3`.

Posted in Uncategorized | Tagged | Leave a comment