Assuming you’ve googled this topic at all, you are bound to have come across this article from 37signals, pssst-your-rails-application-has-a-secret-to-tell-you. You can basically build an initializer doing all the things in that post to get started sending data to statsd.
Statsd is a fairly small node.js application that listens for UDP requests on a configured port and collects them for a short period of time before then sending them over TCP to a graphite server. The upshot is if your statsd daemon goes down, your application continues to operate, thanks to UDP’s fire and forget nature.
Graphite is an interesting beast worthy of some time reading the documentation to become familiar with the whisper database structure and to learn how you might tweak the configuration. A busy graphite server can be as taxing as a database server, making use of SSD drives, etc. Graphite is horizontally scalable, but documentation on scaling graphite in this manner is particularly lacking. This post by Richard Crowley on federated graphite is all I’ve found at the time of this writing.
Now go and instrument all the things.