Posts tagged with: "git"

In some cases you may have the need to run multiple Rails applications with shared functionality. While Rails 3 promises to bring “mounting apps in apps” and the ability to make the whole process simple, for now we’re stuck in the real world. However, it is possible to share components. This post will walk you through how to set up shared components that live in multiple Rails applications at once and even run specs properly.

A few notes before we begin:

• This post focuses on models but the same could be applied to controllers, views, and more.
• I use RSpec so the testing solutions come from that perspective.
• My applications share a database, so I keep all of the migrations in one app and load from a duplicated schema.rb in the other.

Setting Up Your Application

First, you’ll need to create your shared directory. I’m mounting all shared components to my RAILS_ROOT/shared directory. So if I have app1 and app2 then I’ll do this in app1:

mkdir shared
cd shared
touch README
git init
git add .
git commit -m "Initial import of shared repository."
git remote add origin git@some.git:repository/online.git
git push

At this point all we’ve actually done is create a git repository inside our app1 application in the shared directory and pushed it (with an empty README file) to a remote git repository.

What we need to do now is actually delete this git repository and re-add it as a submodule from the remote source (this is from the root of app1 again):

git submodule add git@some.git:repository/online.git shared
git submodule init
git add .
git commit -m "Added shared directory as submodule"

What we did here is add the repository as a submodule using the git submodule command. We then ran git submodule init to update our .git/config to reflect the new submodule.
Finally we committed our changes.

So now we have a submodule living in our application directory, but right now it’s empty and Rails doesn’t know or care about it! Next we’ll set up Rails to make use of the components in the shared directory.

Setting Up Rails To Use Shared Components

Lets say that we’re going to create a shared model, SharedModel. We need to put it in the shared directory but still have it picked up by Rails’s lazy class loader. So in config/environment.rb you
will need to add the following:

config.load_paths += %W( #{RAILS_ROOT}/shared/models )

This tells Rails to look for classes in the shared models path. Now we create our model by creating shared/models/shared_model.rb:

class SharedModel < ActiveRecord::Base

end

When creating shared components I tend not to use Rails’s model generator, preferring instead to create the class by hand and generate a migration separately in my migration-having app.

This is actually all you need to do to get your shared components running in Rails. Next we’ll set up app2 to use the same code!

Setting Up The Second Application

To set up the second application, you basically need to simply repeat the same steps you did for the first application starting with git submodule add. So that would be:

• Add the submodule and initialize it.
• Add the shared directory to the load paths in config/environment.rb

As a note, if you are doing this to an existing application with multiple developers, other developers will simply need to pull from your main application once you’ve pushed it to a remote and run:

git submodule init
git submodule update

To get the latest revision of the submodule locally for themselves.

Changing Shared Components

To modify shared components, just change them like you would normal files in your repository. The only difference is that when you want to commit changes you will need to do so from the shared directory, push and then make a new commit from the root directory. This way you are telling the root repository to track a new revision of the
submodule.

Testing Shared Components

So just because we’re sharing components doesn’t mean that we want to abandon TDD, does it? In fact, it brings up a somewhat interesting problem. I want to have specs that I can run for the shared components that can run in both applications, in fact I want these specs to run in both applications to make sure that the shared components aren’t
having any compatibility issues. While this isn’t extremely difficult to set up, it’s not easy, either.

The first step is to create a spec directory inside your shared submodule, and create a spec_helper.rb that simply points back out to the root application’s.

In shared/spec/spec_helper.rb:

require File.dirname(__FILE__) + '/../../spec/spec_helper'

We also need to create a pending spec for our SharedModel to make sure that these are running. In shared/spec/models/shared_model_spec.rb:

require File.dirname(__FILE__) + '/../spec_helper'

describe SharedModel do
  it 'should have a pending spec'
end

The good news is that if you run autospec from your shared directory, you should be able to see your pending spec run (you will need to create a second spec.opts file in the shared/spec directory for this to use your preferred options). You should push out the changes in your shared directory and get all of your applications up to date. The bad news is that this is the only place your specs will run at the moment. Let’s change that for the better.

Note: You will need to perform the following steps in each of your Rails apps that use the shared components.

First to get your specs running with rake spec we will need to modify the task found in lib/tasks/rspec.rake by performing the following substitution:

# When you see this...
FileList['spec/**/*/*_spec.rb']

# Change it to this...
FileList['spec/**/*/*_spec.rb', 'shared/spec/**/*/*_spec.rb']

That takes care of the spec rake tasks, but there’s still the matter of autotesting from your application root. This requires a custom .autotest file in your application root that looks like this:

Autotest.add_hook :initialize do |autotest|
  autotest.add_mapping(%r%^shared/models/.*\.rb$%) { |_, m|
    autotest.files_matching %r%^shared/spec/.*_spec.rb$%
  }
end

This will automatically pick up changes to your shared models and re-run the specs for all of your shared models when they change. You could get more granular than this, but that’s a topic for another day.

Wrapping Up

Now that all of this is done you should be able to freely develop models in the shared directory and have them completely integerated into your normal development workflow. It’s quite a bit of work, but once you iron out the kinks it runs beautifully!

Way back in the day we used to use Campfire and Subversion. Time happened, it’s the future, and now we use Present.ly and Git. Remember how those commit messages would just show up in Campfire whenever anyone did an svn commit -m ‘whatever’? Wouldn’t it be nice to be able to publish Git commit messages to Present.ly just like the old days with Campfire and Subversion?

We can do it. We’ve been doing it. We’re doing it right now.

The first thing you’ll need to know is that Git has hooks. Lots of them. They live in the hidden ‘.git/hooks’ folder in your project. Their names are relatively descriptive and tell you when these hooks will fire. The one we’re particularly interested in right now is ‘.git/hooks/post-commit’.

Open up that file, there’s stuff in there, delete it and add the following hotness instead:

Once you’ve updated your account, username, password, and the group to which you want to send your messages – you need to make this bad man active so it will fire at your will. The zombies will never see it coming:

chmod +x .git/hooks/post-commit

Keep in mind that none of these hooks will fire unless they are marked as executable. This will bite you; it did me, and I’m still recovering. It’s okay son, that’s how we earn our stripes.

Make some changes, queue ‘em up, hit commit, and BLAM: you’re knee deep in dead zombies and commit messages! But how do you kill something that’s already dead…

A few days ago, Chris asked on our Intridea instance of Present.ly if anyone knew how to ask git how much code was written by a certain author. This kind of request came up fairly regularly at my previous job, and as far as I know, there isn't a built-in git command for it, so I took a few minutes and came up with something.

First of all, "git log --numstat" will give us output like this

commit c82826f6616a68b84303176c8a702e389dfc7be5
Author: paul 
Date:   Tue Feb 10 22:18:34 2009 +0000

    Refactor for increased speed

53      41      app/controllers/degrees_controller.rb
1       1       app/views/degrees/demographics.xml.builder
2       14      spec/controllers/degrees_controller_spec.rb
79      53      spec/views/degrees/demographics_xml_builder_spec.rb


commit 93bf620d210afde1835133f0feeb5ba8a5076d3c
Author: paul 
Date:   Tue Feb 10 22:16:01 2009 +0000

    Improving specs for degrees

1       1       app/controllers/degrees_controller.rb
3       1       db/schema.rb
44      0       spec/controllers/degrees_controller_spec.rb

Which is very strongly patterned. When I see strongly patterned output in the console, the first thing I think of is awk. Write ups for awk abound (my personal favorite is here) but the basics are simple: you define a series of [pattern, action] pairs. When the awk script is run, each line of input is compared to the patterns in turn, and when a pattern matches, the associated action is executed.

Let's look at the evolution of this "score authorship" script as an example. Save this as "git_score.awk" and then (from inside a git repo) run "git log --numstat | awk -f /path/to/git_score.awk"

/^Author:/ {
   author           = $2
   commits[author] += 1
}

END {
   for (author in commits) {
      print author, commits[author]
   }
}

Patterns are delimited by forward slashes, and actions are surrounded by curly braces. The special pattern END matches once the end of input has been reached. The pattern /^Author:/ matches the lines beginning with "Author:", and the variable $2 is automatically set by awk to the second word on that line. Arrays in awk can be autovivified (they spring into existence when referenced), and they default to zero. Also, awk, like PHP, uses associative arrays throughout (in ruby/perl parlance, hashes and arrays are the same type). Running the above file should give you a listing of authors who contributed to the repo and a count of the number of commits that author made.

Convinced that this would work, I continued:

/^Author:/ {
   author           = $2
   commits[author] += 1
}

/^[0-9]/ {
   more[author] += $1
   less[author] += $2
}

END {
   for (author in commits) {
      print author, "inserted", more[author], "and deleted", less[author], "lines over", commits[author], "commits"
   }
}

Now we count changes by adding a pattern to match any lines that begin with a digit.

1       1       app/controllers/degrees_controller.rb
3       1       db/schema.rb
44      0       spec/controllers/degrees_controller_spec.rb

Lines like the above, which correlate to lines inserted, lines deleted, path/to/file/changed. We save those to a running tally of changes per author by using two more autovivicated arrays, printing the totals when we finish.

At this point, Chris mentioned that he'd like to ignore counting contributions to files in the framework directory. That's accomplished easily enough, and while we're at it, let's count files as well as insertions/deletions.

/^Author:/ {
   author           = $2
   commits[author] += 1
}

/^[0-9]+ +[0-9]+ +framework/ { next }


/^[0-9]/ {
   more[author] += $1
   less[author] += $2
   file[author] += 1
}

END {
   for (author in commits) {
      print author, "inserted", more[author], "and deleted", less[author], "lines over", file[author], "files"
   }
}

It's important that the exclusion pattern come BEFORE the inclusion pattern. The "next" action will skip matching any further patterns and immediately start over on the next line of input. If we wanted to also skip files in vendor, we could make the pattern

/^[0-9]+ +[0-9]+ +framework|vendor/

One more problem: Chris asked for percentages, and at this point we're outputting counts. This may not be the neatest way to convert between the two, but it is an easy one (unless you have an author named "tot"!)

/^Author:/ {
   author           = $2
   commits[author] += 1
   commits["tot"]  += 1
}

/^[0-9].*framework/ { next }

/^[0-9]/ {
   more[author] += $1
   less[author] += $2
   file[author] += 1

   more["tot"]  += $1
   less["tot"]  += $2
   file["tot"]  += 1
}

END {
   for (author in commits) {
      if (author != "tot") {
         more[author]    = more[author] / more["tot"] * 100
         less[author]    = less[author] / less["tot"] * 100
         file[author]    = file[author] / file["tot"] * 100
         commits[author] = commits[author] / commits["tot"] * 100

         printf "%s added %.0f%% and removed %.0f%% of the lines accounting for %.0f%% of the files changed and %.0f%% of the commits\n", author, more[author], less[author], file[author], commits[author]
      }
   }
}

I showed this to Chris (elapsed time ~30 minutes). He was satisfied, but Simon jokingly asked for a more machine-friendly output (CSV or YAML). I considered switching to ruby (the .to_yaml method was tempting) but decided that it wasn't necessary yet, since YAML is such a simple markup.

/^Author:/ {
   author           = $2
   commits[author] += 1
   commits["tot"]  += 1
}

/^[0-9]+ +[0-9]+ +vendor|framework/ { next }

/^[0-9]/ {
   more[author] += $1
   less[author] += $2
   file[author] += 1

   more["tot"]  += $1
   less["tot"]  += $2
   file["tot"]  += 1
}

END {
   for (author in commits) {
      if (author != "tot") {
         more[author]    = more[author] / more["tot"] * 100
         less[author]    = less[author] / less["tot"] * 100
         file[author]    = file[author] / file["tot"] * 100
         commits[author] = commits[author] / commits["tot"] * 100

         printf "%s:\n  insertions: %.0f%%\n  deletions: %.0f%%\n  files: %.0f%%\n  commits: %.0f%%\n", author, more[author], less[author], file[author], commits[author]
      }
   }
}

There you have it. Authorship percentages for a git repository. Just save the above to a file and run it as "git log --numstat | awk -f /path/to/the/file/you/just/saved". Or create an alias in your .shell_rc file.

I've been using Git for almost a year now, but I didn't really start using Git until recently when I began working for Intridea.  Once I started using Git on a daily basis, dealing with multiple projects, and multiple branches per project, I would occasionally make the mistake of changing code on the wrong branch.  While annoying, it was easily fixed by stashing the changes and applying the stash to the proper branch.

As much as I love git stash, this began to get old, and constantly hitting up git status to check the current branch wasn't cutting it.  After a bit of googling, I found this.  It describes how to add the current branch name and its clean/dirty status to you terminal prompt.

Just add this to your .bash_profile: 

function parse_git_dirty {
  [[ $(git status 2> /dev/null | tail -n1) != "nothing to commit (working directory clean)" ]] && echo "*"
}
function parse_git_branch {
  git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e "s/* \(.*\)/[\1$(parse_git_dirty)]/"
}

export PS1='\u:\[\033[31;40m\]\w\[\033[0;33m\]$(parse_git_branch)\[\e[0m\]$ ' 

And you should end up with something that looks like this:

As you can see, I like to use a bit of color to help things stand out.

So far this has been immensely helpful.  With this info at a glance, I always know where I am and how I last left things.

We here at Intridea are avid users of Unfuddle as it’s by far the most feature-filled-while-remaining-simple solution out there for software project management. Unfuddle comes with Subversion hosting, but it’s harder and harder to ignore the strengths of Git as a SCM solution. Not one to sit idly by when there’s hackery to be done, I put together a rough tool for anyone who has a GitHub account that provides a webhook to automatically create Unfuddle changesets.

All you need to do is grab the bridge code from It’s GitHub Home like so:

git clone git://github.com/mbleigh/github-unfuddle.git