Saturday, January 10, 2009

A Gem Is Born: Announcing Gemviz

Final (potentially) update: Now appearing on github:
git://github.com/davidrupp/gemviz.git

Update to first update: I realized that I did not specify a gem dependency on ruby-graphviz in my hoe-generated Rakefile. This is in place as of version 0.1.2.

To quote Groundskeeper Willie in The Simpsons - Treehouse of Horror V: "Och, I'm bad at this!"

We now return you to the first update.

Update: Peter Cooper correctly points out in his comment that Gemviz has a hard dependency on Graphviz, which isn't installed by default on Macs. I tend to take it for granted because it's one of the first things on my checklist to install when breaking in a new Mac. I use The Mac OS X Edition of Graphviz, as referenced on the Graphviz home page.

We now return you to the regularly scheduled original post.

Original Post: I can't really call myself a hotshot Ruby programmer until I release a RubyGem, now can I? Well, I still won't call myself a hotshot Ruby programmer yet...but, as of today -- and the release of versions 0.1.0 and 0.1.1 (for documentation purposes) of my new gem gemviz to rubyforge.org -- at least I'm one step closer!

Check it out, yo:
[sudo] gem install gemviz
Now you can do e.g.,
gemviz rails
(assuming you have Rails installed as a gem which, if you don't by now: huh?), which produces this graph on my system:


In fairness, I should disclose that Gemviz is only about 30 lines of code, and makes heavy use of other people's work to do its magic. Here's how it works:

* it scrapes the output of the system command gem dep gem_name --pipe.
* for each dependency thus listed, it adds that dependency to the list of dependent gems.
* it does the same recursively for each dependent gem until it finds that the current gem has no dependencies, or that some (recursively) dependent gem is not installed on the system.
* it builds a Graphviz graph from the graph so built.
* it runs the awesome tred utility on the resulting Graphviz graph, which translates the original messy graph into one that properly represents all of the transitive dependencies.

If you're not sure what that last bit is about, don't feel bad: I didn't either at first. So here's the picture that's worth a thousand words (give or take):

The difference is subtle, but telling. The second graph shows that the rails gem has dependencies on activesupport and actionpack. Which is true, as far as it goes, and is reflected in the output of the gem dep command thusly:
gem dep rails
Gem rails-2.2.2
rake (>= 0.8.3, runtime)
activesupport (= 2.2.2, runtime)
activerecord (= 2.2.2, runtime)
actionpack (= 2.2.2, runtime)
actionmailer (= 2.2.2, runtime)
activeresource (= 2.2.2, runtime)

But it turns out that gem dep doesn't tell the whole story. rails does depend on those other gems, but only through its dependencies on activerecord, actionmailer, and activeresource, as shown in the first graph. That's what the tred utility (part of the Graphviz package) does: reduce the second graph to its minimal equivalent representation in the first graph.

Here's another more fun example, for merb, prior to transitive reduction:


And post tred:

This implementation owes much to the existing DepGraph, especially its use of Graphviz and tred. I basically just extracted the simple use case of graphing a gem at a time, and neglected to implement a bunch of DepGraph's options. For instance, the output of gemviz is always a single .png file per requested gem, named after the gem and placed in the current directory.

Keep in mind that this is a very simple initial release. Please let me know if you find it useful, and especially what improvements you can think of.

17 comments:

Peter Cooper said...

A note on resolving the graphviz dependency would be useful. There are a few ways to have a graphviz library working on Ruby and I don't fancy trying them all :-) What should I be doing to get this to work?

David Rupp said...

D'oh! Sorry, Peter -- I just take Graphviz for granted. Thanks for pointing out this deficiency.

Whenever I get a new Mac, one of the first things on my checklist is to download and install Graphviz for Mac OS X. It just downloads as a .dmg and installs very nicely.

Greg said...

Hi,

Nice gem. Thanks for using ruby-graphviz !

Greg

bryanl said...

Peter, I just installed the ruby-graphviz gem.

David Rupp said...

@Greg: It's my pleasure, believe me. :-)

@bryanl: Good call. As I updated above, that should now be installed as a dependency when gemviz is installed. My bad -- this is my first time using hoe.

technicalpickles said...

I got a failure about not being able to require 'graphviz'. Looks like a missing gem dependency on 'ruby-graphviz'.

David Rupp said...

@technicalpickles: Try upgrading to the latest, version 0.1.2. I added that very dependency. Thanks for the heads-up, and please let me know if upgrading doesn't fix it for you.

bryanl said...

Could you put it up on github? i'm in a forkin' mood

David Rupp said...

Done and done. git://github.com/davidrupp/gemviz.git

Michael said...

Am I missing something simple?

Thanks!

working$ gem list|egrep "viz|rails"
gemviz (0.1.2)
rails (2.1.0)
ruby-graphviz (0.9.0)
working$ gemviz rails
creating ./rails.png
/opt/local/lib/ruby/gems/1.8/gems/gemviz-0.1.2/bin/../lib/gemviz.rb:34:in `delete': No such file or directory - temp.dot (Errno::ENOENT)
from /opt/local/lib/ruby/gems/1.8/gems/gemviz-0.1.2/bin/../lib/gemviz.rb:34
from /opt/local/lib/ruby/gems/1.8/gems/gemviz-0.1.2/bin/../lib/gemviz.rb:15:in `each'
from /opt/local/lib/ruby/gems/1.8/gems/gemviz-0.1.2/bin/../lib/gemviz.rb:15
from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `require'
from /opt/local/lib/ruby/gems/1.8/gems/gemviz-0.1.2/bin/gemviz:11
from /opt/local/bin/gemviz:19:in `load'
from /opt/local/bin/gemviz:19

David Rupp said...

@Michael: 'temp.dot' is an intermediate file that has the Graphviz representation of the un-transitive-reduced graph. It gets transformed by the Graphviz 'tred' and 'dot' commands into gem_name.png and then deleted. This all happens in a begin/ensure block, and presumes that nothing will happen to prevent temp.dot from being created in the first place.

But it definitely looks like something is preventing temp.dot from being created in your setup. Do you have Graphviz (not just ruby-graphviz) installed? Take a look at the Graphviz home page for an installer for your platform.

The only other thing I can think of right now is that you're running the 'gemviz rails' command in a directory to which you don't have write permissions, which could prevent the file from being created to begin with.

Please do let me know if neither of these problems seems to be the case.

David Rupp said...

@Michael: To follow up, I was able to recreate the error you describe by running gemviz in a directory to which I did not have write permission. Try changing to your home directory or someplace else where you know you have write permission and running gemviz from there.

Michael said...

Sorry, just getting back to this now. Thanks for the quick replies!

So, it turned out to be the Graphviz install. I had installed this MacOSX version (linked from graphviz.org). Command line tools didn't work with this (dot, tred, etc). What made those work for me was removing that version and installing from macports instead ('port install graphviz').

Strange thing now is I get png's with all objects on the same level - no hierarchy and no connecting lines. Hmmm...It's prolly my setup, sorry for the noise.

Michael said...

Nevermind...

sudo port uninstall graphviz
sudo port install graphviz-devel

...fixed it. Now I can say: 'nice gem'!

David Rupp said...

@Michael: Excellent news. Thanks for doing the detective work on your setup and for posting your experience here. Happy gemviz-ing!

Kieran P said...

Cool gem. However, got this error:

$ gemviz rails
creating ./rails.png
Error: Could not find/open font : Times-Roman

The image was made but looks terrible. Any ideas?

David Rupp said...

@Kieran P: What platform are you running on that doesn't have a Times-Roman font installed?! Fonts (along with the rest of the actual graphic representation) are handled/required by Graphviz. I would suggest finding and installing a Times Roman font and doing whatever is necessary to make your Graphviz installation aware of it. Other than that, I'm not sure what to suggest. Please do let me know how it goes.