Metamagical class variables in Ruby part 2

In my previous post Metamagical class variables in Ruby I described an oddity of Ruby causing a problem in a plugin I was writing at the time. It was about class variables that were set by dynamically defined methods. After a lot of asking around the general advise was just to avoid class variables, but I still couldn’t sleep.

I just had to figure it out, so I took another look at it and experimented away in the metamagical world of Ruby. And that has given me some new insights on the odd behaviour, of which I’m till not sure whether it’s a feature or a quirk.
It turns out that the manner in which a method that sets a class variable has been defined affects the class the class variable is actually stored in. This can be the class you intended to set it on, its metaclass or both. And to make it a little more complicated this also depends on whether the class variable had already been initialized or not.

This script shows several ways of adding a setter method to a class and for each of those outputs whether it has set the class variable on the class itself or its metaclass after calling it. Note that no class variables are being initialized anywhere before calling the method:

On class? On metaclass?
1. def within class definition yes no
2. def outside class definition yes yes
3. class_eval yes yes
4. instance_eval yes yes
5. define_method on metaclass yes yes
6. Include module into metaclass no yes
7. Extend with module no yes

Methods 2 to 5 set the class variable on both the class itself and its metaclass, however if the class variable has already been initialized on the class itself, calling the method WILL NOT override the currently set class variable but WILL set a new class variable only on the metaclass!

Now lets go back to the original case which caused problems:


module ClassMethods
  def authenticate_with_header(header, options = {})
    #...
    @@configuration = {}
    cattr_reader :configuration
    #...
  end
end

ActionController::Base is extended with this module, making the authenticate_with_header method available from within any controller’s class definition. Calling this method will initialize the class variable @@configuration. Now according to the results of our previous experiment calling a setter method which was mixed in by means of extending with a module initializes the class variable only on the metaclass. Now if you take a look at cattr_reader’s implementation you can see it uses class_eval to evaluate code that does a number of things: 1. it checks whether a class variable with the specified name already exists, 2. if it doesn’t it initializes the class variable with a value of nil and 3. it creates a getter method. As we’ve seen, code mixed in using class_eval will look on the class itself for class variables, i.e. not on its metaclass. This means calling cattr_reader will initialize another class variable on the controller class itself (step 2). Now there are two class variables, on both the class itself and its metaclass being respectively nil and {} (an empty hash). The getter method will look on the class itself, find nil there and return that. The class variable on the metaclass just remains there, orphaned, never to be requested again…

Congratulations if you’re still with me at this point. Your efforts haven’t been pointless, keep up just a little longer, because next I will tell you how to get around this ‘problem’. To avoid confusion we need to make sure we’re never setting class variables on metaclasses. Fortunately there’s a method that lets you set class variables (opposed to a regular assignment using ‘=’), and it will always do this on the bottom level class for you. It is called class_variable_set. The following example illustrates its difference with normal assignment:


class Foo
  def self.metaclass
    class << self; return self; end
  end
end

module Setters
  def set_a
    @@a = true
  end
  def set_b
    class_variable_set "@@b", true
  end
end

Foo.extend Setters

Foo.set_a
Foo.set_b

Foo.class_variables # => ["@@b"]
Foo.metaclass.class_variables # => ["@@a"]

The advise given in my previous post was to avoid the usage of class variables and I’d say that still holds true. However, if that is out of your control or when class variables are really the only way to solve your problem then be sure to use class_variable_set and class_variable_get, especially when you’re dealing with dynamically added methods like in Rails plugins.

Posted September 1st, 2009 by Sjoerd Andringa
 

Comments

 
  1. “Methods 2 to 5 set the class variable on both the class itself and its metaclass, however if the class variable has already been initialized on the class itself, calling the method WILL NOT override the currently set class variable but WILL set a new class variable only on the metaclass!”

    This is a bit misleading. When getting a class variable, the interpreter walks up the inheritance chain, including meta-classes (that in the *implementation* gets inserted between the “real” class and its parent) and picks the first matching class variable.

    So when the same class variable show up in both the metaclass and the class, that is because the class variable in question is actually defined on the metaclass and *not* the class. It is visible in the class only because the class does not have a corresponding class variable of its own (or, as it turns out, sometimes it won’t be visible unless you explicitly use the metaclass).

    As for the rest, some of them are simple to figure out:

    For #3, the method is actually defined on *Class*, not on the metaclass of Foo. But since Class sits above both Foo and the metaclass of Foo in the inheritance chain, it looks like it’s present there. I think that is what happens for #2, #4 and #5 as well - if you extend your example to get the class variable from Object and Class as well, you see that the class variables for #2 through #5 are all defined on them. I haven’t actually looked in detail at how MRI defines class variables so I’m not 100% sure on why it behaves like that for 2,4 and 5 though.

    #6 and #7 set the class variable of the *module*. Think of @@foo as referring to a variable defined on the class in the scope surrounding the “def”. In this case that’s the module, not the class that the method later gets included into. I agree it’s still confusing, though,

    Posted September 1st, 2009 16:56 by Vidar Hokstad
  1. I just found the following comment by Matz, not sure if its helpful though:

    Class variables look up follows the inheritance hierarchy starting
    from the nearest class or module. In this example, look up for @@x in
    meth starts from C (nearest class), then A. The reason is to make
    scope resolution as static as possible.

    matz.

    Posted September 7th, 2009 19:23 by Ygor
  1. Thanks for your explanation Vidar. It has triggered me to dig a little deeper. Below is an elaboration of my findings…

    You say that the interpreter walks up the inheritance chain and you are right about that, however Foo does not inherit from its metaclass, instead it’s an instance of its metaclass. That means the interpreter will not look there when requesting Foo’s class variables. (Instead it will look in Foo’s superclass, which is Object.) The following seems to prove this, but please correct me if I’m overseeing something:

    
    >> class Foo
    >>   def self.metaclass
    >>     class < < self; return self; end
    >>   end
    >> end
    => nil
    >> Foo.class_variables
    => []
    >> Foo.metaclass.send :class_variable_set, :@@a, true
    => true
    >> Object.send :class_variable_set, :@@b, true
    => true
    >> Foo.class_variables
    => ["@@b"]
    

    This implies that when the class variable shows up on both the class itself and its metaclass what is going on must be either:

    1. The class variable is initialized in both places (Foo itself and its metaclass) so there are two actual class variables. Or…
    2. The class variable has been initialized on a class from which both Foo and its metaclass inherit it.

    When I extend the example code to print the class variables on Object like you suggested we can see that for examples 2 to 5 the class variable is also visible in Object. That tells us that for those cases the class variable wasn’t really set on both Foo and its metaclass, but only on Object (i.e. the latter of the 2 options above). So it turned out my previous hypothesis was false like you say. However you also state that the class variable is set on Class, but the only reason it is visible there is because Class inherits from Object (not the other way around). This means that the class variable is also visible in pretty much any other class, e.g. String.

    So then why is the class variable set on Object instead of Foo?
    I believe that in none of examples 2 to 5 a new scope is opened (for class variables anyway). This means the class variable will be set on the current scope, which in case of my example (and in irb sessions as well) not surprisingly, is Object:

    
    >> self.class
    => Object
    

    So the assignment has to be evaluated in the scope of the class you’re trying to set it on. By passing in a string into class_eval instead of using a block the assignment wíll actually be evaluated within the scope of Foo:

    
    >> class Foo; end
    => nil
    >> Foo.class_eval do
    ?>   def self.set_a
    >>     @@a = true
    >>   end
    >> end
    => nil
    >> Foo.class_eval "def self.set_b; @@b = true; end"
    => nil
    >> Foo.set_a
    => true
    >> Foo.set_b
    => true
    >> Object.class_variables
    => ["@@a"]
    >> Foo.class_variables
    => ["@@b", "@@a"]
    
    

    So you are right about examples 6 and 7 in which a module is used to mix in the setter methods. The assignment is evaluated in the scope of the module hence the class variable is set on the module:

    
    >> class Foo; end
    => nil
    >> module SetA
    >>   def set_a
    >>     @@a = true
    >>   end
    >> end
    => nil
    >> Foo.extend SetA
    => Foo
    >> Foo.set_a
    => true
    >> SetA.class_variables
    => ["@@a"]
    >> Object.class_variables
    => []
    

    The reason why, in case of extending Foo with a module, the class variable is also visible on Foo’s metaclass is this: When extending a class with a module what you’re really doing is including the module into that class’ metaclass. And when you’re including a module into a class a ‘a sort of special secret superclass’ (p.51of ‘Design Patterns in Ruby’ - Russ Olsen) is inserted in its inheritance chain. This class is a direct pointer to the module and is referenced when looking for a class variable. So when extending Foo with SetA like above, a hidden class that points to SetA is inserted in the inheritance chain of Foo’s metaclass.

    
    >> # continuing from above
    >> class Object
    >>   def self.metaclass
    >>     class < < self; return self; end
    >>   end
    >> end
    => nil
    >> Foo.metaclass.class_variables
    => ["@@a"]
    
    Posted September 8th, 2009 08:56 by Sjoerd Andringa
  1. [...] Innovation Factory Dev Blog » Metamagical class variables in Ruby part 2 [...]