A Deep Dive Into the Ruby Map Method

ruby

I recently took a look at how the ruby puts method works. Continuing with the same theme, let’s take a look at the maps method from the Enumerable mixin.

The Enumerable mixin is one of the greatest parts of the ruby programming language. Anything that mixes in Enumerable gets access to 51 very useful and commonly used methods useful for working with, well, things that can be enumerated. The mixin is used in common data structures: both Array and Hash include Enumerable. Many other libraries implement some subset of Enumerable; for example ActiveRecord::Relation which is used by Rails.

The Method

What does the map method do? The method signature, map {|obj| block } -> array, tells us that map takes a block and returns an array. The documentation tells us that the returned array is constructed by running the block on each element of the object that receives map, and adding the return of the block to the array. Hence:

1
2
[1, 2, 3, 4, 5].map{|x| x * 2} #=> [2, 4, 6, 8, 10]
{:one => 1, :two => 2}.map{|k, v| k.to_s + " " + v.to_s} #=> ["one 1", "two 2"]

A common pattern is to call a method on each object in the collection and use the return value of that method. As a shortcut, you can do this by passing “&:method_name” instead of an explicit block.

1
[1, 2, 3].map(&:to_s) #=> ["1", "2", "3"]

Though it is most common to pass a block using the curly brace ({}) syntax to map, it is perfectly valid to pass a ruby block in any of the other normal ways. Hence, the following is perfectly valid:

1
2
3
4
["jim", "KIM", "rOb"].map do |name|
  name.downcase.capitalize
end
#=> ["Jim", "Kim", "Rob"]

You can also pass the block using a Proc object:

1
2
proc = Proc.new {|arg| arg.to_s}
[1, 2, 3].map(&proc) #=> ["1", "2", "3"]

Examples

How can we use the map method in practice?

One of the best ways to use map is to try and refactor your code in a more functional style. If you are used to programming in lower level languages, say Java, you might be used to writing code in this pattern:

1
2
3
4
new_array = []
old_array.each do |element|
  new_array.push(element.do_something)
end

This code can be made much more concise with map:

1
old_array.map{|x| x.do_something}

A good rule of thumb is to remember that nothing can be returned from calling .each, so it is best used for cases where your code to cause side effects like being saved to a database or printed to the screen. If your .each block isn’t doing either of those things, it probably can be replaced with an alternative method.

The map method really shines when combined with a couple more enumerable methods. Suppose we have a Person object that has an age. Here, we calculate the variance of the ages in one line:

1
2
mean = people.map(&:age).inject(:+) / people.count
(people.map{|x| (x.age - mean) **2}).inject(:+)/people.count

This isn’t good style, but it shows the power that map gives you to write concise, functional code. Note that here we use map twice: the first time, it is part of people.map(&:age).inject(:+), and is used to extract the ages from the set of people before they are summed. The outer map block is used to get the squared deviation of each age from the mean, which is required to calculate the variance.

How is Map Defined?

Like most of the core ruby libraries, map is actually defined directly in C.

To get the hang of using map, look at some of your each loops and see if you can replace it with a call to map. On line 2025 of enumerator.c, we see that map is actually defined exactly the same was as collect, which is a synonym function. It refers to the C function lazy_map - happily, ruby tries to keep enumerables lazy when it can - which in turn delegates to an internal method lazy_set_method and calls lazy_map_func. The lazy_set_method takes care of associating the code block you pass with the lazy enumerator; lazy_map_func actually calls the block on each element1.

The key insight here is that ruby tries to keep its enumeration lazy by default, which while making the underling C code much harder to understand, provides some great benefits for the end user. You can code with confidence knowing that map will keep your lazy sequence lazy!

Happy coding!


  1. I think. Still learning this C stuff.

A Deep Dive Into the Ruby Puts Method

ruby

Most programmers are familiar with the traidition of the “Hello World” program. New programmers use it as their first program. Experienced programmers use it to prove that their new project can run correctly.

Ruby makes “Hello World” trivially easy:

1
puts "Hello World"

The puts method is one of the workhorses of the ruby core library. It is the most common of the methods in the IO module. But what is actually going on here?

The method signature of puts is as follows:

1
puts(obj,...) -> nil

That is, puts takes zero or more objects and returns nil.

Puts is defined in the IO core class1. This class ships with the core of ruby and provides ‘the basis for all input and output in ruby’ 2.

When you call puts without an explicit context, as with the hello world program, the method call is delegated through the main object that exists at the top level of all ruby programs to an IO object that is bound to the standard input and standard output. Ruby provides constant objects STDOUT, STDIN, and STDERR by default, which are all IO objects bound to the appropriate ‘files’3.

Hence, the two lines are functionally equivalent:

1
2
puts "Hello World"
STDOUT.puts "Hello World!"

Where is puts defined? Like many of the core ruby classes, IO is defined directly in the underlying C code. The current source of the IO code can be found in io.c.

puts is actually defined in two places; once using the rb_define_method C function to associate the rb_io_puts C function with the puts method on the ruby IO class, found around line 12187 in io.c. puts is also defined with the rb_define_global_function C function on line 12106, to associate a special version of puts, rb_f_puts, with the ruby Kernel class. These methods provide the actual functionality of puts, which is a thin wrapper around the more basic write method.

Puts provides the following functionality:

The last bit means that calling puts on nested arrays will print each element of each array with .to_s on a separate line. To print an array in the format to which you are accustomed to defining them, use .to_s on the array explicitly.

The fact that puts will not add a newline if the string to be printed already ends in a newline is worth noting. Because of this functionality, the following lines are equivalent, which can be an issue when debugging:

1
2
puts "Hello"
puts "Hello\n"

Happy Coding!


  1. http://ruby-doc.org/core-2.2.0/IO.html#method-i-puts

  2. http://ruby-doc.org/core-2.2.0/IO.html

  3. Philosophically, in Unix, ‘everything is a file’. So writing to or reading from the terminal is just reading and writing to a file that is connected to the terminal.

Devise, OmniAuth, Facebook and Rails 4.2

rails

I work on a lot of ruby on rails web applications. Usally, to get the ball rolling, we start with just a regular email and password authentication system using devise. This is a great way to get started since it is well supported, familiar, and allows us to conveniently create and log in with various test accounts.

However, in 2015, nearly every web application will need to have authentication via some OAuth21 service. Several large networks allow this - Google, Facebook, GitHub, Twitter, and LinkedIn are the most popular, though there are strong arguments that developers really should support systems like OpenID as well. Thankfully, getting authentication going from a variety of OAuth providers is pretty easy with Devise and Omniauth. However, there are a few gotchas.

Initial Setup of Omniauth-Facebook

Assuming you already have Devise installed and set up, you’ll need to add the omniauth gem to your gemfile, and a gem for one of the omniauth authentication providers. For this example, we’ll use omniauth-facebook.

Currently, Facebook is pushing developers to not use the traditional redirect-based authentication system, and instead use Facebook’s JavaScript SDK 2. Setting this system up is a bit different than a classic OAuth setup, and was new to me.

You’ll need to create an app on developers.facebook.com (you’ll need a Facebook account). When you do so, select ‘website’ as the app type. The nice onboarding page will prompt you to ‘Setup the Facebook SDK for JavaScript’; I copied this code into a partial so that it does not pollute my views. It’ll then prompt you to add a like button - this is a reasonably good test of whether the Facebook Javascript SDK is working properly. Don’t forget that you will likely have to add an override to your ad blockers.

Once this is going, you can follow the instructions that Facebook provides on setting up the JavaScript SDK and login.

Gotcha, Turbolinks!

You will find that Rails’ use of turbolinks prevents the login button from being set up correctly when the page is loaded; the JavaScript you included from Facebook only runs when the page is loaded. There is, however, a solution from Nick Reed.

Mr. Reed uses CoffeeScript, so I’ve translated his solution to regular JavaScript. Just remove the script Facebook had you put at the start of your <body> element and put this script in one of your regular javascript files in /app/assets.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
var fb_root = null;
var fb_events_bound = false;

$(document).ready(function() {
  loadFacebookSDK();
  if(!fb_events_bound) {
    bindFacebookEvents();
  }
});

function bindFacebookEvents() {
  $(document).on('page:fetch', saveFacebookRoot);
  $(document).on('page:change', restoreFacebookRoot);
  $(document).on('page:load', function() {
    if(FB) {
      FB.XFBML.parse();
    }
  });

  fb_events_bound = true;
}

function saveFacebookRoot() {
  fb_root = $('#fb-root').detach();
}

function restoreFacebookRoot() {
  if($('#fb-root').length > 0) {
    $('#fb-root').replaceWith(fb_root);
  } else {
    $('body').append(fb_root);
  }
}

function loadFacebookSDK() {
  window.fbAsyncInit = initializeFacebookSDK;
  $.getScript("//connect.facebook.net/en_US/all.js#xfbml=1");
}

function initializeFacebookSDK() {
  FB.init({
    appId: '<your app id>',
    status: true,
    cookie: true,
    xfbml: true
  });
}

With this, you’re almost done. You have the user’s browser sending requests to Facebook’s servers, receiving authorization, and then returning the bits of information you need to log the user in. All you need to do is get that authentication information onto your server - which is pretty standard Omniauth.

This post will be updated once this process is complete.


  1. OAuth and OAuth2 are open standards for authenticating against trusted third party servers.

  2. No doubt to complete their mission to destroy the web.

Scanning With XSANE on a Brother MFC-L2700DW

linux

In a previous post, I documented how to se up a Brother MFC-L2700DW network printer on Linux. At that time, I had not been able to set up network scanning. I still haven’t been able to get network scanning working, but I have been able to scan documents using XSANE.

As far as I can tell (at least on Xubuntu 14.04), you will need to connect the printer to the computer directly via USB. If you already set up printing via the network, this won’t interfere.

I used a post by on Tarah Wheeler Van Vlack’s for the rest of the details (thanks!), especially for batch scanning.

For me, I had to make one adjustment - running XSANE as root. This is strongly discouraged by the XSANE documentation, but in my experience there wasn’t any issue with running XSANE as root.

The only problem that I encountered is that, as a consequence of this, the generated document will be written with ownership by the root user, so you’ll need to copy it or chown it using sudo.

How Many People Have Been Alive Since 1970?

statistics

Today my friend Steve Spalding asked me a surprisingly deep question: How many people have been 21 years of age or older since 1970? He posited that if I couldn’t come up with a good way to estimate that, another option would be, how many people have been alive since 1970?

This turns out to be a really easy question to answer if you have the right data set, and nearly impossible otherwise. I initially started by looking up the population in 1975 and the population today, shooting for the ‘alive at any time’ number. This didn’t work out as well as I had hoped - the number I initially came up with, which was right off, was about 9.2 billion.

If, however, you go to the actual UN Data set that most of the links you find are based on 1, you can download a file called “WPP2012_POP_F07_1_POPULATION_BY_AGE_BOTH_SEXES.XLS” 2. This file has estimates of the world population every year since 1950, every five years, by five year age groups.

So, we can answer the question of how many people have been aged 20 or older since 1970- start with the population aged 20 or older in 1970 (1,935,700,000) Then add the “20-24” population for every five year estimate since. Each person can be in the “20-24” group only once, so we aren’t double counting anyone. Finally, add the “Medium Fertility” estimate for the 20-24 age group for 2015, as the data was published in 2012.

The total? About 6,432,521,000 have been aged 20 or older in the world since 1970.