Monthly Archives: January 2014

ruby to find latitude/longitude for a list of cities

Posted on by | 3 Replies

I have a relatively short list of cities which I want to plot on a world map. The list is a little too long for a manual lookup, but I don’t know exactly how I’ll use it in an app, so I figured out how to do it with some simple ruby in irb using the lovely geocoder gem.

I had my cities in a spreadsheet. Selected a few cells, and was able to simply paste into irb and split the string on newlines to get an array of the cities.


gem install geocoder
irb
>> require 'geocoder
>> a = "Honolulu, HI
>> Boston, MA
>> New York, NY".split("\n")
 => ["Honolulu, HI", "Boston, MA", "New York, NY"] 

>>a.map do |city| 
    d = Geocoder.search(city)
    ll = d[0].data["geometry"]["location"]
    puts "#{city}\t#{ll['lat']}\t#{ll['lng']}" 
end

Honolulu, HI    21.3069444  -157.8583333
Boston, MA  42.3584308  -71.0597732
New York, NY    40.7143528  -74.00597309999999

Then I could copy/paste the irb output back into my spreadsheet. Ta Da!

It appears that the free Google API has some kind of throttling, so this is only good for short lists of <20 cities.

rspec: mixing transcations and truncation database clearner strategies

Posted on by | 2 Replies

I’m using capybara-webkit for integration testing with rspec, which is awesome, because it is faster than other full-browser testing solutions, like Selenium, but it is slower than RackTest (default for RSpec testing). RSpec provides a nice way to specify an alternate driver when running Javascript tests, but configuration can be a little tricky. I got this working via this excellent blog post via Sarah Mei, who was pairing with me for the day.

I think it is important to actually understand the code that I copy/paste, so I took a little time to read up on the details which I’ve summarized below.

Favorite Testing Gems

I won’t elaborate on RSpec, the concepts in this post likely applies to test-unit as well. I written before about why RSpec is my favorite.

Rack::Test

When using RSpec in Rails, we use the rspec-rails gem, which configures a bunch of stuff that makes it easy to get started. By default, integration tests will use rack-test, a lovely little gem that supports methods like get, post, put and delete and handles the rack request and response objects. (via platformatec) It maintains cookies and follows re-directs, but is far from a full-browser, most notably, pages won’t execute Javascript. Rack::Test is quite lightweight, with tests running in the same process as your Rails code. Its speed is a huge advantage and worth retaining for your tests that don’t need more.

Capybara

Capybara is the world’s largest rodent, and the Ruby community is also the name of a favorite gem for testing the content of web pages. (A successor to Bryan Helnkamp‘s WebRAT, so named for Ruby Acceptance Testing, which instigated the rodent naming theme) Capybara is wonderful with its support for many “drivers” which allow for a consistent API across different solutions that offer different levels of browser support, with different performance characteristics.

Capybara::Webkit

Thoughtbot kindly created the capybara-webkit gem a few years ago, which I’ve found to be more reliable and performant than Selenium, and is my favorite choice for testing pages that need Javascript.

One of its creators, Joe Ferris, explains how it works (via stackoverflow)

  1. Capybara boots up your rack application using webrick or thin in a background thread.
  2. The main thread sets up the driver, providing the port the rack application is running on.
  3. Your tests ask the driver to interact with the application, which causes the fake web browser to perform requests against your application.

Database Cleaner

The DatabaseCleaner gem is super helpful for our typical Rails app that relies on a database. We always want a “clean slate” when we start our tests and this nifty gem gives us a bunch of options with a consistent interface for various database choices.

Configuration

To configure these solutions correctly, it is critical to understand that with Capybara::Webkit our target app code is running in a separate process from our tests. This means that when we set up our test data RSpec is running in one process and needs to actually write to the database, then our app code reads from the database from another process. Wheras with Rack::Test, the tests and the target code runs in the same process. That’s why we can’t use a “transaction” strategy to reset our test environment with Capybara::Webkit. Instead we use the “truncation” strategy, which simple blows away all of the data after each test run.

Why bother with transactions?

Truncation will work just as well with Rack::Test as transcations, so why introduce the complexity of two different configurations? The Database Cleaner README explains: “For the SQL libraries the fastest option will be to use :transaction as transactions are simply rolled back.” Sarah Mei elaborated on this by reminding me that the commit to the database is what takes the most time, and the transaction is never committed, it is simply rolled back at the end of your test. Transactions are pretty speedy, so we want to only use the truncation method when absolutely necessary.

Just Show me the Code

Here’s the configuration that was documented by Eric Saxby from Wanelo, which worked for me as well:

config.use_transactional_fixtures = true

config.before(:each, js: true) do
  self.use_transactional_fixtures = false
  ActiveRecord::Base.establish_connection
  DatabaseCleaner.strategy = :truncation
  DatabaseCleaner.start
end

config.after(:each, js: true) do
  DatabaseCleaner.clean
  ActiveRecord::Base.establish_connection
  self.use_transactional_fixtures = true
end

How does this work exactly?

We are set up to use transactions by default, which is built into rspec-rails and does not rely on DatabaseCleaner. Then, for our JS tests, we tell RSpec not to use transactions and instead instruct DatabaseCleaner to set up before each test runs with DatabaseCleaner.start and then clean up after with DatabaseCleaner.clean.

I have no idea why ActiveRecord::Base.establish_connection is needed, but if we don’t do that, then rake spec hangs after my first JS test with this ominous warning:

WARNING: there is already a transaction in progress

Perhaps someone reading this can explain this detail, but happy to have a configuration that works and hope this helps other folks who want fast tests that run reliably.

why more women conference speakers?

Posted on by | Leave a reply

In 1997, I attended the Grace Hopper Conference for the first time. I was already quite accomplished in my career. My first startup had been acquired. I had worked as a software developer on both After Effects and Shockwave. My code was used by hundreds of millions of people. I believed that I had eradicated all impostor syndrome issues.

The Grace Hopper “celebration of women in technology” was then held every three years. There were a few men scattered about the conference, but every speaker was a woman and every talk was a technical talk. I sat there, surrounded by more technical women than I had every seen in my whole life. I listened to talk after talk that stretched my technical abilities. I remember ones about parallel processing, new compiler tech and how one researcher was making chips to sequence genomes. It was amazing. This was a great conference independent of the gender of its speakers and audience. I started to think about the innovative work I was doing and what parts of it might stand out and be interesting to speak about. I caught myself thinking: “I could do that.” And then I snapped to attention: I had never realized that I had been holding myself back.

Later as I searched for role models, I struggled to find well-known women who were pursuing a technical leadership role where they would continue to “do the work” rather than pursuing a management track. Often, I would suddenly realize in a conversation with a male engineer that I served as an “existence proof” of a competent woman engineer.

There’s a lot of research that supports the need for more, visible women (ditto for other unrepresented minorities). The stereotype threat, where simply being reminded that you are part of a negatively stereotyped group can cause your performance to falter, is easily enacted when you see no one like you on stage, and few like you in the audience. Not to mention, the simple discomfort of a professional conversation mistaken for sexual invitation. The book Women Don’t Ask referenced research that showed: if there are less than 30% of a visible minority, and someone from that minority speaks, most people (both from the majority and minority group) will assume that person speaks as a representative of their minority group. XKCD has a delightful illustration of this phenomenon.

When Sarah Mei and I started the RailsBridge workshops, we had a hypothesis that there could be a simple solution to this problem. Statistically there were more women engineers in the SF Bay Area than there were Ruby engineers, which may still be true today. I had been wanting to learn Ruby for a few years before I did. There’s alway new tech that we all want to learn. What if, we simply taught more women Ruby… could we change the balance? Sarah Mei gave a great talk on how that succeeded.

For the first workshop, we struggled to come up with women teachers. Of course, those we found or remembered, had been there all along, but had stopped attending the male-dominated events. There were women in the community — not 50% by far, but a lot more than the 3% we were seeing. Once the workshops created a community where women felt welcome, more women started coming to the meetups. These events are social, as well as professional development opportunities. Research supports that people don’t learn well when they feel uncomfortable. Experienced engineers don’t actually need meetup events or conferences in order to learn new tech. However, I believe the conferences need us. The industry needs us to be visible and outspoken or we will never overcome the very real, though often unintended, sexism and racism that exists in the world today.