Using FactoryGirl to test Padrino web applications using Padrino and Sequel

This article helps you use the widespread FactoryGirl gem to test your controller code of Padrino web applications when you decided to use Sequel as your database helper library.


Choice of components

Yes, Padrino is still my favorite web framework. Looking at all the hoops that Rails developers have to jump through I don't feel so bad when I need some trickery to accomplish my goal with Padrino. I refuse to use too much magic and Padrino makes it pretty explicit how it works. Same is true for Sequel which is both automatic (it understands the database schema automatically) but at the same time explicitly. Now it was time that I started using proper test code before I made my current web application any more complex.

Learning from others

Looking at a list of open-source Padrino-based web applications I thought I could learn about how to test my code properly. But I hardly found any non-trivial tests. Few applications tested their helpers as unit tests. Some of them at least tested their models because that seemed a rather simple thing to do. But none of them taught me anything on how to check my controllers. Some developers even argue that testing controllers is a waste of time. Perhaps it is but I still want to try it and come to my own conclusions.

Testing mumbo-jumbo

If you are like me then you may not have used a lot of tests. And you may be confused about fixtures, factories, stubs and mocks. I'm not a life-long expert on the topic either but let me give a brief explanation. Or skip this if you know it already.

Why tests at all?

The idea of tests is that you not only write the required code for your application but extra code that tests the code you have written. After all we are human and the number of bugs that you add to any application grows with the application's complexity. Say you write a web application that provides a company phone book. After your application is half done you click through all the pages and try out a few links and enter some data into the forms. You wouldn't really need tests for that. But your application will require change later because your users will come up with new ideas or install new versions of the libraries (Ruby: gems) that you use. And after every non-trivial change you should check your application all over again because otherwise some parts may break that you did not expect to break. If you develop a number of tests for what you would check manually then you can just run the tests over and over again and be pretty confident that your changed haven't broken anything elsewhere.

An even more professional approach is test-driven development (TDD). Following the TDD methodology you first write a test that would check if your newly implemented feature works. Of course your feature does not even exist so the test fails (which is what it must do). Then you implement the actual feature and in the end see that your test succeeds. And then you can start refactoring (optimizing) your code. As long as the test succeeds you can be quite sure that you are doing well. If you intend to use tests at all then I think that TDD is the best way to proceed. You spend time writing the tests anyway and this way you can think of what your new piece of code should do.

Fixtures

Let's take the imaginary company phone book web application again. Say you want to write tests that make sure you get a list of contacts, that you can enter new contacs, delete some of them and edit them. Testing an application that stores its data in a database requires that you have some data to play with. If you have no contacts in your database then how will the list of contacs look? Will you test for the string "Sorry – there are no contacts yet."? Boring.

So fixtures are basically sample data that are loaded into the database to run tests on. Ruby-on-Rails used to work with fixtures a lot. You could add a CSV file and Rails would load that data into the database before your test code would run. That way you may already have 100 contacts in your database and can test if your application can properly search, edit and delete the data. The tests can even alter the data – no problem – at the end of the tests the database is wiped again anyway. There can even be different fixtures to test different scenarios of your application.

Fixtures are pretty helpful. But their major disadvantage is that they are static. If you add a new field to a database table then you need to edit all your fixtures and add that field there, too. Depending on the complexity of your database schema that can cost quite some hair.

An example fixture might look like this:

idnamephone
1John Doe555-2185
2Jane Daisy555-2967
3Jim Dorian555-1957

Factories

This is nowadays the recommended way to deal with data in tests. I just said that fixtures have a drawback in that they are static and require constant editing if your database schema changes. That is exactly what factories want to avoid. A factory is code that creates (database) objects. You could write a factory for the contacts like this:

factory :contact do
  sequence(:id) { |n| n }
  name "John Doe"
  phone "555-xxxx"
end

This factory allows you to create new contacts. To create the same data as the fixtures above your could would look like:

@contact1 = create(:contact, phone: "555-2185")
@contact2 = create(:contact, name: "Jane Daisy", phone: "555-2967")
@contact3 = create(:contact, name: "Jim Dorian", phone: "555-1957")

The "create" call each created a new database row/object and saved it into the database. This is already simpler than the fixtures above because if you have to add a field then you just edit the factory and are done with it.

Now in your tests it usually doesn't matter if a contact is called "John Doe" or "Jane Daisy". So you could as well create three contacts that are all called "John Doe" with the same phone number like this:

@contacts = create_list(:contact)

Yes, that's essentially everything you need to write. And suddenly it becomes easy to create sample data. You don't even create all that sample data once and then run all tests over it. Instead you just create the minimal data that you need for any of your test cases.

At first I was a bit confused whether factories actually create rows in the database. I learned that it depends. The FactoryGirl library that I will present further down distinguishes between the ".build" and ".create" methods. "build" just creates an object im memory that you can use. "create" creates an object (with "build") and then calls ".save" on it to store it in the database. Obviously "build" is faster because things just happen in memory and not in the database. But if you want to test your Padrino controllers you surely need to have actual database rows so you must use "create".

Stubs/Mocks

A further abstraction are stubs/mocks. (I believe there is even a dinstinction but I don't use any of them seriously so I didn't delve into it any more.) They create objects that provide the absolute minimum functionality (in means of object methods) to help with your tests. My criticism is that mocks/stubs are articial objects that are not even related to your database rows/objects any more. So you are using something completely distinct to test your actual application.

In my opinion stubs are not useful in the context of testing database-driven web applications. I just use them in very rare cases where I e.g.load data from a URL on the internet for my tests. The stub then just pretends to load the data from the internet but instead loads the data locally.

Adding FactoryGirl

Are you still with me? Great. Sorry for the lengthy introduction but I never found a good short explanation of the why's of tests, fixtures and factories. Everybody seemed to assume that the topic is so crystal clear that it would not deserve further explanation. I didn't think so.

So now that you know what the purpose of factories are let's pimp your Padrino web application project with FactoryGirl – which is the most widely used library for factories in Ruby. First add the gem to your project's "Gemfile":

group :test do
  gem 'factory_girl'
end

Run "bundle" to install the gem.

I assume you are using "rspec" as a testing component here. And that you use some way to clear your test database before you run the tests. I have written another article on that topic that I recommend reading. So please edit your spec/spec_helper.rb file of your project. After the "RSpec.configure" block add this code (which is also described in the FactoryGirl documentation):

# Init FactoryGirl
FactoryGirl.definition_file_paths = [
    File.join(Padrino.root, 'spec', 'factories')
]
FactoryGirl.find_definitions

Now when you are running your rspec tests it will first read all files in the spec/factories directory and use them as factories. So you need to "mkdir spec/factories".

Adding factories

For any database model that you use you should now create a factory. This is the actual code you would need for your company phone book application:

FactoryGirl.define do

factory :contact do
  sequence(:id) { |n| n }
  name "John Doe"
  phone "555-xxxx"
end

See the FactoryGirl introduction to understand how such a factory is supposed to be coded.

The save! caveat

There is a slight catch when using FactoryGirl. Originally it was written for Rails applications that use ActiveRecord as a database helper library. When you run FactoryGirl.create then in the background an object is created using FactoryGirl.build and then saved using "save!". ActiveRecord objects have a .save! method – but Sequel objects don't. Fortunately this is easy to fix by aliasing ".save!" to just ".save".

For that workaround just create a file models/factorygirl-sequel.rb and make it contain:

class Sequel::Model
  alias_method :save!, :save
end

Simple as that.

Trying out the new factory

If you want to try it then run a Padrino console in the context of testing:

$> padrino c -e test

=> Loading test console (Padrino v.0.12.3)
=> Loading Application YourProject::App
2.0.0p247 :001 > FactoryGirl.find_definitions
 => ["/home/johndoe/projects/yourproject/factories", "/home/johndoe/projects/yourproject/test/factories", "/home/johndoe/projects/yourproject/spec/factories"]
2.0.0p247 :002 > FactoryGirl.create(:contact)
 => #<Contact @values={:id=>1, :name=>"John Doe", :phone=>"555-xxxx"}>
2.0.0p247 :003 >

As you see FactoryGirl has created a new "Contact" object, initialized it with some default data (that you defined in the factory) and saved the object to the database. (If you are using database cleaning code as recommended then this contact will get removed after you quit the Padrino console.)

Using the factory in a controller test

Recent versions of Padrino automatically add testing code for every controller that you create using "padrino gen controller…". So if you have a controller called "app/controllers/contact.rb" then Padrino has at the same time created a test file at "spec/app/controllers/contact_controller_spec.rb". Make that test file looks like this:

require 'spec_helper'

RSpec.describe "/contacts page" do
  # Get the list of contacts and make sure it loads
  describe "List of contacts"
    before do
      @contact = FactoryGirl.create(:contact)
      get "/contacts"
    end

    it "loads correctly" do
      expect(last_response).to be_ok
    end
  end

  # Create a sample contact and search for it
  describe "Search in contacts" do
    before do
      @contact = FactoryGirl.create(:contact)
      get "/contacts?search=#{@contact.name}"
    end

    it "loads correctly" do
      expect(last_response).to be_ok
    end

    it "returns: body contains the search result" do
      expect(last_response.body).to include('John Doe')
    end
  end
end

This is just a simple example that tests your /contacts and /contacts?search=… controller actions. You can see that a sample contact called "@contact" is created and saved to the database. And you can refer to the data in that @contact object later (@contact.name).

Associations

In any real-life application you will surely use associations/relationships like one-to-many or many-to-many. For example one of your contacts could have multiple phone numbers that are stored in "PhoneNumber" objects. It took me an hour to understand how to create such a sample contact with a number of phone numbers using factories. This is essentially how the factory code should look:

factory :contact do
  sequence(:id) { |n| n }
  name "John Doe"
  phone "555-xxxx"

  trait :with_phonenumbers do
    after :create do |c|
      create_list :phonenumber, 5, contact: c
    end
  end
end

The concept of "traits" in FactoryGirl is basically "additional data". So you can create a simple "Contact" object. But if you use this trait then after the contact was created the factory will create 5 more phone numbers and assign them to the contact. Of course this requires that you defined the one-to-many relationship in your model. Creating such a contact with related phone numbers is as simple as:

@contact = FactoryGirl.create(:contact, :with_phonenumbers)

Debugging

Running your tests just means executing "rake spec" on the shell. But often the output is very long and doesn't point you to the actual problem. So debugging helps. I'm using Ruby 2.0 so my approach is to use the "byebug" gem. Older versions of Ruby have different debugging modules – I'm not dealing with that right now. To add the byebug gem once again editor your "Gemfile" and add:

group :test do
  gem 'byebug'
end

And run "bundle".

Now whenever you add the line "byebug" somewhere into your tests then "rake spec" will stop at that line and you can use the debugger to examine local variables and step forward.

Conclusion

So if you want to test your controllers with minimal effort then consider using FactoryGirl as shown above. Once you wrote a few factories your tests will become very easy to write.

I also recommend the betterspecs.org web site that gives real-life examples on how to improve your tests. They also have a good chapter on factories.

What is your experience with testing and Padrino? Drop a comment.

How to use user authentication in your Padrino apps with padrino-warden

Nearly every web appliation requires user authentication. This introduction should help you save the evening I just spent trying to figure out how to use the Warden authentication layer in Padrino web applications.

What the Rack?

TL;DR -> If you want password-based authentication in your Padrino app – use this guide. 🙂

As you probably know Padrino (and many other Ruby frameworks) use Rack – which is the interface between the Ruby code and the actual web server that speaks HTTP. You can roughly compare it with CGI or WSGI (for Python). The advantage of Rack is that you can influence the way between the HTTP request, your application and the HTTP response. You can run the data through many different layers (called "middleware") to mess with request and response. Warden is a Rack-based middleware that does exactly that. And it is designed to provide a mechanism for authentication in Ruby web applications. Warden is rather basic and acts transparently unless it is told to interfere. It is controlled through a Rack environment variable env['warden'].

Then sinatra-warden was conceived to make integration of Warden into Sinatra-based applications easier. Well, the software hasn't been maintained for two years and its bug reports as just as old. But it seemed to serve as an example for the maker of padrino-warden. And since Padrino is based on Sinatra it helped make Warden easier to integrate into Padrino applications. So basically you just need t use the padrino-warden gem, configure how a successful login would work (depending on your database model or application logic) and create a template for a login page.

This may be trivia for experienced Ruby web developers. But it took me a while to figure out how things work together.

Installation

Add to the Gemfile of your project:

gem 'padrino-warden', :github => 'jondot/padrino-warden'

Why not just use the 'padrino-warden' gem that could get loaded from RubyGems? Well, the RubyGems version is 0.1.0 from 2010 – at least at the time I wrote this article. I already asked the maintainer to update the gem because his version on GitHub is actively maintained and currently (January 2014) version 0.19.0. I usually discourage using gems from GitHub because they are moving targets and next time you deploy your application you will probably get different code and different results.

Now in your shell from your project directory run…

bundle install

…and after a few seconds padrino-warden – and as a dependency warden – is installed.

Enabling Warden in your application

Next you have to register the Padrino::Warden module for your application. Just edit your app/app.rb file and near the other "register" lines add:

register Padrino::Warden

Configure how your model supports authentication

I assume that you already have a database model set up where you store usernames and passwords. In my application the model is called User and I use the email address as the identifier and a password. My model also has an ".authenticate" method that expects the email address and the password and either returns the user object (if login was successful) or nil/false (if login failed).

Once that is set up and worked (preferably with test code) you can add a file lib/authentication.rb (do not call it lib/warden.rb!) and make it look like this:

Warden::Strategies.add(:password) do
  def valid?
    params["email"] || params["password"]
  end

  def authenticate!
    if user = User.authenticate(params["email"], params["password"])
      success!(user)
    else
      fail!("Could not log in")
    end
  end
end

Warden::Manager.serialize_into_session { |user| user.id }
Warden::Manager.serialize_from_session { |id| User[id] }

Make sure that you customize the parts printed in bold letters for your purpose. The "valid?" method only tries the authentication if either of the fields is present. The "User. authenticate…" call should point to a method in your database model that returns the user if authentication was successful. Your method will likely search for the right user in the database based on the given email address and verify the password. And at the end change "user.id" into something that returns the ID (primary key) of that user record. Also turn "User[id]" into code that gets a certain user account from the database by its ID. "User[id]" works for Sequel. If you use ActiveRecord you will likely use something like "User.find(id)".

Routes

Oh, by the way: padrino-warden adds a controller named "session" to your application. Just run "padrino rake routes" to see the extra routes you now have:

URL                               REQUEST  PATH
(:sessions, :unauthenticated)       GET    /unauthenticated
(:sessions, :unauthenticated)       PUT    /unauthenticated
(:sessions, :unauthenticated)      POST    /unauthenticated
(:sessions, :unauthenticated)     DELETE   /unauthenticated
(:sessions, :unauthenticated)     OPTIONS  /unauthenticated
(:sessions, :unauthenticated)      PATCH   /unauthenticated
(:sessions, :unauthenticated)      LINK    /unauthenticated
(:sessions, :unauthenticated)     UNLINK   /unauthenticated
(:sessions, :login)                 GET    /sessions/login [1]
(:sessions, :login)                POST    /sessions/login [2]
(:sessions, :logout)                GET    /sessions/logout [3]
(:sessions, :oauth, :callback)      GET    /sessions/oauth_callback

The most interesting routes are the ones I have marked [1], [2] and [3]. The process goes like this:

  1. The user gets redirected to /sessions/login and the application will send them an HTML form with a field for the email address (or username) and the password. This form sends it data per POST method to /sessions/login. Yes, the URL is the same – but the method has been GET at first and is now POST. The controller code for /sessions/login is already there (from padrino-warden). You just need to create the template file in app/views/sessions/login.slim (or login.erb or whatever template language you use) and create the form there. In SLIM it looks as simple as this:
    = form_tag url(:sessions_login), :method=>:post
      = email_field_tag :email
      = password_field_tag :password
      = submit_tag

  2. The user has pressed the submit button and the form gets sent to the POST method action of /sessoins/login which is also build into padrino-warden. So there is nothing to do for you here. If the authentication was unsuccessful the login page is shown again. If the authentication worked the user gets redirected to '/'.

  3. If the user decides to log out they can be linked to /sessions/logout which invalidates the login session.

Restricting controllers and actions

Of course your users won't login voluntarily. You will have to restrict certain controllers or actions. This is rather simple. Either add these lines to make the entire controller require authentication:

  before do
    login
  end

Or just add the "logged_in? or login" expression at the beginning of the controller's methods you want to secure.

Who is logged in?

padrino-warden provides a "user" (alias: "current_user") variable for your templates that contains the actual user object according to your database model. So these lines (SLIM syntax) will show who is logged in or provide a login link:

- if logged_in?
  a href="#" Logged in as #{current_user.email}
- else
  a href="#" Anonymous

Objects

By the way these are the objects that padrino-warden gives you:

  • warden → Warden object
  • authenticate / login → render the login form
  • authenticated? / logged_in? → true if the user is logged in
  • logout → invalidate session
  • user / current_user → user object if a user is logged in

Authorization

Now I will have to start thinking about how to realize authorization… suggestions welcome. 🙂

Clearing RSpec test databases with Padrino and Sequel

Nowadays applications should use thorough test suites. Test-driven development is not just another stupid hype. It makes you confident that your code works and avoids situations where you have to explain your clients why that old bug suddenly appeared again. That is especially true when writing web applications. But this is not about convincing you to write tests. This little snippet makes sure your tests run on a clean database. 🙂

Many developers seem to take an easy route to testing their web applications. They rather use an SQLite in-memory database for testing by specifying their database as “sqlite:///”. Of course your database would be clean on every run of your tests. Oh, and it’s really fast. But in many non-trivial cases you don’t test well. SQLite has many limitations opposed to “real” DBMSs like PostgreSQL or even MySQL. So if you want to enforce date fields or foreign keys SQLite may lull you into a false sense of security. I recommend using the real DBMS to make the tests more realistic.

Bad:
when :test then Sequel.connect(“sqlite:///”, :loggers => [logger])

Good:
when :test then Sequel.connect(“mysql://testuser:testpassword@server/database_test”, :loggers => [logger])

One caveat is though that your tests will inevitably write data into the test database. Unless you delete that data your next run of tests will start with that data from the previous run. That will lead to funny results. So you need to make sure that before each test run the tables are emptied.

First I tried to use the database_cleaner gem. Unfortunately it does not support DELETE’ing rows using Sequel but can just TRUNCATE tables. Well, TRUNCATE is usually faster anyway. Unfortunately MySQL has a shortcoming: it does not support to TRUNCATE tables with foreign keys and an “ON DELETE CASCADE”. So you have to DELETE all rows to trigger the delete cascade. So the combination of MySQL, Sequel, database_cleaner and DELETE-CASCADE does not work. I believe that database_cleaner’s functionality is aimed at at ActiveRecord that does not care about referential integrity on a database level. But I am using Sequel because I do care about my database schema.

Surprisingly the functionality of database_cleaner was pretty simple in this case. So to finally provide you with the solution… just add this code into your Padrino applications’s spec/spec_helper.rb code within the “RSpec.configure do |conf|” block:

conf.before(:each) do
  existing_tables = Sequel::Model.db.tables
  tables_to_preserve = [:schema_info, :schema_migrations]
  tables_to_be_emptied = existing_tables - tables_to_preserve
  tables_to_be_emptied.each do |table|
    Sequel::Model.db[table].delete
  end
end

It will get a list of tables and run “DELETE FROM …” on them. However it will spare the “schema_info” and “schema_migration” tables because they contain valuable meta information needed for migrations.

Voila… now you can safely run your tests and be sure that you have clean database tables on every test.

Sequel model pagination with Padrino

Padrino is a nice lightweight web framework in Ruby for developers who don't like the monolithic approach of Ruby on Rails. But there are fewer articles on the internet yet. So this article explains how to use pagination over database models when using the Sequel database library with the Padrino web framework.

The Padrino web project needs to use Sequel as the ORM/database component. So I assume that you find ":orm: sequel" in your .components file.

Your database configuration must be set in config/database.rb. At the end of the file also add:

Sequel::Model.db.extension(:pagination)

This enables the pagination extension that comes with Sequel. It does the page calculations similar to what pagination modules like will_paginate are doing and returns a set of items to display in a view/template. It does not have functionality to render the paginator in the view so you may still want to use will_paginate to render the navigation in your view/template.

So now any Sequel dataset has a .paginate() method added. Examples:

@page = Sequel::Model.db[:table].paginate(@page, size)
@page = MyModel.dataset.paginate(@page, size)

As you see you can't use "MyModel.paginate()" directly. Opposed to what ActiveRecord does Sequel distinguishes between models and datasets. But you will still get an array of (ORM) objects back that you can manipulate and save back to the database. So all you need to do really is add ".dataset".

One caveat is that the .paginate(page_nr, number_of_items_per_page) requires the page_nr parameter to be an integer. But the params[] hash that you can use in a Padrino controller contains either "nil" if the parameter is missing or a string. So you need to turn "nil" into 1 (the first page). And any other page like "15" needs to be turned into an integer 15.

So a typical two-liner in your controller looks like:

@page = (params[:page] || 1).to_i
@page = MyModel.dataset.paginate(@page, size)

I still wanted to be able to make this shorter and talked to Jeremy Evans – the maker of Sequel and he suggested to add a method to the Sequel::Model class. So if you are as lazy as me then add a file model/paginate.rb and insert:

# These helper methods add a .page method that works for pagination with ORM objects and datasets.
# Basically they save the "page = (page || 1).to_i".

 

# Define a class method to paginate through ORM models
class Sequel::Model
  def self.page(page, per_page=10, *args)
    # The page number comes as a string from the Sinatra controller.
    # Turn it into an integer and make it 1 if it was nil.
    page = (page || 1).to_i
    dataset.paginate(page, per_page, *args)
  end
end

 

# Define a class method to paginate through datasets
class Sequel::Dataset
  def page(page, per_page=10, *args)
    # The page number comes as a string from the Sinatra controller.
    # Turn it into an integer and make it 1 if it was nil.
    page = (page || 1).to_i
    paginate(page, per_page, *args)
  end
end

This file gets loaded automatically by Padrino. And as your models inherit from the Sequel::Model class and datasets are instances of Sequel::Dataset you get this class method, too. So now in your Padrino controller you can simply write:

@page = MyModel.page(param[:page], 10)

for models or

@page = MyModel.grep(:name, '%foo%').page(param[:page], 10)

for datasets. The reason I call this method ".page" and not ".paginate" is to avoid name clashes with Sequel::Dataset.paginate.

Sequel offers this set of instance variables to @page that you can use:

  • page_size → max items on this page
  • page_count → number of pages (at least 1)
  • current_page → number of this page (starts at 1)
  • pagination_record_count → number of all items
  • current_page_record_range → [startitem..enditem]
  • current_page_record_count → items on this page
  • first_page? → true if this is the first page
  • last_page? → true if this is the last page
  • next_page → number of next page or nil (if last)
  • prev_page → number of previous page or nil (if first)

 

If you want some more help in your views to render the navigation (something like "First | Previous | 1 2 3 4 | Next | Last") then install will_paginate:

padrino gen plugin will_paginate

And just call <%= will_paginate @page %> in your view/template. will_paginate lets you customize the look and layout of that navigator.

By the way: MyModel.paginate will still use the Sequel pagination and not will_paginate's. All you use from will_paginate is the <%= will_paginate … %> view helper. will_paginate has functionality built-in that translates Sequel's pagination to its own convenctions.

Have fun.

© 2021 workaround.org - Proudly powered by theme Octo