Ruby Heredocs in Array Constants

While writing a spec that uses different chunks of csv and txt data I was wondering about the best way to define multi-line strings in array constants.

Normally, I would use Heredoc to define a single multi-line string like this:

CSV_CHUNK = <<-CSV
10, "a", "b"
20, "c", "d"
30, "e", "e"
CSV

Perfect. The unattractiveness starts when adding more chunk definitions. It usually ends up with having CSV_CHUNK_0, CSV_CHUNK_1, CSV_CHUNK_3 and so on in place. Thats a bit unfortunately. For example this hinders to use normal array iteration like each and friends.

So, my question was if there is a way to simply add chunk after chunk to an array. Sure its possible:

chunks = []
chunks <<<<-CSV
10, "a", "b"
20, "c", "d"
30, "e", "f"
CSV
chunks <<<<-CSV
40, "a", "b"
50, "c", "d"
60, "e", "f"
CSV

This is valid Ruby syntax. Actually its just the << method of Array plus the Heredoc syntax. ( Yes, you can add a space inbetween :) )

But, since we are altering a variable we can't use a constant here. To use a constant, we have to do the Heredoc definition inline in the Array declaration:

CHUNKS = [
  <<-CSV ,
10, "a", "b"
20, "c", "d"
30, "e", "f"
  CSV
  <<-CSV ]
40, "a", "b"
50, "c", "d"
60, "e", "f"
   CSV

Although, this looks pretty scary, its again valid Ruby syntax. As many other languages Ruby allows a comma in front of the closing square bracket. We can use this to pretty up this construct and to make it more readable:

CHUNKS = [
  <<-CSV ,
10, "a", "b"
20, "c", "d"
30, "e", "f"
  CSV
  <<-CSV ,
40, "a", "b"
50, "c", "d"
60, "e", "f"
   CSV
]

AMQP: Integrating Spring and Rails with RabbitMQ

Why?

Spring web applications tend to grow and become bigger and more complicated over the time. So it's often a good choice to source out a specific responsibility and process it in an other app made for the specific job. You can build a clean interface to the new app and maybe have an extra team just working on it. Plus you have a free choice of the programming language. I like Ruby.

Another big reason is that you don't want to do all computations in the time of one request of your Spring webapp. If something could be processed later, why not improve the response time and latency of your web app by asynchronously handing the job over to another app.

Maybe you guessed it: This can easily done with the AMQP protocol and one or many RabbitMQ broker. AMQP stands for Advanced Message Queuing Protocol which is an open standard and RabbitMQ is an excellent open sourced server or broker that implements this protocol. With AMQP you can asynchronously send messages in a standardized protocol that is supported by the vast amount of programming languages. There even is a spring project called spring-amqp and of course a ruby gem called amqp by tmm1. To get an idea how RabbitMQ works I encourage you to check the projects homepage.

Getting started

First install RabbitMQ. Be sure to install RabbitMQ of version greater than 2. Otherwise you get "Protocol Mismatch" Errors.

If your on a Mac and have brew installed its easy like this:

$ brew install rabbitmq

Next install the amqp gem and start rabbitmq. You can use the following scripts to verify that sending and receiving messages over AMQP works:

# consumer.rb
require "rubygems"
require "mq"

AMQP.start do
  queue = MQ.queue('hello.world.queue')
  
  queue.subscribe do |word|
    puts word
  end
  
end

# producer.rb
require "rubygems"
require "mq"

AMQP.start do
  queue = MQ.queue('hello.world.queue')
  
  i = 0
  EM::add_periodic_timer(1) do
    queue.publish "hello world #{i+=1}"
  end
  
end

The Consumer simply subscribes to the hello.world.queue. If a message arrives the subscribe block gets called and prints the message to the console. The Producer just pushes a message every second to the queue. Since AMQP already uses EventMachine, you can use all features of EventMachine like add_periodic_timer and so on.

If everything works fine, you should see the Consumer receiving "hello world"-messages.

Now we build a new Rails 3 app and configure it to process messages.

$ rails new consumer_app

You need to put the dependencies for tmm1's amqp gem and the thin server in your gemfile:

gem 'amqp', :require => 'mq'
gem 'thin'

Why do we use thin as webserver? Simply because thin provides a running EventMachine reactor. Without this reactor EventMachine will block the current thread and no HTTP request would be processed. If you like to use an other server than thin you need to activate the EventMachine reactor manually. To do this, just put

Thread.new { EM.run }

in an initializer.

Now, create a Model called example_message.rb with following content:

# app/models/example_message.rb
class ExampleMessage

  def self.start_listen
    AMQP.start do  
      MQ.queue('hello.world.queue').subscribe do |msg|
        puts msg
      end
    end
  end

end

The listen-method can be triggered by a Rails initializer. But the AMQP block will block during the boot process of Rails when we call ExampleMessage.listen directly. We can get around this by using EventMachine to call it later in the next available time slot.

# config/initializers/example_message.rb
EM::next_tick do
  ExampleMessage.listen
end

Notice: My colleague Tobias Sunderdiek found out, that this only works for Ruby Version greater than 1.8.7 with a patchlevel around 320. It will not work with the current Ruby Enterprise Edition, which has a patchlevel of 253.

Now, if you start your Rails app and the Producer script you should see your app receiving messages.

Lets try to switch from the simple Producer script in Ruby to a simple Producer built in Java that uses the same queue. Luckily the Spring-AMQP project comes with an example that exactly does this called helloworld. You can fetch the Spring-AMQP sources using git as shown here:

$ git clone https://github.com/SpringSource/spring-amqp.git

To compile and run the helloworld example navigate to samples/helloworld and type:

$ mvn compile
$ mvn exec:java -Dexec.mainClass="\
  org.springframework.amqp.helloworld.async.Producer"

Those Spring-AMQP examples are a great starting point to find out how Spring-AMQP works. From there it is a easy step to build a Service that does the message sending for a specific use case. Here is a simple Service that uses the RabbitTemplate to send messages:

@Service
public class HelloWorldMQService {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    public void sendMessage(int i) {
        rabbitTemplate.convertAndSend("hello world "+i);
    }
}

The RabbitTemplate must be configured before it can be used. To do this you need a AbstractRabbitConfiguration as shown below:

@Configuration
public class HelloWorldConfiguration 
        extends AbstractRabbitConfiguration {

    protected final String helloWorldQueueName = "hello.world.queue";

    @Override
    public RabbitTemplate rabbitTemplate() {
        RabbitTemplate template = 
            new RabbitTemplate(connectionFactory());
        template.setRoutingKey(this.helloWorldQueueName);
  
        return template;
    }

    @Bean
    public ConnectionFactory connectionFactory() {
        SingleConnectionFactory connectionFactory = 
            new SingleConnectionFactory("localhost");
        connectionFactory.setUsername("guest");
        connectionFactory.setPassword("guest");
        return connectionFactory;
    }

Now you can integrate this AMQP service with your Spring webapp.

If you like to dig deeper in AMQP, Ruby and Spring I encourage you to check out this article and this presentation on infoq.com.

Cheers,
Arbo

Testing SOAP Webservices with RSpec

SOAP webservices are widely used in enterprise environments. Although they feel a bit clumsy in comparison to slim REST services, sometimes you have to deal with them.

The great thing is, to test such a service you are often free to use any tool you like. I like RSpec!

To query a web service you just need a few lines of code. I recommend Savon as SOAP client. It is used as shown here:

require 'rubygems'
require 'savon'

WSDL_URL  = 'http://www.webservicex.net/geoipservice.asmx?wsdl'

client = Savon::Client.new WSDL_URL
response = client.get_geo_ip do |soap|
  soap.body = { "wsdl:IPAddress" => "209.85.149.106" }
end
puts response

The response object can be converted to hash with the to_hash method, so you can fetch all values simply like you would do it with any other hash.

Now, the rest should be easy and is just a normal RSpec test:

require 'rubygems'
require 'savon'

WSDL_URL  = 'http://www.webservicex.net/geoipservice.asmx?wsdl'

RETURN_CODE_OK    = "1"
RETURN_CODE_ERROR = "0"

describe "Geo IP Webservice at #{WSDL_URL}" do
  
  # helper method
  def get_geo_ip_result ip
    response = @client.get_geo_ip do |soap|
      soap.body = {"wsdl:IPAddress" => ip}
    end
    response.to_hash[:get_geo_ip_response][:get_geo_ip_result]
  end
  
  before :all do
    @client = Savon::Client.new WSDL_URL
  end

  it "should yield a country name" do
    result = get_geo_ip_result "209.85.149.106"
    result[:country_name].should_not be_nil
    result[:return_code].should eql(RETURN_CODE_OK)
  end
 
  it "should return error for malformed ip address" do
    result = get_geo_ip_result "not.an.ip.address"
    result[:return_code].should eql(RETURN_CODE_ERROR)
  end
 
  it "should fail if no ip address is submitted" do
    lambda { @client.get_geo_ip }.should raise_error
  end

  # ...

end

Happy testing!

EDIT:

@dbloete pointed me to the fact that with RSpec 2 you can expect errors even more readable:

it "should fail if no ip address is submitted" do
  expect { @client.get_geo_ip }.to raise_error
end