Testing pipelined HTTP requests with WebMock

We’ve just released a small gem called webmock-net-http-pipeline
(how’s that for a mouthful?) that helps us with testing Ruby code that makes
calls to external services via pipelined HTTP requests, by enabling us to mock
those requests via WebMock. Read on for more details…

 What is HTTP Pipelining?

HTTP Pipelining is a feature of HTTP 1.1. It basically provides
the ability to send multiple idempotent requests (principally GET and HEAD)
along a single HTTP connection, without waiting for each response individually.
The image below is a representation of how pipelining multiple requests can be
of benefit when trying to minimise wait time between the start of the first
request, and the end of the last response:

pipelining.png

Pipelining allows us to minimise the time spent waiting for responses from the
external service by providing the ability to send all requests first, then
receive all the responses in one go. The HTTP 1.1 specification states that
compliant servers must return responses in the same order that the requests are
received in (and TCP guarantees the ordering on the wire), so it’s simple to
batch requests without having to untangle a mess of response objects when they
come back.

 Why do we use it?

We use HTTP pipelining for efficiency when making a number of calls to a single
internal HTTP service. There are a few articles around on
the web implying that there’s not really much benefit to be had from pipelining
requests over fast connections. However, that’s not been our experience.

When someone using our mobile application performs a search, the request is
effectively split in to two parts. Firstly, a sub-request gets made to one
internal HTTP service to perform the search and return a list of member IDs;
the application then requests the details of those 20 members from a separate
HTTP service, so they can be displayed to the end-user. The following
benchmark script mimics the requests to retrieve the member details using both
serial and pipeline methods.

require "net/http/pipeline"
require "benchmark"

http = Net::HTTP.start("member-service.wld", 80)
http.pipelining = true

requests = (1..20).map { Net::HTTP::Get.new("/members/123456789") }
repeat = 100

Benchmark.bm do |bm|
  bm.report("series:  ") do
    repeat.times do
      requests.each { |r| http.request(r) }
    end
  end

  bm.report("pipeline:") do
    repeat.times do
      http.pipeline requests.dup
    end
  end
end
               user     system      total        real
series:    1.170000   0.110000   1.280000 (  4.419424)
pipeline:  1.010000   0.110000   1.120000 (  2.371025)

The numbers speak for themselves: a 46% speed increase by using pipelined
requests. Running this a number of times to avoid any issues with occasional
network latency, garbage collection and so on, this number comes out as just
over 43%, still making pipelined requests nearly twice as fast as individual
requests.

Our library of choice for performing pipelined requests is
net-http-pipeline by Eric Hodel. It sits on top of Net::HTTP (part of
the Ruby standard library) and gives us just the right amount of power without
over-complicating things.

 So what’s the problem with testing?

We use the excellent WebMock library for both unit and integration
tests. It’s a great tool for mocking out HTTP requests so you don’t have to
rely on external services for your regular test runs. The problem we had was
that, due to the way net-http-pipeline makes use of the underlying socket
connections within Net::HTTP, it bypasses WebMock’s mocking behaviour.

WebMock’s Net::HTTP support works by effectively overriding the
Net::HTTP#request method
, through which all other request
methods get routed. In the vast majority of cases, net-http-pipeline uses the
“internal use only” method Net::HTTPGenericRequest#exec directly,
bypassing WebMock’s behaviour.

 Introducing webmock-net-http-pipeline

I was butting up against exactly the problem above a couple of days ago: trying
to refactor some tests which seemed to be testing the mocking ability of the
test library, rather than testing the behaviour of the application itself. I
started digging into both WebMock and -pipeline to try and understand why they
didn’t just “work” together; after that digging, I came to the conclusion that
directly modifying either library to work with the other was a yak-shave too
far, so started looking for a more pragmatic approach.

Because the unit tests I was writing were relatively simple (pipeline x GET
requests and do something with the responses), and during the tests I had no
reason to care about over-the-wire performance, I went for the simplest
approach I could: don’t pipeline the requests. Now I don’t mean that my app
didn’t pipeline the requests: I made the library not pipeline them. If we’re
mocking requests, who cares if the “requests” are made in parallel or one by
one?

After getting that far, I shared what I had with some of the team here, at
which point Mat pointed me to a very similar, but cleaner approach he’d
already used in one of our other apps; it’s that approach which is now at the
centre
of this very small, but useful gem.

 How do I use it?

With great ease. Simply install the webmock-net-http-pipeline gem (either
directly or via your project’s Gemfile), require webmock/net/http/pipeline
after you’ve already required webmock, and you’re good to go. Mock your
pipelined requests to your heart’s content.

require "webmock"
require "webmock/net/http/pipeline"

include WebMock::API

host = "www.example.com"
stub_request(:any, host)

http      = Net::HTTP.start(host, 80).tap { |http| http.pipelining = true }
requests  = (1..3).map { Net::HTTP::Get.new("/") }
responses = http.pipeline(requests)

p responses   #=> [#<Net::HTTPOK 200  readbody=true>, ...]

Please note that, as strange as it may sound, webmock-net-http-pipeline does
not actually require net-http-pipeline at all: it simply mimics the behaviour
of that library within the WebMock framework.

If you’ve got any suggestions for tidying the library up, or any comments about
the approach, then please either comment here, raise an issue on GitHub, or
send me a tweet.

 
0
Kudos
 
0
Kudos

Now read this

What open-source can learn about customer engagement

Running a successful open-source project is very much like running a business: it takes skill, time and money (there may not be a direct monetary cost, but there’s an opportunity cost in everything). You need to market your product: why... Continue →