queueing reverse proxy (deprecated)

The qrp package itself is deprecated.

The author of qrp has moved onto Unicorn

The Rubyforge qrp project page is now renamed Quack Ruby Projects for one-off projects that otherwise would not have a home.

Rubyforge: rubyforge.org/projects/qrp/ git repository: bogomips.org/ruby/qrp.git

Queueing Reverse Proxy (qrp)

  Ever pick the wrong line at the checkout counters in a crowded store?
  This is what happens to HTTP requests when you mix a multi-threaded
  Mongrel with Rails, which is single-threaded.

  qrp aims to be the simplest (worse-is-better) solution and have the
  lowest (adverse) impact to an existing setup.

Background:

  An existing Rails site running Mongrel with nginx proxying to them.

  Unlike Apache, nginx fully buffers HTTP requests from clients before
  passing them off to Mongrels, which allows Mongrels to dedicate more
  cycles to running Rails itself.

Problem:

  Rails is single-threaded; this is (probably) not easily fixable.

  By default, Mongrel will accept and queue requests while Rails is
  handling another request.

  Some Rails actions will take longer than others; and sometimes
  several seconds can be required to respond to an HTTP request.

  This problem is exacerbated if the Rails application queries
  third-party servers for information.

  Any queued requests inside Mongrel running Rails must wait until a
  slow Rails action has finished before they can run.

  If another Mongrel in the pool becomes free, then the requests that
  got queued behind a still-busy Mongrel would still be stuck and unable
  to get to the free Mongrel.

  Disabling concurrency on the Rails Mongrel (with "num_processors: 1"
  in the config)[1] will cause clients to be rejected outright and users
  will see 502 (Bad Gateway) errors.

  Bad Gateway errors getting returned to clients are bad, a slightly
  slower site is still better than a broken site.

  The developers also lack the resources to migrate to thread-safe
  platform (such as Merb or Waves) at the moment.

Solution:

  Disable concurrency in Mongrels running Rails is part of the solution.

  Then setup a qrp or two as a backup member in your nginx
  configuration.

  Connections will normally go directly from nginx to Rails Mongrels (as
  before).  However if all your regular Mongrels are busy, *then* nginx
  will send requests to the backup qrp instance(s).

  Once a request gets to qrp, qrp will retry the all the members in a
  given pool until a connection can be made and a response is returned.

  This avoids extra data copies of requests for the common (non-busy)
  case, and requires few changes to any existing infrastructure.

  Having fail_timeout=0 in the nginx config for every member of the
  Rails pool will allow nginx to immediately re-add a Rails Mongrel to
  the pool once the Rails Mongrel has finished processing.

  --- highlights of the nginx config:
  upstream mongrel {
    server 0:3000 fail_timeout=0; # Rails
    server 0:3001 fail_timeout=0; # Rails
    server 0:3002 fail_timeout=0; # Rails
    server 0:3003 fail_timeout=0; # Rails
    server 0:3500 backup;         # qrp
    server 0:3501 backup;         # qrp
  }
  --- highlights of the qrp config:
  # same Rails upstreams as in the nginx config
  upstreams:
   - 0:3000
   - 0:3001
   - 0:3002
   - 0:3003
  # ...
  --- highlight of the mongrel config[1]:
  num_processors: 1

Other existing solutions (and why I chose qrp):

  Fair-proxy balancer patch for nginx - this can keep new connections
  away from busy Mongrels, but if all (concurrency-disabled) Mongrels in
  your pool get busy, then you'll still get 502 errors.

  HAProxy - This will queue requests for you, but only if it makes all
  the connections to the backends itself.  This means you cannot make
  other HTTP connections to the backends without confusing HAProxy;
  which (IMHO) defeats the purpose of using HTTP over a custom
  protocol.

  Swiftiply - admittedly I haven't tried it.  It seems to require
  changes to our current infrastructure in deployment and monitoring
  tools.  Additionally, the extra layer between nginx and Mongrel hurts
  performance for _every_ request, not just those that get unlucky.
  This also seems to take away the flexibility of being able to talk to
  any individual Mongrel process using plain HTTP.

Caveats:

  Do not use qrp to proxy to upstreams that may return large responses.
  qrp will slurp the entire response into memory before sending back to
  the original client.  qrp was designed for dynamically-generated
  webpages that can be rendered within a user's web browser, not for
  serving large files.

Logging Format:

  "START" lines are logged when a request hits qrp:

    2008-07-10T17:36:30+0000 14952.65 1 START GET /path?q=s for 127.0.0.1
      timestamp ---/          /    / /  /     /     /              /
      PID -------------------/    / /  /     /     /              /
      request ID ----------------/ /  /     /     /              /
      active thread count --------/  /     /     /              /
      "START" (request started) ----/     /     /              /
      HTTP method -----------------------/     /              /
      request URI ----------------------------/              /
      client IP (or X-Forwarded-For) -----------------------/

  "OK" lines are logged when a request is complete:

    2008-07-10T17:36:30+0000 14952.65 1 OK 0:9007 197 0 0.260095
      timestamp ---/          /    / /  /   /     /  /  /
      PID -------------------/    / /  /   /     /  /  /
      request ID ----------------/ /  /   /     /  /  /
      active thread count --------/  /   /     /  /  /
      "OK" (request completed) -----/   /     /  /  /
      host:port backend used ----------/     /  /  /
      response size bytes (incl. header) ---/  /  /
      retry loops ----------------------------/  /
      total time (START - finish) --------------/

    request ID: the request number for a given process, this is an
    integer that increments every time a request is passed to this qrp
    process.

    active thread count: number of active qrp threads in this process
    including this one.  This can be used to tell how busy qrp was
    at the time the request completed.

    retry loops: number of times a request passed through all available
    backends without finding one free.  Lower is better, a high number
    indicates that your backends are overloaded.

Footnotes:

 [1] - The current version of mongrel (1.1.3) does not handle the
       -n/--num-procs command-line option, and hence the current
       mongrel_cluster (1.0.5) is broken with it:
         http://mongrel.rubyforge.org/ticket/14

       A better solution would be to use mongrel_cow_cluster (also a
       development of mine) as it handles the "num_processors:"
       directive correctly in the config file and also supports rolling
       restarts.

Files

Classes/Modules

Methods

[Validate]

Generated with the Darkfish Rdoc Generator 1.1.6.