XS Blueprints:rproxy

Notes on an rproxy implenentation for the XS.

We want to have an optional, easy to switch on rproxy setup on the XS. rproxy can replace or complement the main XS http proxy - if we have a stable, efficient, high quality http proxy on the XS, pairing it with rproxy would be a good idea.

As long as we are using Squid, it's not such a good idea :-(

Rationale

rproxy is a proxy that expends HTTP 1.1 proxying and provides much better performance over very bandwidth constrained linkes. It requires an "upstream" rproxy to match the downstream rproxy. For deployments where the schools are connected to the internet with very limited uplinks, rproxy can be a huge win.

Modern internet is made of moderately dynamic content --- the content is mostly static but un-cacheable due to rotating banners or "latest news" boxes on the side. Even when content changes, the template for the content is often 80% of the HTML downloaded. rproxy provides huge gains with such content.

More about what rproxy is

• http://rproxy.samba.org/
• http://ozlabs.org/~rusty/rproxy.html

How it would work

For a deployment with limited connectivity into the schools

• setup a co-located rproxy server at the ISP
• enable rproxy on the XSs deployed, pointing to the ISP

Note: the rproxy server at the ISP must only accept requests from valid clients - either from a defined netblock or whitelisted IP addresses. An open rproxy is an open proxy!

Issues with rproxy

• The working version uses an old version of librsync - which may have odd bugs and is generally unsupported, untouched, unknown
• There's a half-finished port to the new librsync. Martin Poole started it a while ago, and abandoned it.
• Does not stream the content through - blocks until the content is fully fetched before delivery to requester.
• Unknown performance and memory footprint when hit by many users concurrently
• Needs packaging, init scripts, etc...
• Unclear if it behaves reasonably well with ajax "streamed" requests - it may break or add problematic latencies, etc

TO DO

• Test and ensure it works with modern internet usage patterns. Gmail, youtube, etc...
• package, configuration, init scripts...
• Test under load, paired with a conventional http proxy - may need performance / memory trimming
• Port to modern librsync
• Make it work with streamed content (possibly the hardest task!)