bandwidth versus latency

Helping Developers Understand Network Performance Limitations

by Brent Chapman
<[email protected]>

Brent Chapman of Covad Communications Company is a UNIX system administration and IP networking expert, author of Majordomo, and coauthor of the O'Reilly & Associates book Building Internet Firewalls.


Software developers and integrators frequently don't understand what performance they can reasonably expect from a corporate network. This often leads them to design and implement software that works great on a LAN but is utterly unusable when deployed across a WAN. To address this at our company, I put together the following short email message to our developers, explaining the basics of network performance (particularly the difference between bandwidth and latency), describing how this affects client-server applications, outlining what performance they should expect from our corporate network, and suggesting guidelines for their client-server development efforts. If you face the same kind of constraints in your business, you might consider providing something similar (with numbers changed appropriately) to help your own developers understand their environment better. You can also apply this same principle in other areas, such as CPU, memory, or disk performance, to help folks understand what they can expect.

• I'd like to suggest that any application or service that you're designing to run over our corporate WAN should be designed so that it performs adequately under the following conditions:
• Single user

• Single task (no other applications using the WAN link simultaneously)

• 128 kb/s bandwidth

• 100 ms latency (one way)

We will have much more bandwidth than that available to most corporate sites (but not all sites, and not all the time). We should use that bandwidth, however, to accommodate multiple simultaneous sessions (from multiple users, from multitasking individuals, or from some combination of both), rather than counting on it to provide adequate performance for any individual session.

If you keep these guidelines in mind as you develop applications and services, it will go a long way toward ensuring that our applications and services work well across our entire corporate WAN.

I'm not suggesting these numbers arbitrarily. There is some logic behind them: 128 kb/s bandwidth is what you can expect out of an ISDN line. Right now, for cost reasons, all our corporate regional offices are limited to 128 kb/s burst speed for bandwidth, on a public-carrier frame relay WAN. Each office should have more bandwidth available in a few months when [something confidential] happens, but they'll still have backup connectivity that's limited to 128 kb/s (either their current frame relay service, or dial-on-demand ISDN service). And they'll still have folks who want to use the apps from home, some of whom will be limited to only ISDN bandwidth (128 kb/s).

As for latency, 100 ms one-way is about what we can expect out of our coast-to-coast frame relay­based IP WAN during peak usage periods each day. One-way latency on our LAN at headquarters is usually 3-5 ms or less. A task where a client/server application or service does 10 round-trip conversations between the client and the server on a path with 5 ms one-way latency can still complete in 100 ms (1/10th of a second), which "feels fast"; that same task on a link with 100 ms latency will take 2000 ms (2 seconds), which almost certainly "feels slow." . . .

The upshot of this for software developers is:

• Don't develop applications or services that need to transfer data between the client and server at more than 128 kb/s in order to avoid "feeling slow."

• Don't develop applications or services that require lots of little back-and-forth messages between the client and server; you'll get eaten alive by the latency.

Remember to include whatever numbers are appropriate for your site and to clearly outline the performance constraints and expectations plus the technical explanation for them. Since it's easier for everyone to accommodate design specifications up front, rather than retrofit applications after the fact, everyone will appreciate this "heads up" information.