[GRLUG] Testing Comcast vs AT&T

[Adam Tauno Williams]

On Sat, 2011-07-09 at 10:32 -0400, Bob Kline wrote:
> Can you elaborate please?

I'm pretty sure I've gone over this before [probably in response to one
of your many posts about ISP "performance"], but anyway...

1 - The ***ONLY*** way to draw conclusive performance benchmarks
regarding a circuit require the testing of the circuit from near to far
end.  Testing from near end to some remote point far beyond the
end-point of a circuit does *NOT* provide a conclusive measurement of
the circuit's performance.  Please stop claiming that it does.

2 - Making useful/valid performance measurements about a circuit if you
cannot meet the near-far-end requirement of #1 is non-trivial.  You
*certainly* can *NOT* do so using a file-transfer or web-site.
  A - You need to measure a traffic *mix*
  B - You need to measure over a long period of time
  C - You need to measure between many remote points.
  * Otherwise what you are measuring is very close to meaningless.

Also throughput does not correlate 1:1 with latency, or vice-versa.  And
depending on the application one may be more important than the other
(hence 2A).  Application performance is what really matters.

The Internet is vast and complex and subject to all the issues that
effect vast and complex systems:
  - You do not know the connectivity condition or throughput of the
remote.
  - Your route between near-end and remote may change, even moment by
moment, as routes leave and enter the routing table or as traffic
pattern shifts.  [And your actual traffic, not being tiny ICMP packets,
may not follow the route that an ICMP traceroute demonstrates]  
  - The near-network or remote-network may shape your traffic.  And they
may do this in sophisticated ways - such as slowing reducing the
priority of long-lived streams.
  - If the remote is multi-homed you may be communicating with a
physically different location of example.com at different times. [or
maybe even simultaneously].
  - Session setup and overheard with TCP, as well as packet windowing
factors, may impact performance.  And various remotes may calibrate
these differently. Different Operating System and versions can
occasionally demonstrate surprisingly different implementations of
window management and nagle.

If you want to know the throughput of your connection then setup
something that can monitor the interface and produce a graph - over a
long period of time;  this typically involves something like MRTG/RRD
(which can be OpenNMS, Nagios, ZenOSS, there are many these days).  If
you are interested in latency then add ICMP monitoring to the solution
for a few single-homed remote hosts.  Then over the course of a week or
so you can see how latency rises and falls [or possibly doesn't].

If your near-end circuit is actually hitting a throughput ceiling you
will be able to see this in the charts - provided your communication is
not concentrated toward a single remote end [in which case that ceiling
may be a result of something beyond your near-end throughput].

> When I upload or download a
> file of known size in a known time,
> there is an average speed involved
> that clearly has meaning.

Upload/Downloads what? from where? when?  It certainly means something.
Just as the trajectory of a single ejection plume in the chromosphere
means "something".  It is not valid grounds to criticize [or compliment]
a network.



-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.