Cisco routers & their performance – with ZBFW, MQC, NAT, IPSec, et cetera enabled

   8Mbps -> Cisco 860VAE
  15Mbps -> Cisco 819
  15Mbps -> Cisco 880
  15Mbps -> Cisco 1921 (1RU)
  25Mbps -> Cisco 890 (1RU)
  25Mbps -> Cisco 2901 (1RU)
  35Mbps -> Cisco 2911 (2RU)
  50Mbps -> Cisco 2921 (2RU)
  75Mbps -> Cisco 2951 (2RU)
 100Mbps -> Cisco 3925 (3RU)
 150Mbps -> Cisco 3945 (3RU)
 4321 ->  50Mbps ->  100Mbps throughput performance license*
 4331 -> 100Mbps ->  300Mbps throughput performance license*
 4351 -> 200Mbps ->  400Mbps throughput performance license*
 4431 -> 500Mbps -> 1000Mbps throughput performance license*
 4451-X -> 1000Mbps -> 2000Mbps throughput performance license*
 ASR1001-X -> up to 20Gbps (1RU) integrated 2.5Gbps ESP, upgradable via license; Integrated RP 8G RAM, upgradable to 16GB*
 ASR1002-X -> up to 36Gbps (2RU) integrated 5Gbps ESP, upgradable to 10, 20, or 36Gbps; Integrated RP 4G RAM, upgradable to 8GB or 16GB*
 ASR1004 -> up to 40Gbps (4RU) ESP10, ESP10-N, ESP20 (all single); ESP40 (single or dual); ASR1000-RP1 4GB RAM, ASR1000-RP2 8GB or 16GB RAM (both single)*
 ASR1006 -> up to 40Gbps (6RU) ESP10, ESP10-N, ESP20, ESP40, ESP100 (all single or dual); ASR1000-RP1 4GB RAM, ASR1000-RP2 8GB or 16GB RAM (both single)*
 ASR10013 -> up to 40Gbps (13RU) ESP40, ESP100, ESP200 (all single or dual); ASR1000-RP2 8GB or 16GB RAM (both single)
 *IOS-XE: platform hardware throughput level XYZ

FAX OVER IP FoIP V17 V.17 G3 14400baud V34 33600baud V.34 SuperG3 T38
Phase A: Call establishment
CNG: comfort noise generation
 FAX generates it to identify itself as a FAX rather than a human
CED: constant/flatline tone pulse - determine G3 speed
ANSAM: SuperG3 capable, negotiation..
Phase 1: Network Interaction: V.34 or not?
Phase 2: Line probing & Ranging: Excnahge capab, round-trip delay,
 determine bandwidth and SNR
Phase 3: Primary Channel Equalizer Training: goes back to Phase 2
 if modulate and demodulate
 signals responding as expected
Phase B: pre-message procedures:
Called fax:
CSI: called subscriber ID (name of company, called number, et cetera..)
DIS: digital identification signal: the capabilities: compression rates, speeds..
Calling fax:
TSI: transmitting subscriber ID (name of company, called number, et cetera..)
DCS: digital command signal: based on the DIS this is what we will use
TCF: training check (not used in V.34), "quality of sound sent across"
Called fax:
CFR: confirmation to receive - TCF success, everything is okay
FTT: failure to train - TCF has failed..
Phase C: message transmission
Sending device starts short training (resnyc) then (w/ or w/o ECM error correction mode)
sending FAX image data FCD FCD FCD .. FCD (frames called Fax Code Data) then when all sent
RCP RCP RCP (exactly 3 RCP frames - return to control for partial page) "I finished with this section"
Phase D: post-message procedure
Sending side:
PPS - partial page signal: chacklist of how many frames has been sent
 PC page count, BC block count, FC frame count
Receiving side:
MCF - Message confirmation: PC BC FC actually transferred
PPR - if something went wrong.. Partial page request (missing frames?)
Phase C & D without ECM:
Sending side:
short training (resync) -> T4-non-ecm-data -> MPS (partial page signal)
short training (resync) -> T4-non-ecm-data -> repeat until end of data then
EOP end of procedure
Receiving side:
MCF - Message confirmation: PC BC FC actually transferred
RTP - if something went wrong.. and Retrain positive RTP have some concerns go back to Ph.B/Retrain negative RTN did not work at all
Phase E: call release
Sending side:
DCN disconnect
Training check frame (TCF)
The training check frame is the phase of a fax call when the sending fax transmits a sequence of zeros to the receiving fax machine at the highest common data rate negotiated during prior phases. This check determines whether the line quality is adequate to pass information at the desired rate. If the line quality is good, the receiving fax machine receives this sequence of zeros without error. If line quality is not good, bit errors will occur during reception, and the total amount of zeros is not received.
The duration of the zero sequence sent by the transmitting fax is 1.5 seconds. The T.30 protocol states that to successfully receive this training data, 1.5 seconds +/- 10% of these zero symbols should be received. If successful, the receiving fax sends back a confirm signal (CFR) to the transmitting fax. If the standard is not met, the receiving fax sends back a fail to train (FTT) signal, indicating to the transmitting fax that a lower data rate should be tried.
NaturalFax reports the results of training information. This information can be used to determine the quality of the training event, and therefore the quality of the line. This metric can be used to determine whether a receive problem is the result of poor line quality, insufficient training, or a compatibility issue. The rx_training_zeros metric reports the length of zeros received in tens of milliseconds. A good value for the TCF is greater than 1.35 seconds. If values less than this are reported, the receiver is not able to demodulate the incoming signal appropriately. This results in bit errors.


Cisco router/switch/AIM performance sheets


How to use IPerf properly – additions to the TCP throughput post

IPerf is a tool to measure network throughput.

It is very efficient but if one use it without proper knowledge the
results can lead you to trouble.

It can measure with both TCP and UDP.

You should know how TCP and UDP works. In TCP we have
acknowledges periodically by the nature of the operation. In UDP
there are no acknowledges by the protocol, the application should
handle the packetloss (if any..) and the control.

Let’s see two examples.

FTP/SCP copy. This connection is using TCP. Before we can send
the next packets we should wait for the acknowledge of the sent
packets. Send-send-wait4ack-send-send-wait4ack-send-send-wait..
If some packets lost TCP will resend it.

VoIP telephony. This connection is using UDP. The phones sending
packets periodically at a predefined rate with predefined size
of packets therefore at a predefined bandwidth. No matter what
happens (drops can occure) IP phone will send-send-send-send…

You can see that the TCP flow is “under control”, but UDP can go
“out of control”. This we can use for measuring the throughput.

The IPerf tool is a client-server application. Basicaly the client
sends the data to the server. After they joined the client starts
to send data to the server based on the parameters on both sides.
We can set to make a single direction, bidirectional one-after-one
and bidirectional parallel measurement. We can also set to do the
test with one or several other parallel sessions and we can set
the size of the packets also. This means that with the proper
settings you can simulate file transfers (FTP, SCP, SMB..) or for
example VoIP calls (if you know the codec sample rate/sample size
you can calculate the proper packet size and bandwidth parameters).

Because when you measure you send packets from client to server
you should consider this behavior when you set up the application
instances. The direction is the key.

If you want to test the FTP download speed from A to B you should
start the server at site B and the client on site A but the FTP
server is at site A and the client is at site B. To test the upload
from A to B you should swap the roles: A should be the client and
B should be the server. This is important because you should set
the receive window size to the value you want to measure at. Here
are some examples: standard is 64KB, Windows XP/Vista/Win7 17.5KB,
FTP 8KB, SCP 64KB, SMB 4K blokk size.. Or you should check your
application what you want to test (SAP for example). This should be
set at least the server side, but you should set it on client side also if
you want to measure both directions parallel.

Testing/simulating VoIP is also not so complicated, you only need
the codec/sample rate/sample size/sample interval and you can
calculate the packet payload size and the bandwidth parameters.
You can use the already attached picture above (assembled by me).

Let’s start the command line part.

Usage: iperf [-s|-c host] [-p port] [-t secs] [-w bytes] …
       iperf [-h|–help] [-v|–version]

  -f, –format    [kmKM]   format to report: Kbits, Mbits, KBytes, MBytes
  -i, –interval  #        seconds between periodic bandwidth reports
  -l, –len       #[KM]    length of buffer to read or write (default 8 KB)
  -m, –print_mss          print TCP maximum segment size (MTU – TCP/IP header)
  -p, –port      #        server port to listen on/connect to
  -u, –udp                use UDP rather than TCP
  -w, –window    #[KM]    TCP window size (socket buffer size)
  -B, –bind         bind to , an interface or multicast address
  -M, –mss       #        set TCP maximum segment size (MTU – 40 bytes)
  -N, –nodelay            set TCP no delay, disabling Nagle’s Algorithm
  -V, –IPv6Version        Set the domain to IPv6

Server specific:
  -s, –server             run in server mode
  -D, –daemon    run the server as a daemon

Client specific:
  -a, –tcp_bandwidth \    for TCP, bandwidth to send at in bits/sec
  #[KM]    (default no bandwidth limit used)
  -b, –bandwidth #[KM]    for UDP, bandwidth to send at in bits/sec
                           (default 1 Mbit/sec, implies -u)
  -c, –client       run in client mode, connecting to 
  -n, –num       #[KM]    number of bytes to transmit (instead of -t)
  -t, –time      #        time in seconds to transmit for (default 10 secs)
  -F, –fileinput    input the data to be transmitted from a file
  -I, –stdin              input the data to be transmitted from stdin
  -P, –parallel  #        number of parallel client threads to run
  -S, –tos       #        set type-of-service for outgoing packets
  -T, –ttl       #        time-to-live, for multicast (default 1)
  -W, –windowSizeSuggest  Run the client so as to suggest a suitable window size (default off)

  -h, –help               print this message and quit
  -v, –version            print version information and quit

[KM] Indicates options that support a K or M suffix for kilo- or mega-

The TCP window size option can be set by the environment variable
TCP_WINDOW_SIZE. Most other options can be set by an environment variable

To measure the throughput for FTP connection from A to B you should issue
Site B: iperf -s -w 8K
Site A: iperf -c site.B.IP.address -fk -i 10 -m -t 60
This will start a simple FTP-like connection and do it for 60 seconds,
make bandwidth report in every 10s in kbps (printing MSS).

To measure the throughput for SCP connection from A to B issue this:
Site B: iperf -s -w 64K
Site A: iperf -c site.B.IP.address -fk -i 10 -m -t 60
This will start a simple SCP-like connection and do it for 60 seconds,
make bandwidth report in every 10s in kbps (printing MSS).

Please check the TCP throughput calculations article to evaluate the results.

To generate VoIP/G711-like, bidirectional traffic for 4 “calls” issue this:
Site A: iperf -c site.B.IP.address -u -mss 160 -b 65000 -S 184 -P 4 -fk -i 10 -t 300
Site B: iperf -s -u -mss 160 -b 65000 -S 184 -P 4 -fk -i 10
This will start 4 simple bidirectional VoIP/G711-like flows for 300 seconds
marked with DSCP EF (TOS 184), make bandwidth report in every 10s in kbps.

To check a link to see if you can use it at the full capacity you
can saturate the link with these commands (the -b value must be
high enough, equal or more than the link capacity):
Site A: iperf -c site.B.IP.address -u -mss 160 -b 10M -fk -i 10 -t 300
Site B: iperf -s -u -b 10M -fk -i 10
You can do 2 separate or one simple bidirectional measurement.

The JPerf is a GUI for IPerf, you can set the parameters in a Java GUI.

A functionality decreased IPerf is implemented in the Cisco IOSes.
It’s name is TTCP, it is unsupported and you should know that in
this case you utilise the router CPU and this can limit/tamper the
results, use it on your own risk and and not trust it.

Please always check both sides when you evaluate the results.
You should check what is sent and what is received and see the
correct conclusion. Hope this helps.

Maximum TCP throughput – the user experiences “low bandwidth” when up/downloading

Maximum TCP throughput calculations:
There are several important things what you should know before reading this.
Please check TCP/IP fundamentals at
Common window sizes:
Standard 64KB
Windows XP 17.5KB
SMB 16K/4K

TCP throughput: << this is a calculator, it can show you the throughut weakness 

Data throughput in TCP based on TCP window size and latency:
FTP TCP window 8KB -> 65356bit
Maximum TCP transfer = 65356bit/0.06s=1.038Mbps


General TCP window 64KB -> 524288bit
Maximum TCP transfer = 524288bit/0.06s=8738133,33bps = 8.533.33Mbps


DataMover TCP window 256KB -> 2097152bit
Maximum TCP transfer = 2097152bit/0.06s=33.33Mbps


Throughput can never exceed window size divided by round-trip time.
window = 17520byte = 140160
rtt 60ms = 0.06s
Max link throughput (based on window and rtt) = 140160bit/0.06s=2336000 = 2281,25kbps


The capacity of a pipe is its bandwidth multiplied by round-trip time.
BW = 2Mbps = 2097152bps
rtt = 60ms = 0.06s
Max link capacity (based on BW and rtt) = 125829,12bit = 15728,64byte


Formula to calculate the optimal TCP window size:
Bandwidth-in-bits-per-second * Round-trip-latency-in-seconds = TCP window size in bits / 8 = TCP window size in bytes
BW = 2Mbps = 2097152bps
rtt = 60ms = 0.06s
2097152bps * 0.06s = 125829,12bit = 15728,64byte — ideal window size


Formula to calculate Maximum Latency for a desired throughput
TCP-window-size-bits / Desired-throughput-in-bits-per-second = Maximum RTT Latency
TCP window size = 17520byte = 140160bit
desired BW = 2Mbps = 2097152bps
140160bit / 2097152bps = ~0.067ms is the max RTT for the desired 2M throughput



Please consider that these values are IP packets, not frames (or Layer2 bandwidths) or the packets’ data transers.
Layer2 overheads will “decrease” the link capacity: