What's the max speed on Ethernet?

My port scanner (masscan) can transmit at the maximum speed of Ethernet. So what does that mean?

A 1-gbps Ethernet link has the precise bit-rate of 1000000000.0 bits-per-second (bps). Depending on the accuracy of the clock oscillator, this may be a few bps faster or slower, but at least this is the standard rate.

But for port scanning, the more important metric is "packets-per-second" (pps). We have to divided the bit rate by packet size in order to get the packet rate. In this post, I "show my work" showing the math to derive this.

Layer 1

TCP/IP is at layer 3 and 4 of the OSI model. There is additional overhead in layers 1 and 2 that we have to account for.

The layer 1 overhead is the following components:

• 12 bytes for the inter-frame gap (silence)
• 8 bytes for the preamble
• 60 byte minimum layer 2
• 4 byte CRC

Ethernet demands a 12-byte gap of "silence" on the wire when nothing is being transmitted. That's how it can tell when one packet ends and another begins. This is known as the "inter-frame gap".

After the silence comes the "preamble" bits in the sequence ...010101011, or alternating ones and zeroes ending in two ones. That's because it takes several bits for the sending and receiving clocks to become synchronized during which the first few bits will be lost.

Finally, at the end of every Ethernet packet is a the CRC (also known as the FCS). This is a checksum over the entire packet in order to detect if anything got corrupted. After the CRC comes the inter-frame gap of the next frame.

A further complication is that there is a minimum of 60 bytes of contents (minus the above fields). That's because back in the 1980s, multiple Ethernet devices were attached to the same wire - a wire that could stretch for kilometers around a college campus. The speed of light is not infinite. Two different machines could transmit for some time before they became aware of the colliding signal from the other machine. The minimum packet size is the time the transmitter must keep transmitting for their signal to reach the far ends of the cable, and for any returned signal to travel back. We've had "switched" Ethernet for 20 years now where collisions no longer happen, but we still have this legacy.

When you add up the above fields, you'll see that the minimum Ethernet packet size is 84 total bytes at layer 1. Divide this in 1,000,000,000 bits-per-second, and you get the number 1,488,095 packets-per-second. In case you are glancing over this blog post and not reading the details, I'll put this in big scary letters to catch your attention:

1,488,095 pps

These numbers scale up/down precisely according to speed. The original 10-mbps Ethernet from the 1980s had a maximum rate of 14,880 packets/second. Today's 10-Gbps Ethernet has a maximum rate of 14,880,952 packets/second.

Maximum TCP/IP bit rate

The above calculations are for Ethernet, but remember that Ethernet only gets you as far as the nearest router. At the router, all that Ethernet information is stripped off.

When doing a port scan, the size of the packets will be 20 bytes for the IP header and 20 bytes for the TCP header, or 40 bytes total. That's less than half the size of the minimum Ethernet frame. Doing the math, this comes out to an effective bit-rate for just the TCP/IP parts of 476-mbps. Again, in case you aren't paying attention:

1488095 X 40 = 476 mbps

Thus, when trying to port scan at the full 1-gbps of your network adapter, your ISP might only measure the TCP/IP bandwidth of 476-mbps. That depends, though, on the link technology your ISP uses to connect to the rest of the Internet. Your ISP may charge you for the layer 1 that happens on the other end of the router, which is usually less than Ethernet's overhead, but more than zero.

Note that this disparity only happens because we are using very small packets, where per-packet overhead dominates. In normal Internet usage, packets are 500-bytes in size, and the per-packet overhead is usually ignored.

Cheating at Ethernet 

In practice, you can cheat at Ethernet. The inter-frame gap, preamble, and minimum packet size don't need to be that large. You can shrink the inter-frame gap from 12 bytes down to 5. You can shrink the preamble from 8 down to 4 bytes. You can get rid of the minimum packet size completely.

The size of the packet then shrinks from 84 bytes down to 67 bytes:

• 5 bytes for inter-fame gap
• 4 bytes for preamble
• 14 bytes for Ethernet
• 20 bytes for IP
• 20 bytes for TCP
• 4 byte CRC

On a 1-gbps link, this comes out to a speed of 1,865,671 packets per second. This increases your effective TCP/IP-only bit-rate from  476-mbps to 597-mbps.

But bad things happen to cheaters.

For example, let's say that you are shrinking the inter-frame gap and preamble, then sending packets at the maximum rate into the switch. When the switch forwards the packet onto the next hop, it'll use the correct gap/preamble size. This causes two problems for you. The first is some "buffer bloat" and latency, forcing the switch to fill whatever buffer space it'll have to store packets. Then it will cause the switch to drop packets, reducing your 1.8mpps down to the 1.5mpps it can handle.

On the other hand, it may allow you to get something for free. At least in one instance, it appeared as if it allowed us to exceed the bandwidth limitations of our ISP. The router would happily forward such packets, but the accounting function didn't record "invalid" packets. Thus, we were able to transmit packets for "free".

Because things break unexpectedly, you probably don't want to cheat in the real-world, but it's still useful knowledge. When testing, simply pushing to the limits causes you to miss flaws. To properly test something, you need to push past the limits.

Other problems

For some reason, Linux can't hit the 1.488-million-packets/second limit on 1-gbps Ethernet. Using the identical system, but a 10-gbps Ethernet, Linux can exceed 2-million-packets/second. While this is far below the max 10-gbps limit, it's above the 1-gbps limit.

But in practice, Linux only transmits about ~1.3-million-packets/second on a 1-gbps link. I don't know why this is.