Introduction

Despite the focus on memory testing, in release 0.24 I also added network bandwidth testing. The results are graphed.

bandwidth also tests some libc functions and, under Linux, it attempts to test framebuffer memory access speed if the framebuffer device is available.

This program is open source and covered by the GPL license. Although I wrote it mainly for my own benefit, I am also providing it pro bono, i.e. for the public good.

Change log

Release 0.32

Release 0.31

Release 0.30

Release 0.29

Release 0.28

Release 0.27

Release 0.26

Release 0.25

1. Test is now bidirectional.
2. Port# is specifiable.

Release 0.24

The node that runs the test is the leader and the others are the transponders.

The network test cannot be combined with the memory bandwidth tests.

Just to clarify, you need two computers to run the test:

• Machine A: The computer that is running (leading) the test.
• Machine B: The computer that is the transponder.
• Sent data is from A to B.
• Received data is from B to A.

Release 0.23

• Linux
• Mac OS/X
• 32-bit Windows
• Windows Mobile ARM

• x86: performs 128- and 32-bit transfers
• x86_64: performs 128- and 64-bit transfers

For each architecture, I've implemented optimized core routines in assembly.

Bandwidth 0.23 builds on the novelty of 0.22's register-to-register transfer speeds by including transfers to, from, and between vector registers (XMM). It also adds a test of memory copy speeds.

Revision 0.21 added now performs both sequential and random reading and writing of a range of progressively larger chunks of memory to permit you to effectively test several types of memory:

• Level 1 cache
• Level 2 cache
• Main memory

Revision 0.20 added a novel and helpful improvement: Graphing. Using my bmplib, the program generates an 1280x720-pixel graph depicting the results.

Revision 0.19 added another novel improvement: When it performs 128-bit writes using SSE2 instructions it does it in two ways:

1. It writes into the caches with MOVDQA.
2. It writes bypassing the caches w/MOVNTDQ.

Results from iOS

Observations of running one instance of bandwidth

The first interesting thing to notice is the difference in performance between 32, 64, and 128 bit transfers on the same processor. These differences show that if programmers were to go through the trouble to revise software to use 64 or 128 bit transfers, where appropriate and especially making them aligned to appropriate byte boundaries and sequential, great speed-ups could be achieved.

A second observation is the importance of having fast DRAM. The latest DRAM overclocked can give stupendous results.

A third observation is the remarkable difference in speeds between memory types. In some cases the L1 cache is more than twice as fast as L2, and L1 is up to 9 times faster than main memory, whereas L2 is often 3 times faster than DRAM.

Note: Since I added graphing to bandwidth, I am no longer updating this table.

Running multiple instances of bandwidth simultaneously

On my Core i5 dual-core system, with DDR3 (PC3-8500) memory, the maximum RAM bandwidth ought to be 8500 MB/second.

Running on just one core:

• Reading, it maxes out at ~7050 MB/second from main memory.
• Writing through the caches, it maxes out at ~5120 MB/second to main memory.
• Writing and bypassing the caches, it maxes out at ~5520 MB/second to main memory.

Read:

Write, bypassing caches:

When I've got two instances of bandwidth running at the same time, one on each core, the picture is a little different but not much.

• Reading, the total bandwidth from main memory is ~8000 MB/second, nearing the memory's maximum, or 14% faster than running just one instance of bandwidth.
• Writing without bypassing the caches, the total bandwidth to main memory is ~5650 MB/second, which is 10% faster than one instance.
• Writing with the cache bypass, the total bandwidth to main memory is ~6050 MB/second, which is 10% faster than one instance.

Read:

Write, not bypassing caches:

Write, bypassing caches:

Thus, to really ascertain the upper performance limit of the main memory, it behooves the earnest benchmarker to run multiple instances of bandwidth and sum the results.

Graphs for memory bandwidth tests

Intel Xeon E5-2690, rated at 2.9 GHz with Turbo Boost speed 3.8 Ghz, with two sticks of DDR3 1600 MHz RAM (rated 12.8 GB/s/channel but not using dual-channel mode), running 64-bit Linux.

Intel Core i5-2520M, rated at 2.5 GHz but running at the Turbo Boost speed 3.2 Ghz, with two sticks of DDR3 1333 MHz RAM (rated 10.6 GB/s/channel), running 64-bit Ubuntu 10.

Intel Core i5-520M at 2.4 GHz with 3MB L2 cache, running Mac OS/X Snow Leopard, 64-bit routines:

Intel Celeron at 2.8 GHz with 128kB L2 cache and PC2700 DDR memory running 32-bit Linux 2.6:

Note the difference

Not using the nice command:

Using nice -n -2:

Intel Core 2 Duo P8600 at 2.4 GHz with 3MB L2 cache, running Mac OS/X Snow Leopard, 64-bit routines:

Intel Core i5-540M at 2.53 to 3.07 GHz with 3MB L3 cache, running 64-bit Linux:

Intel Core i3-330M at 2.16 GHz with 3MB L3 cache, running 64-bit Linux:

Intel Pentium T4300 at 2.1 GHz with 1MB L2 cache, running 64-bit Linux:

Intel Core 2 Duo T5600 at 1.83 GHz, with 2MB L2 cache, in a Mac Mini:

Intel Core 2 Duo T5200 at 1.6 GHz, with 2MB L2 cache, running Linux 64-bit:

Intel Core 2 Duo E8400 at 3 GHz, with 6 MB L2 cache:

OMAP 3530 Cortex A8 in a Beagle Board running 32-bit Linux 2.6.29 at 720 MHz (turbo mode):

Graphs for network bandwidth tests

Here is another loopback test but for a 2.8 GHz Celeron running 32-bit Linux:

And here is the Mac communicating with the Linux box over Wifi:

Download

• Sources

Be nice

Compiling

On Intel Linux, you need a copy of NASM and the GCC suite. A decent distro will supply these. Simply type "make bandwidth32" or "make bandwidth64" to produce the Intel executables.

Lastly, to compile for 32-bit desktop Windows you need the GCC toolchain in the form of Cygwin. Type "make bandwidth32". The executable will not run outside of Cygwin because it requires a Cygwin DLL.

Commentary

Big AVX qualification

Sequential versus random memory access

Generalizations about memory and register performance

1. Reading is usually faster than writing.
2. L1 cache accesses are significantly faster than L2 accesses e.g. by a factor of 2.
3. L1 cache accesses are much faster than main memory accesses e.g. by a factor of 5 or more.
4. L2 cache accesses are faster than main memory accesses e.g. by a factor of 3 or more.
5. L2 cache writing is usually significantly slower than L2 reading. This is because existing data in the cache has to be flushed out to main memory before it can be replaced.
6. If the L2 cache is in write-through mode then L2 writing will be very slow and more on par with main memory write speeds.
7. Main memory is slower to write than to read. This is just the nature of DRAM. It takes time to charge or discharge the capacitor that is in each DRAM memory cell whereas reading it is much faster.
8. Framebuffer accesses are usually much slower than main memory.
9. However framebuffer writing is usually faster than framebuffer reading.
10. C library memcpy and memset are often pretty slow; perhaps this is due to unaligned loads and stores and/or insufficient optimization.
11. Register-to-register transfers are the fastest possible transfers.
12. Register-to/from-stack are often half as fast as register-to-register transfers.

A historical addendum

SSE4 vector-to/from-register transfers

What about ganged mode?

Whether you are using dual-channel mode depends not only on your motherboard and chipset, but also on whether your BIOS is configured for it.

The default BIOS setting for this, referred to as the DCT or DCTs feature, is often unganged i.e. the two memory sticks are not acting together.

What is a DCT? This refers to a DRAM Controller. The fact that in the unganged mode each channel is independent means that there is need for a DCT for each channel. A motherboard and chipset supporting a wide path to your RAM will likely provide as many DCTs as there are channels, as needed for unganged mode.

In the BIOS settings you will either see a simple selection for ganged versus unganged mode, or it may refer to which actual DRAM controllers are assigned to which channels e.g. DCT0 (first DRAM controller) goes to channel A and and DCT1 (second controller) manages channel B.

If your computer doesn't have an old-style PC BIOS but rather uses UEFI like Apple devices do, you may not have the option to alter the ganged/unganged setting. So consumers are disempowered firstly in that settings may not be accessible, but secondly in that the details of how UEFI works are actually proprietary and subject to an NDA. Therefore: UEFI is bad for consumers.

Q: Does ganged mode actually improve speed?
A: People say it generally does not improve it, or reduce it, which is why it is not enabled by default. Ganged and unganged offer about the same performance for any given application running on one core.

Q: If unganged mode requires more silicon (one DCT per channel) but has the same performance as ganged mode, then why not enable ganged by default and remove the extra DRAM controllers?
A: Because unganged mode offers more flexibility in letting multiple cores and hyperthreads access different areas of memory at once.

Q: How can maximum performance be achieved realistically?
A: With the help of the OS, a program could allocate its in-memory data sets across the DIMMs (obviously this is a virtual memory issue) to avoid the bottleneck of all of its data going through just one channel.

Why is L1 cache read speed so amazingly fast on XYZ CPU?

After a few calculations, it became clear why this was happening:

1. 96 GB at the processor's peak speed of 3.2 GHz means a transfer speed of 32 bytes i.e. 256 bits per cycle. But the XMM registers are only 128 bits wide...

2. Newer Intel CPUs have YMM registers however, used by Advanced Vector Extensions (AVX) instructions. These are 256 bits wide and are composed of the existing 16 XMM registers with an additional 128 bits per register for a combined 256 bits per register.

3. However notice, my test is not loading YMM registers, it's loading XMM. What the microcode is doing is, per cycle 128 bits is being transferred to each of two separate registers, which in a straightforward hardware design would share the same input wires.

Therefore, Intel has designed the circuitry so that two XMM registers can be loaded in one cycle. It seems likely that either the L1-to-XMM/YMM path has been expanded to 256 bits or reading from L1 is possible at a rate of two per cycle (dual data rate). More like the latter case.

My Xeon has a 20 MB shared L3. Will it be fast?

This was proven to me recently when a person with a dual-CPU Xeon 2690 system (20 MB L3, 8 cores and 4 channels per CPU) ran bandwidth on 8 cores out of 16, resulting in each core effectively having only 5 MB of L3. If he had been running bandwidth on all cores, obviously each would effectively have only 2.5 MB of L3 to use.

If one were to use an AMD Opteron with 10 cores and 30 MB of shared L3, the worst case situation would be each core having effectively 3 MB of L3, which is the same as a lowly consumer-grade Core i5.

Thus with the Xeon and Opteron used with a memory-intensive application running on each core, perhaps one's priority should be on:

• Using the fastest possible RAM.
• Choosing a CPU with a large L2 cache (Opteron is 512 kB but Xeon is 256 kB).
• Organizing the data set as best as possible in shared memory.

Why is the 128-bit performance of the Turion so horribly bad?

Author