Comments

These tests provide a decently rigorous picture of the capabilities and differences among several popular linux bittorrent clients, running on one system under one linux distribution over one type of ADSL broadband provisioning. Note that the tests are all for a single torrent, chosen for its large and well seeded swarm to provide as much reproducibility as any bittorrent swarm can likely offer. Among the things these tests do not provide are:

(1) Scaling results for clients under multiple torrent loads;
(2) Performance differences between clients when each has been optimally configured beyond initial setup defaults;
(3) Performance differences between clients on more generously provisioned broadband;
(4) Features or performance of daemon, remote, web ui, or other variants on the standard client; and
(5) Effects (if any) of different linux distributions upon bittorrent client performance.

Special Note

[*] Memory use reports for rTorrent by this python script are not representative of actual memory use by rTorrent while downloading. Developers involved with multiple competing clients tested here noted this discrepancy. One stated, "rTorrent relies more upon the [kernel] page cache table than most other clients, and so memory usage reported in the table does not necessarily reflect how much free memory a system would need to run rTorrent with satisfactory results."

During tests, rTorrent's info screen confirms its own reported memory use differs significantly from that reported by the python script. That screen's "Memory usage:" field varies dynamically during download between ~22.0 MB and ~40.0 MB, cycling in 30-second intervals, most often from a "floor" of ~25.0 MB and increasing to ~36.0 MB before falling back to ~25.0 MB and repeating. Thus, a rough estimate of actual memory use in these tests is ~38 MB.

Notes

[1] Deluge 1.2.0 had an overdownloading bug frowned upon by some trackers; this issue is fixed in 1.2.1 and later versions. The key to this fix is libtorrent-rasterbar 0.14.9, more than changes to Deluge itself.

[2] Awaiting solution to a trivial 1.2.1 build problem on the test system, two interim runs used 1.2900-dev with libtorrent-rasterbar 0.14.8.

[3] Deluge recently switched from Subversion to Git; this build is from a fresh git pull of the master branch, incorporating libtorrent-rasterbar 0.15.0.

[4] KTorrent forks on load, preventing the time utility from recording accurate start/end/CPU-use. Memory use difference on this run may be due to this fork (the python ps_mem.py ktorrent report was captured, but the time report was not). Subsequent tests were run with time using the ktorrent --nofork option, which enables time functionality.

[5] Monsoon has a poor endgame strategy. Repeatedly but not always, it would download to about 99% completion and then hang indefinitely on the last few pieces. Monsoon appears ill-suited for normal use, and was tested here only for comparison with prior tests.

[6] This test run used rTorrent from the Fedora 12 repository due to a naming conflict between libtorrent-rasterbar and libtorrent-rakshasa, before I found a workaround.

[7] This is the same libtorrent-rakshasa version as in the Fedora 12 repository, but both rTorrent and libtorrent-rakshasa were built from project source for this and subsequent test runs.

[8] PEPCAK attack. Terminal was in the wrong directory to find python ps_mem.py, and by the time I realized it the torrent had finished downloading.

[9] uTorrent memory use only. Wine components reported separate memory use: explorer.exe, 3.8 MiB; services.exe, 1.2 MiB; winedevice.exe, 2.8 MiB; and wineserver, 2.1 MiB; total 9.9 MiB. Wine version: wine-1.1.38-1.fc12.i686.

[10] ClarkConnect have rebranded themselves ClearOS. I have found their products exemplary since I began using them in 2003 (they don't pay me).

[11] Normal system updates, including the Fedora 12 updates-testing repository, were enabled during the test runs, and kernels changed during that span. Client tests were significantly staggered within the testing time span, and none of the operating system changes appeared to have any observable effect upon any client performance.