Today I’ve been trying to debug terrible network performance on the SPARCstation-10, it has been frustrating, but I have made some progress…
On the 10 Mbit/sec built-in LANCE (“le”) interface, I could get 1 MByte/sec throughput on large file FTP transfers to the desktop PC (itself equipped with 100Mbits/sec interface); but using the SBus 100 Mbit/sec HappyMeal (“hme”) on the SS10, throughput was only 24 KByte/sec max.
For a large collection of small files (<24 KByte each), throughput using HappyMeal almost disappeared entirely – average 3KByte/sec, ouch! Conversely, LANCE interface managed 80 KByte/sec average for the small-file collection, much more like it. Strangely, the small files FTP over HappyMeal would run very quickly for 10 seconds or so, then pause for 30 seconds, then burst for a bit more, then went-to-sleep again, and so on.
First I tried replacing the 10BaseT cable connecting the SS10 HappyMeal to the Buffalo AirStation hub/router, but no improvement even with commercial 100BaseTX-rated sealed-unit cables.
Next I tried adjusting the Inter-Packet Gap device-driver parameters (ipg0, ipg1, ipg2) to slow the HappyMeal interface down to see if that avoided the burst-sleep-burst-sleep behaviour. Nope.
Check the lights on, and reset the hub/router? No change.
Why was LANCE working properly, but HappyMeal so very broken?
Eventually, I remembered a strange firmware/hardware issue from the mid-90’s I had seen with a SPARCserver-1000E equipped with hme SBus cards: the auto-speed-negotiation function of the SBus “hme” interfaces did not always agree with the same facility on some 100Mbit/sec hubs and routers.
What would happen is that the HappyMeal would advertise full-duplex capability, but in a way that the hub could not understand. This often resulted in the HappyMeal using full-duplex mode but the hub running in half-duplex mode. This caused the SBus cards’ own transmissions to collide on the wire with it’s incoming packets, causing the HappyMeal ethernet segment to be in an almost constant state of jamming-signal-to-reset (see the Ethernet CSMA/CD specs), thus very few packets were getting through.
Ultimately, the “solution” was to limit the SS10 HappyMeal driver to half-duplex mode via the following lines in /etc/system:
- set hme:hme_adv_autoneg_cap = 1
- set hme:hme_adv_100T4_cap = 0
- set hme:hme_adv_100fdx_cap = 0
- set hme:hme_adv_100hdx_cap = 1
- set hme:hme_adv_10fdx_cap = 1
- set hme:hme_adv_10hdx_cap = 1
Strangely, though, this does not afflict the PCI-based on-board HappyMeal (PCEX) of the Sun Ultra-10 with Solaris 2.6, nor does it effect SBus-hme-direct-to-SBus-hme via a cross-over cable. Thus I must conclude that the SBus SunSwift cards’ hme interface does not quite adhere to the 100MBit/sec Ethernet autonegotiation standards. (Update 25th August: it appears that it may be a device-driver problem – the on-board PHY of the HappyMeal controller requires an idle time of 3 seconds before reading it’s config registers).
So I’ve now got rock-solid 5MByte/sec throughput without jittering. Not the 10MByte/sec I was hoping for, but at least it’s better than the 1MByte/sec that the onboard interface can do.
I wonder, maybe messing with the link-partner-… or other driver settings could get it upto full speed? Or perhaps there is an hme firmware update available?
UPDATE: 26th August 2012
I managed to get automatic full-duplex working by interposing a 4-port mini ethernet switch (with proper “duplex” and “late-collision” indicator lights!) between the SS10 and the AirNet switch; even with no custom driver config, the HappyMeal interface auto-negotiates both speed and duplex with the mini bridge properly. Even so, the SS10 is only able to sustain 6MByte/sec throughput. Oh well, better than what I started with…