Thunderbolt, USB 3.2 and even more 10Gbit fun…

So where we left off last time, I had a 10Gbit Thunderbolt 3 adaptor, and a mid-2014 Macbook Pro that I couldn’t plug it into.

I found out that you can in fact use the Thunderbolt 3 to 2 adaptor in a “backwards” configuration. I purchased the 2m TB2 cable - quite expensive at $65 for the cable. This is because although Thunderbolt 3 allows for short passive cables, all TB2 cables are active. Surprisingly, purchasing brand new from Apple was actually the cheapest option over anything on Amazon or eBay.

I bought the TB3 to 2 adaptor off Amazon for $9. That’s suspiciously cheap, and the one I got was labelled for sale in Brazil. Given Amazon’s approach to stock fungibility, it probably wasn’t even the one that the seller put into the system.

There’s just one catch - it doesn’t provide bus power from the TB2 side to the TB3 side. I looked for a third party adaptor that did, but Startech was the only 3rd party vendor to ever make one and that one doesn’t even support reverse mode.

Just to make sure I was right about this limitation, I tried it anyway. As expected, it didn’t work.

The other way round - a TB3 Mac using a bus powered TB2 Gigabit adaptor works fine.

So what we need is something to inject bus power - preferably something quite cheap. I looked for a suitable cable, but finding nothing (I’m really surprised I can’t find something), I heard others had success with docking stations.

Searching eBay finds that the StarTech TB3DKDPMAW was available for as low as $60 (without power brick). It only takes a 12v DC barrel jack, so this seems pretty cheap and easy.

Why so cheap and unwanted? It only does 15w USB PD back to the host, meaning that unlike other popular docks on the market, you can’t charge your laptop with it.

This isn’t a problem here, because we can’t charge through Thunderbolt 2 anyway, so you’d expect that at this point, I’d have bought this obscure bit of largely unwanted hardware that might do exactly what I want. But I didn’t.

By the time I get it shipped to me, and dig up a suitable power brick, it’s approaching half the cost of a CalDigit TS3+, a very popular Thunderbolt 3 laptop dock, that has tons of ports and the capability to charge the connected laptop at near 90 watts.

When I inevitably buy a new MacBook Pro, I can use the dock with that, rather than relegating the old dock to the bottom of a “random cables” drawer - making this a much better investment.

So I borrowed a CalDigit TS3+ dock from someone who already had one, while I wait for this one to arrive.

Read on, and we’ll see what works.

The Thunderbolt 2 cable was plugged into the MacBook Pro, and connected to the 3-2 adaptor plugged into the upstream Thunderbolt 3 port. The OWC 10GBE adaptor is connected to the downstream Thunderbolt 3 port. Just to see what would happen, I connected my portable SSD to the USB 3.2 10Gbit/s port on the back. Would this work even when the dock isn’t connected to the host by USB?

The ethernet card showed up and worked - however, pretty soon the link dropped and the CRS317’s fans went to full bore. The link came up, and dropped again. Plugging the ethernet adaptor into a M1 MacBook revealed the same behaviour, unlike earlier success. Removing and reinserting the S+RJ10 SFP+ module into the CRS317 seemed to fix the problem, and after that things worked, until the problem later re-occurred.

I have a suspicion that the S+RJ10 is defective - it was clearly used when I got it (despite being advertised as new), and I wouldn’t put it past Amazon to have re-shipped a RMA return. I’ve requested a return for “defective or not as described” - and they want it back in “original packaging” - sadly I’ve thrown out the anti-static bag it came in, from a Seagate hard drive, which had been crudely ripped in half, folded over, and sealed with sticky tape when I got it.

So I ordered one from FibreStore (aka, selecting the 80m variant. Why would I do this when it’s only on a 10m cable? It advertises lower power usage, figuring it’d run cooler (it’s widely reported these modules get quite hot) and it may be more useful if I need it for a longer cable run in future.

This one’s a bit more tempramental though. It won’t do gigabit unless you manually set the switch port end to gigabit. It appears to have no support at all for 2.5 and 5Gbit NBaseT modes - only doing 10Gbit. If you manually set it to 2.5 or 5, the link is detected but no packets pass through.

So I ended up ordering a third one - from Shenzhen Wiitek. It seems to be widely recommended and apparently has full support for NBaseT. We’ll see how that one goes, and I’ll probably hang onto the FibreStore one for a dedicated 10Gbit service, keeping the other one as a spare for use with the Plugable USB3 2.5Gbit adaptor.

So now I’ve got one SFP that does 2.5 just fine but doesn’t like 10Gbit, and another that does 10 but not do 2.5.

Anyway, with the Fibrestore SFP+ in place, how does it perform?

Using iperf in both directions at once shows we can get about 9Gbit in, and between 4-6Gbit out. This is likely due to the 2014 MBP being slower than the M1 - which in this setup can max out the interface both ways.

The real world test, is of course, how fast we can copy from the fileserver, and for that it doesn’t disappoint - nearly double the speed of the USB 3 adaptor, with lower CPU load. This is a similar speed to what I get on my desktop PC, so the bottleneck here is most likely the ZFS pool.

To answer the previous question - what about the USB ports on the dock? The SSD did show up, and does work. The reason why is that the USB ports on the CalDigit dock are not connected to a USB hub. It has PCIe USB controller chips in it, attached to the Thunderbolt bus - an ASM1142 USB 3.2 controller for the rear USB C port and the USB A port next to it, and two FL1100 chips for the rest of the USB 3 ports.

The SSD is a USB C NVME enclosure with an Intel 665p NVMe SSD installed. The enclosure supports USB 3 at 10Gbit - however System Profiler reports it’s only running at 5Gbit. The M1 MacBook sees it as 10Gbit, so the enclosure definitely supports 10. Let’s run some speed tests and see what we get.

There’s a big difference here - from the rear 10Gbit USB C port we’re seeing 731.1MB/sec, which is 5848.8Gbit/sec. This clearly shows it must actually be running in 10Gbit mode otherwise this would be impossible. Similar to the USB 3 ethernet adaptor which showed 1Gbit while actually being connected at 2.5, I guess.

Connected to the front USB C port labelled as 5Gbit/sec, we get 348.1MB/sec, about half the speed we saw before. This is a little disappointing, though, given that the inbuilt ports on the MacBook Pro with their 2014 era USB 3 controller can manage 413.3MB/sec, but still quite quick.

Oh, and the SD reader works - it’s on the USB 3.1 controller. But with a 2014 MBP, you have one built in, unlike the current models.

I haven’t tested the sound, ethernet or monitor outputs on the dock - I’ll give those a test in a future post.