Intel Thunderbolt 4 Update: Controllers and Tiger Lake in 2020
by Dr. Ian Cutress on July 8, 2020 9:00 AM EST- Posted in
- CPUs
- Tiger Lake
- USB4
- Thunderbolt 4
- TB4
Wired connectivity is converging onto two standards: USB4 and Thunderbolt 4. Both of these are set to debut by the end of the year in Intel’s upcoming Tiger Lake platform, and to set the scene Intel is updating us on the scope of its Thunderbolt 4 efforts.
Thunderbolt 4 is going to be a superset of TB3 and USB4, meaning that any Thunderbolt 4 Type-C host will be able to accept TB4, TB3, USB4, and USB 3/2/1 connections. Thunderbolt 4 will offer speeds up to 40 Gb/s, and there are a number of requirements for Thunderbolt 4 hosts and devices in order to be certified.
These requirements include:
- Video, support 2x 4K or 1x 8K (No detail at what refresh/bit-rate/chroma)
- Data, PCIe at 32 Gbps (storage up to 3 GB/s)
- Support for TB4 docks with four TB4 ports (one upstream, 3 downstream)
- PC Charging on at least one port (for laptops up to 100 W)
- Wake from sleep by peripherals connected to a TB4 dock
- Requires Direct Memory Access protection (more on this later)
The first devices to come to market with TB4 support will be Intel’s Tiger Lake platform, for use in laptops, which will have TB4 baked right into the silicon. TB4 will be one of the base requirements for Intel’s Project Athena certification program, and Intel is set to release a new TB4 2m cable for most use cases. Intel is also working on optical 5-50 meter cables, which will also now support the TB4 multi-port accessory architecture, enabled by 4-port TB4 hubs.
There will be two host controllers, known as Maple Ridge JHL8540 and JHL8340, for use as host controllers in desktops, workstations, and laptops. We are told the package size and power requirements are essentially in-line with previous Titan Ridge TB3 controllers. The device controller, for use in docks, monitors, storage devices and such, is the Goshen Ridge JHL8440. All three of these controllers will be available by the end of the year.
Thunderbolt 4 has no costs associated with using it, the branding, or the logo, however there is a branding/logo license. Intel has opened up the TB4 standard, but as it stands is the only company that has publically announced its intention to make host and device controllers. The cables will be branded with the Thunderbolt Logo and a number 4, however ports on systems will only have the Thunderbolt Logo, making TB3 and TB4 use indistinguishable (we disagree with Intel that users don’t check what their system uses so using the same logo makes no difference to these users – we think the same argument can be used in order to showcase the logo on devices with the number 4).
Here’s a main slide showcasing the difference between all the different standards. Ultimately TB4 is backwards compatible with TB3 and USB4/3/2/1 standards.
One of the key focal points in our briefings with Intel is that Thunderbolt 4 will have an additional requirement this time round – Direct Memory Access protection to prevent physical attacks. In our briefing (and shown on the slide below), Intel initially stated that this requires Intel VT-d technology, which raised questions about Thunderbolt 4 being limited to Intel only systems.
The spokesperson later clarified that in order to gain certification, DMA protection is required, and Intel is using VT-d to do it. Intel refused to comment on how other vendors might implement DMA protection, stating that it would be up to them. While additional security protections are always a good thing, they ideally need to be based around open vettable standards, something which might limit Thunderbolt for another generation as an Intel-only technology (whereas USB is far more ubiquitous).
We’ve reached out to AMD late in the day, and they’ll be supplying a comment soon. I will update this post when I get something.
Update, AMD says the following:
The “Zen 2” architecture supports DMA security in pre-boot and OS environments via AMD-Vi (IOMMU) on USB and PCIe interfaces.
When asked specifically if AMD-Vi meets the requirements for TB4, AMD stated:
If the question is to do we support DMA? The answer is yes. Any questions about if this would satisfy another companies requirements for an interface they are developing would need to be directed at them.
When we asked Intel to confirm if AMD-Vi met the requirements for TB4's DMA protection, we were given the following quote:
Thunderbolt is open to non-Intel-based systems. Like any other system, devices must pass Thunderbolt certification and end-to-end testing conducted by third-party labs. Thunderbolt 4 requirements include Intel VT-d based or an equivalent DMA protection technology that provides IO virtualization (often referred to as IO Memory Management Unit or IOMMU), as well as OS implementation support. If the equivalent technology supports prevention against physical attacks, then that should meet the requirement.
Overall, TB4 seems like a very capable standard, providing backwards compatibility, top class speeds, as well as new connectivity topologies. It will be interesting to see what the additional cost of adding TB4 onto various systems will be with Intel’s controllers, especially in the desktop space.
Facebook
Twitter
RSS
76 Comments
View All Comments
Techtree101 - Wednesday, July 8, 2020 - link
Total is 80 Gb/s for single direction connections like DisplayPort 2.0.repoman27 - Wednesday, July 8, 2020 - link
This isn't an arbitrary limitation. The discrete controllers only have a PCIe 3.0 x4 connection to their host or device. The signaling rate for PCIe Gen3 is 8 Gbit/s, which x 4 = 32. But then there's 128b/130b encoding which reduces the effective bandwidth to 31.508 Gbit/s. PCI Express is a packet based protocol, so in addition to payload data, there's also framing, headers, and CRC, plus packets used by the transaction and data link layers for management. On most systems, this protocol overhead brings the payload capacity down to ~26.6 Gbit/s or 3325 MB/s.The Thunderbolt 3 link is two bonded full-duplex channels operating at a signaling rate of 20.625 Gbit/s with 64b/66b encoding, which works out to a full 40 Gbit/s for data. However, the controller has to take the PCIe packets from the host / device, further encapsulate them and then transport them over the Thunderbolt link. The protocol overhead (which I believe may be further increased by the way channel bonding is implemented) results in peak Thunderbolt 3 PCIe payload throughput of ~2750 MB/s. However, that is probably due in part to limitations of the lane adapters rather than the Thunderbolt channel being completely stuffed.
Tiger Lake will be the first Intel platform to support PCIe Gen4, however it also has integrated Thunderbolt 4. The Maple Ridge controllers are probably PCIe Gen3 only for the sake of keeping power requirements in line. But who knows, maybe they'll suddenly support PCIe Gen4 once Rocket Lake is released.
jeremyshaw - Wednesday, July 8, 2020 - link
I do agree with the overhead of PCIe on the PCIe side of things, however, Intel disagrees with your assessment of TB3's own tunneling. Of course, in extremely un-Intel fashion, they have almost NO technical data on this publicly released, despite it being an optional part of the USB4 protocol and the basis of the DP2.0 protocol. Basically, the overhead of packaging a PCIe-only stream will not somehow reduce the B/W of TB3's 40Gbps down to ~22Gbps. Intel's own claim (shown by Anandtech on their 19FEB2019 review of an eGPU box) is 22Gbps max for PCIe, arbitrarily. This is why I'm not buying the overhead, since TB3 (in the same Intel claim in that Anandtech review) shows it can handle ~18Gbps PCIe + ~22Gbps video (specifically, their claim is 5k@60Hz with 10bit color - the data rate for that is calculated to ~27Gbps... I can only assume they really meant 8bit color, which is closer to 22.18Gbps).repoman27 - Wednesday, July 8, 2020 - link
The PCIe Gen3 x4 back end limits PCIe payload throughput to 26.6 Gbit/s. That's the max any PCIe Gen3 x4 device can muster (unless you bump up the max TLP payload size).The PCIe protocol / lane adapters in Intel's Thunderbolt 3 controllers can only manage 83% of that rate, but they're also dealing with round-trip latencies on the Thunderbolt link that are orders of magnitude higher than a typical PCIe link manager has to deal with and having to send multiple copies of certain types of control packets to accommodate Thunderbolt channel bonding. As I said, the limitation isn't because Thunderbolt bandwidth is saturated, but it's also not arbitrary. That's simply as fast as she'll go.The DisplayPort tunneling strips out all of the bit stuffing and most of the blanking from the regular DP stream and then encapsulates it, so you need to take that into account with your bandwidth calculations. The USB4 Specification is free to download from the USB-IF if you want to go plumbing the depths of how the protocol works.
jeremyshaw - Wednesday, July 8, 2020 - link
I already downloaded the USB4 spec on the day it was released, and read through it. Unless if it has massively changed since then (it hasn't), it talks a lot about implementations and some theory, but TB3 is largely handwaved as (paraphrasing) "follow TB3 protocol," etc. For example, the DP tunneling aspect of the TB3 compatibility mode (NOT the same as DP-Alt mode)? It has plenty of sections that devolve into "Follow DP 1.2 spec; follow DP1.4a spec." Where? Oh, right behind that paywall. Speaking of the DP spec, it doesn't state anywhere (nor is it inferred from reading any of the specs) that DP is stripped and reencoded/encapsulated/etc. Same is true of the TB3 implementation. Almost all of the data you see is about implementing the surface layer of TB3 compatibility (bringup/initialization). Everything else is still (paraphrasing) "in the actual data stream, this section is TB3 compatible XXXXXX." Hey, maybe I'm wrong and the USB-IF manual is massively different, nowadays. I recently downloaded it again and read the redline version, and I didn't see any meaningful or insightful improvements in the area of TB3.W.r.t. blanking, timing, etc for DP, HDMI, etc I'm already well aware of it. Intel's own calculations are still wrong. 22Gbps will not get you a tunneled DP with 5k, 60Hz, 10bits. It would barely work at 8bits. If one uses raw data calculations, it is ~26.5 Gbps for 5k, 60Hz, 10bits - this is easy for anyone to verify, instead of trusting Intel's claims.
As for the overhead, I'm still not seeing the explanation on how 26.6Gbps becomes 22Gbps out of a 40Gbps channel, especially when that 40Gbps channel is capable of saturating itself unidirectionally, with mixed PCIe and DP tunneling. Even if there is some sort of fundamental mis-timing and misalignment that just saps everything away (which again, would be easier to see if we had any sort of technical data on these, beyond tiny snippets of hints from Intel/Partners, and vague, dismissive references from other standards bodies), it would not explain away the ability to have a mixed PCIe and DP load reach 40Gbps.
Unless if anyone has any hard technical data on TB3's internal logic and functionality w.r.t. the PCIe bandwidth limitation, we're still at an impasse. The limit is still arbitrary (unless if we have differing definitions of arbitrary) and we are still in the dark as to why TB3 has a 40Gbps link with only ~22Gbps PCIe passthrough.
repoman27 - Wednesday, July 8, 2020 - link
Apologies. I agree that it is assumed that you are a VESA member and have access to the DisplayPort specs, and that everything pertaining to TBT3 compatibility in that document is vague af. I also agree that that slide from Intel marketing showing the bandwidth calculations for Thunderbolt 3 is clearly flawed, and that without lossy compression you can't make a 5120 x 2880 @ 60 Hz, 10 bpc stream less than 26.54208 Gbit/s. However, I do kinda feel like the protocols themselves are pretty clearly defined in their respective specifications.DisplayPort Tunneling and Alternate Mode are completely different. The former is tunneled over USB4 / TBT3, the latter is native signaling. USB4 DP tunneling is almost the same as TBT3 DP tunneling with a few differences.
From section 10.5.1 SST Tunneling:
"When a DisplayPort SST Main-Link is mapped onto USB4, the continuous Main-Link data stream
is encapsulated into Tunneled Packets. Before encapsulation, all the Stuffing Symbols (within a
TU and during the blanking periods) are discarded by the DP IN Adapter. Stuffing Symbols are
recreated by the DP OUT Adapter when the Main-Link data stream is extracted from the
Tunneled packets. In order to enable accurate reconstruction of the Stuffing Symbols at the DP
OUT Adapter, the DP IN Adapter includes a Fill Count field in each Tunneled Packet. The Fill
Count field specifies the number of Stuffing Symbols that were discarded immediately preceding
the packet."
And from section 13.8 DisplayPort Tunneling:
"TBT3-Compatible mode is Non-LTTPR mode as defined in Chapter 10 with the modifications defined in this section..."
But to bring this back to PCIe tunneling, regardless of the Thunderbolt link bandwidth, PCIe bandwidth between any single device and the host cannot exceed what the PCIe link between the controller and host / device is capable of. It's not that hard to calculate the theoretical maximum throughput of a PCIe Gen 3 x4 link and confirm those calculations via empirical data. It's also not hard to look at the real-world data for peak Thunderbolt 3 PCIe throughput and see that it tops out right around 2750 MB/s or 22 Gbit/s. Strangely enough, Intel marketing used that number in their slide and it proves out in real life. I see three options here:
1. Certain aspects of the Thunderbolt 3 protocol limit PCIe tunneling throughput to less than what the back-end of the controller is capable of.
2. Intel's implementation of PCIe tunneling in their discrete Thunderbolt 3 controllers is not capable of sustaining PCIe tunneling rates greater than 22 Gbit/s and Intel is not keen on discussing why.
3. Intel is arbitrarily limiting PCIe tunneling throughput from ~26 to 22 Gbit/s.
While we may still be in the dark, I certainly feel that option 3 is by far the least likely.
Eliadbu - Wednesday, July 8, 2020 - link
After digging a bit I found an early thunderbolt 3 brief that explains bit about priorities and data rate. While the connector is definitely capable of 40 Gbps, there is an illustration that shows that the maximum data rate as If you transfer just data is 22 Gbps each direction and other bandwidth is not used. Makes scratch my had and wonder why the specification limited for data transfer while if you pass video it can do much more. I guess there are maybe some limitations deep down that limit the data rate from pcie.Deicidium369 - Thursday, January 14, 2021 - link
TB4 is integrated on the Tiger Lake CPU - not connected as an external controller. Intel NUC11 Pro has TB4 and PCIe4 for the M.2 SSD.https://ark.intel.com/content/www/us/en/ark/produc...
Silma - Wednesday, July 8, 2020 - link
Wasn't USB 4 supposed to be fully compatible with Thunderbolt 3 ?Valantar - Wednesday, July 8, 2020 - link
It is. One of the slides in this article even states so.