Hardware IOMMU and xhci controllers

You'd think that after the long time that has passed since the first USB3 controllers appeared on the computer market, Linux would have no big issues with USB3. Well, that's as far from reality as one might imagine.
For the last couple of months I had an annoying problem with my USB3 PCIe controller card. One out of four ports were dead and the other ones occasionally refused to work as well. Luckily at least one port always worked and so I just plugged my USB3-hub into that port and continued doing stuff. Fast forward to this week. I replaced my GPU with a more recent model and in order to fit the new card into the PC-tower, I had to shuffle around other PCIe cards. This also required the USB3-controller card to be removed. After I finally got the GPU into a fitting slot, I started to put the other PCIe cards back into the tower when I found a dark spot on my USB3 controller card:
broken_USB3_card.jpg
To my surprise some electronic part on that card had burnt and left a spot the size of a human thumb nail being entirely black. Now I had my explanation why one of the USB3-ports was always dead. Of course I didn't want to put a semi-broken card back into my PC so I tried to replace it with a spare part. And that's where the real fun began.

Booting Linux with that spare USB3 PCIe-card immediately resulted into the IOMMU shutting down the new USB3-card. What I found out the hard way with three(!) different USB3-controller cards was that no matter what kind of chip is being used on the cards, there's always some issue.
Apparently, cards with a "Renesas" chip don't have IOMMU issues but if you happen to use a card with too many of these chips being soldered onto the card, Linux just shuts the entire controller card down because Renesas chips seem to be the worst possible USB3 controller chips availabe with just way too many quirks.
On the other hand, cards with either "VIA" or "ASMedia" chips get in trouble when a (buggy?) hardware IOMMU is in use. My new card with two ASMedia chips booted nicely when no USB device was connected to it:

xhci_hcd 0000:09:00.0: xHCI Host Controller
xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 8
xhci_hcd 0000:09:00.0: hcc params 0x0200ef81 hci version 0x110 quirks
0x0000000000000010 usb usb8: New USB device found, idVendor=1d6b,
idProduct=0002, bcdDevice= 5.04 usb usb8: New USB device strings:
Mfr=3, Product=2, SerialNumber=1 usb usb8: Product: xHCI Host Controller
usb usb8: Manufacturer: Linux 5.4.51 xhci-hcd
usb usb8: SerialNumber: 0000:09:00.0
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
xhci_hcd 0000:09:00.0: xHCI Host Controller
xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 9
xhci_hcd 0000:09:00.0: Host supports USB 3.1 Enhanced SuperSpeed
usb usb9: We don't know the algorithms for LPM for this host, disabling
LPM. usb usb9: New USB device found, idVendor=1d6b, idProduct=0003,
bcdDevice= 5.04 usb usb9: New USB device strings: Mfr=3, Product=2,
SerialNumber=1 usb usb9: Product: xHCI Host Controller
usb usb9: Manufacturer: Linux 5.4.51 xhci-hcd
usb usb9: SerialNumber: 0000:09:00.0
hub 9-0:1.0: USB hub found
hub 9-0:1.0: 2 ports detected
xhci_hcd 0000:08:00.0: xHCI Host Controller
xhci_hcd 0000:08:00.0: new USB bus registered, assigned bus number 10
xhci_hcd 0000:08:00.0: hcc params 0x0200ef81 hci version 0x110 quirks 0x0000000000000010
usb usb10: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04 usb usb10: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb10: Product: xHCI Host Controller
usb usb10: Manufacturer: Linux 5.4.51 xhci-hcd
usb usb10: SerialNumber: 0000:08:00.0
hub 10-0:1.0: USB hub found
hub 10-0:1.0: 2 ports detected
xhci_hcd 0000:08:00.0: xHCI Host Controller
xhci_hcd 0000:08:00.0: new USB bus registered, assigned bus number 11
xhci_hcd 0000:08:00.0: Host supports USB 3.1 Enhanced SuperSpeed
usb usb11: We don't know the algorithms for LPM for this host,
disabling LPM. usb usb11: New USB device found, idVendor=1d6b,
idProduct=0003, bcdDevice= 5.04 usb usb11: New USB device strings:
Mfr=3, Product=2, SerialNumber=1 usb usb11: Product: xHCI Host
Controller usb usb11: Manufacturer: Linux 5.4.51 xhci-hcd
usb usb11: SerialNumber: 0000:08:00.0
hub 11-0:1.0: USB hub found
hub 11-0:1.0: 2 ports detected


But as soon as I connected some device to that card:

usb 10-1: new high-speed USB device number 2 using xhci_hcd
xhci_hcd 0000:08:00.0: Abort failed to stop command ring: -110
xhci_hcd 0000:08:00.0: xHCI host controller not responding, assume dead
xhci_hcd 0000:08:00.0: HC died; cleaning up
xhci_hcd 0000:08:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0012 address=0x100000000 flags=0x0000]
xhci_hcd 0000:08:00.0: Timeout while waiting for setup device command
usb 10-1: hub failed to enable device, error -62
usb usb10-port1: couldn't allocate usb_device
usb usb11-port1: couldn't allocate usb_device


Yeah! Well done IOMMU! Searching for "IOMMU xhci" results in lots of posts from Linux users who suffer the same problem. Some xhci kernel driver hacker told me that this is an IOMMU issue and usually requires to update the mainboard's BIOS. But my mainboard (Supermicro H8DG6-F) is EOL for quite a while already and the manufacturer told me that there will be no more BIOS updates for this board. So the only other solution was to disable usage of the hardware IOMMU in the Linux kernel. I added the following to my kernel command-line: "amd-iommu=off iommu=soft" and now I can finally use my new USB3 PCIe controller card.

Trackbacks

Trackback specific URI for this entry

This link is not meant to be clicked. It contains the trackback URI for this entry. You can use this URI to send ping- & trackbacks from your own blog to this entry. To copy the link, right click and select "Copy Shortcut" in Internet Explorer or "Copy Link Location" in Mozilla.

No Trackbacks

Comments

Display comments as Linear | Threaded

No comments

The author does not allow comments to this entry