SR-IOV: mainlining?

[jgcodes2020]

Since drivers for SR-IOV have already been completed in this fork, could these drivers be mainlined into the kernel?

[daiaji]

It's interesting. Has anyone tested whether the vgpu driver is available on windows?

[krispan-intel]

1. SRIOV upstream task is on-going, ETA Q4'23
2. Yes, we already test Windows VM and works well.

[daiaji]

Is the test environment using alder Lake's iGPU or arc dGPU? Is it a test using a normal desktop PC instead of a server? How much does vGPU performance lose in VM environment?

[strongtz]

1. SRIOV upstream task is on-going, ETA Q4'23
2. Yes, we already test Windows VM and works well.

Hi, I managed to create a DKMS module using the driver source here. But the GPU driver in my WIndows VM fails with Error 43. I wonder if you're using a modded driver?

My DKMS module: https://github.com/strongtz/i915-sriov-dkms
more information: https://www.reddit.com/r/VFIO/comments/xoeika/sriov_on_intel_uhd_graphics_770_xe_12th_gen

[ChristophSchmidpeter]

Is the test environment using alder Lake's iGPU or arc dGPU? Is it a test using a normal desktop PC instead of a server? How much does vGPU performance lose in VM environment?

Is there any update on whether SR-IOV will be "only" possible on iGPUs or also on Arc dGPUs too?
Is it correct that this feature will be available both on desktop machines and notebooks?

[alx696]

@krispan-intel Does Arc A770 support SR-IOV?

I noticed that the Flex 170 might support SR-IOV , but no card to verify. It would be exciting if customer graphics cards(Arc A770, A750) could also support SR-IOV. Many enthusiasts will buy Intel graphics cards because of this.

2023.04.18 Update :

I tested Arc 770 under Ubuntu 22.04 Server. It is support GPU Passthrough, but need install Intel Indirect Display Driver to support stream work(ex Sunshine). Not support SR-IOV!

# lspci -nn | grep "VGA"
00:02.0 VGA compatible controller [0300]: Intel Corporation AlderLake-S GT1 [8086:4680] (rev 0c)
03:00.0 VGA compatible controller [0300]: Intel Corporation Device [8086:56a0] (rev 08)

# lspci -vs 03:00.0
03:00.0 VGA compatible controller: Intel Corporation Device 56a0 (rev 08) (prog-if 00 [VGA controller])
	Subsystem: Intel Corporation Device 1020
	Flags: bus master, fast devsel, latency 0, IRQ 154, IOMMU group 16
	Memory at 81000000 (64-bit, non-prefetchable) [size=16M]
	Memory at 6000000000 (64-bit, prefetchable) [size=16G]
	Expansion ROM at 82000000 [disabled] [size=2M]
	Capabilities: [40] Vendor Specific Information: Len=0c <?>
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
	Capabilities: [d0] Power Management version 3
	Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [420] Physical Resizable BAR
	Capabilities: [400] Latency Tolerance Reporting
	Kernel driver in use: i915
	Kernel modules: i915

without Capabilities: [***] Single Root I/O Virtualization (SR-IOV)

---

2023.03.22 Update :

I tested Data Center GPU Flex 140 under Ubuntu 22.04 Server. Currently only available in passthrough mode. A single card has two addresses, so one flex 140 card support two vm. Relatively stable, no crashes during testing.

# lspci -nn | grep "Display"
d0:00.0 Display controller [0380]: Intel Corporation Device [8086:56c1] (rev 05)
d4:00.0 Display controller [0380]: Intel Corporation Device [8086:56c1] (rev 05)

SR-IOV driver not ready now.

[basncy]

Works on win10 guest, with some code modification personally. while it is better to wait for official stable release.

[ammgws]

with some code modification personally

What does this mean? Your XML seems different, do you mean you edited the XML?

Mine (as generated by virt-manager) - getting Code43 in the guest:

Yours:

[basncy]

Unlike PCI passthrough, which bound the entire hardware resource to one specific VM, this fancy function enables you to share one GPU with multiple VMs, then you can see the GPU address shows 00:02.x, which is called VF(virtual function), rather than 00:02.0, the physical hardware on host. And it requires support from both the motherboard chip(i915 in my case) and GPU. Your SRIOV function is not powered up, so neither your hardware nor software driver does not compatible.

[truvatech]

@strongtz , question for you, does that modified driver only work on 6.0.2, as mentioned from your latest commit message, or would that work on 6.0.9?

Been trying to get it working for days but no luck.

Basically get to the point where you echo to the sys device, but no matter what it returns permission denied
I'm on Fedora though, if that's the issue.

Otherwise dkms status shows the sriov-dkms driver from your repo, and I know it is loaded as it throws me an error in dmesg, which isn't there with the in-tree i915 module.

[ 66.468702] snd_hda_codec_hdmi hdaudioC0D2: No i915 binding for Intel HDMI/DP codec [ 66.468883] hdaudio hdaudioC0D2: Unable to configure, disabling

Thanks in advance!

[strongtz]

@truvatech 6.0.9 works well on Arch Linux here, both in host and VM. VA-API video acceleration in VM works.
If you're using my modified dkms driver, you will see debug prints in dmesg like
i915 0000:0b:00.0: i915_virtualization_probe: entry

[darkbasic]

Did anybody manage to verify if Arc supports any kind of SR-IOV?

[jsommr]

Did anybody manage to verify if Arc supports any kind of SR-IOV?

SR-IOV is not supported according to https://www.intel.com/content/www/us/en/support/articles/000093216/graphics.html

:(

[darkbasic]

Too bad, it looks like nobody will buy Intel Arc after all :(

[daiaji]

Too bad, it looks like nobody will buy Intel Arc after all :(

Indeed, there is no advantage in performance compared to competing products. The SR-IOV support in the promotion can be said to be the only bright spot. Now even this advantage does not exist. I really can’t think of anyone who would buy such a meaningless product.

[darkbasic]

I really can’t think of anyone who would buy such a meaningless product.

There is one very small use case left: av1 decoding/encoding on a budget. The lowest end Arc GPUs could make sense if all you need is a decoding/encoding accelerator, at least until RDNA3 budget cards get released.

[daiaji]

There is one very small use case left: av1 decoding/encoding on a budget. The lowest end Arc GPUs could make sense if all you need is a decoding/encoding accelerator, at least until RDNA3 budget cards get released.

This is usually one of the few selling points in their commercial propaganda. The disadvantages are obvious, poor drivers, poor performance, and poor prices. It can be said that there are almost no advantages, but if SR-IOV is supported, then these disadvantages can be tolerated.

Even limiting the number of VFs in SR-IOV to one is much better than the current state.

The Arc dGPU is a better description of the fact of wasting sand than the i7-11700K.

---

I learned that the transistor size is the same for all Arc dGPU multimedia codecs, which makes Arc dGPU only valuable as the lowest-end A370.

[myownfriend]

SR-IOV is not supported according to https://www.intel.com/content/www/us/en/support/articles/000093216/graphics.html

That's a huge bummer. I was just about to get one in hopes that it would support SR-IOV in the future.

[susanthenerd]

That's quite sad.

[aviallon]

SR-IOV is not supported according to https://www.intel.com/content/www/us/en/support/articles/000093216/graphics.html

That's a huge bummer. I was just about to get one in hopes that it would support SR-IOV in the future.

Could the drivers be patched to enable unofficial support? Or does this really require firmware support?

[ChristophSchmidpeter]

SR-IOV is not supported according to https://www.intel.com/content/www/us/en/support/articles/000093216/graphics.html

That's a huge bummer. I was just about to get one in hopes that it would support SR-IOV in the future.

Could the drivers be patched to enable unofficial support? Or does this really require firmware support?

I would assume that it was missing there on a physical level altogether or was fused off.

[starquake]

Could it be that it would only work for integrated graphics because of connections?

[darkbasic]

I'd say just lack of willingness to sell any card.

[ChristophSchmidpeter]

I'd say just lack of willingness to sell any card.

Nailed it.

[webczat]

i mean, there was a comment above saying it's postponed due to drm interface changes. However no one said for how long. and until this year I hoped it will not be that long :)

[plantroon]

I read somewhere that the kernel 6.8 will have this, but seems like that's not happening?

If this gets merged (or if it already is?) there seems to be something regarding VF ... https://gitlab.freedesktop.org/drm/xe/kernel/-/commits/drm-xe-next

[ownaginatious]

Tangentially related: anyone know if there are there any interim software based solutions for partitioning 13th/14th gen Intel iGPUs until this lands? I just want to be able to pass the hardware encoding capabilities to more than one VM 😅

[plantroon]

Tangentially related: anyone know if there are there any interim software based solutions for partitioning 13th/14th gen Intel iGPUs until this lands? I just want to be able to pass the hardware encoding capabilities to more than one VM 😅

Kernel 6.1 or whatever older that is known/confirmed working in this thread or similar...

I am testing the Intel Xe driver from kernel 6.8 RC5, even got mesa-git and all that, and it can't even show a desktop without drawing GPU rainbow art all over it. But I bet I am doing something wrong as Phoronix already ran tests on it. But nothing was announced, it is experimental, so I had no expectations.

Still, until the Xe driver matures, they should've at least got i915 working reliably (ie no suspend issues) in mainline. I wish I could sell my laptop at this point and buy something older that at least has gvt-g working. They removed a feature for which they don't have a replacement 2 years later. Basically these GPUs have sriov support, but it was never made properly functional in any software. Hyper-V has it partially working, but it's apparent it was more focused on Windows Sandbox and WSL-g.

[johntdavis84]

Any updates on SR-IOV and kernel 6.8? It's getting rolled into Proxmox 8.2, and I'd really like to be able to use SR-IOV with my Alder Lake i5-12500T iGPU without having to do the dkms-based wizardry here (that has to be done every time Proxmox updates the kernel at all...which is a lot).

See: https://www.derekseaman.com/2023/11/proxmox-ve-8-1-windows-11-vgpu-vt-d-passthrough-with-intel-alder-lake.html

[Daniel15]

If the same stands true for the Intel Windows drivers you might not have an use case case for a mainline SR-IOV kernel.

@darkbasic SR-IOV seems to work fine with Intel's Windows drivers. I have an Unraid server on Linux 6.1.x, running a Windows Server 2022 VM for Blue Iris, and have no trouble using hardware accelerated video encoding/decoding in the VM. I haven't tried anything graphically intensive, but I can definitely see the iGPU being used for video encoding/decoding.

[adapt-L]

https://www.phoronix.com/news/Linux-6.11-DRM-Intel-Xe-Next

[johntdavis84]

https://www.phoronix.com/news/Linux-6.11-DRM-Intel-Xe-Next

Thanks. :)

I see these updates, and they're always "preparing for SR-IOV" and talking about how once released the Xe driver won't be the default. Given that these news releases focus on Xe2-based GPUs and other newer hardware, I'm not quite sure how to predict what we'll actually get.

If Xe is not the default, but complete, should we just be able to switch to it somehow (instead of using i915) and have SR-IOV just start working so long as its enabled in the BIOS and everything else is configured correctly?

I'm trying to get a better understanding of when the current DKMS approach won't be needed anymore for 12th+ gen iGPUs.

[jgcodes2020]

It's been over 2 years since I first opened this issue, can someone from Intel give us an update as to which drivers (Xe or i915) will support SR-IOV and when?

[myownfriend]

I really wish they'd surprise us and have this enabled on the A770 but I'm not counting on it.

[Daniel15]

It's been over 2 years since I first opened this issue, can someone from Intel give us an update as to which drivers (Xe or i915) will support SR-IOV and when?

You might have seen it already, but the DKMS module for i915 SR-IOV support was recently updated to support kernel versions up to 6.9: https://github.com/strongtz/i915-sriov-dkms. It previously only supported up to 6.5.

[johntdavis84]

It's been over 2 years since I first opened this issue, can someone from Intel give us an update as to which drivers (Xe or i915) will support SR-IOV and when?

You might have seen it already, but the DKMS module for i915 SR-IOV support was recently updated to support kernel versions up to 6.9: https://github.com/strongtz/i915-sriov-dkms. It previously only supported up to 6.5.

Yes. I'm hoping. to test this soon. :)

Meanwhile: https://www.phoronix.com/news/Intel-Xe-DRM-Next-Linux-6.12
Intel is targeting Kernel 6.12 for "Intel Xe2 Lunar Lake & Battlemage Graphics," and also "further support for SR-IOV."

[jonas5]

Hello!

The latest kernel tree from intel seems to work with SR-IOV on A770. It is based on the 6.11.0-rc2 kernel version, you might want to compile this manually.

You can find this kernel tree here:
https://gitlab.freedesktop.org/drm/xe/kernel

This is what i used for my configs:

/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on kvm.ignore_msrs=1 vfio_pci.ids=8086:56a0,8086:4f90 pcie_aspm=off"

/etc/modprobe.d/blacklist.conf

blacklist i915
blacklist xe
blacklist snd_hda_intel

[adapt-L]

@jonas5 nice, but you blacklist xe? Did you compile xe as built-in instead of a module?

[johntdavis84]

This is great news! Hopefully this will eventually come to the 12th and 13th gen iGPUs, as well. I would love to be able to use SR-IOV on Proxmox without a DKMS kernel driver. From this and some other things I’ve read, it seems like they’re shooting for 6.12 to have this stable and working, if not default.

…

On Aug 27, 2024, at 10:46 AM, jonas5 ***@***.***> wrote: Hello! The latest kernel tree from intel seems to work with SR-IOV on A770. It is based on the 6.11.0-rc2 kernel version, you might want to compile this manually. You can find this kernel tree here: https://gitlab.freedesktop.org/drm/xe/kernel Screenshot.from.2024-08-27.16-40-11.png (view on web) <https://github.com/user-attachments/assets/79a6d85a-0692-400a-adbd-5d68ca011c4d> This is what i used for my configs: /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on kvm.ignore_msrs=1 vfio_pci.ids=8086:56a0,8086:4f90 pcie_aspm=off" /etc/modprobe.d/blacklist.conf blacklist i915 blacklist xe blacklist snd_hda_intel — Reply to this email directly, view it on GitHub <#33 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGI5CYQ7XQ6XQLZHFDI6WK3ZTSNPFAVCNFSM5Z63XG72U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMZRGI4TENBTGQ4Q>. You are receiving this because you commented.

[darkbasic]

The latest kernel tree from intel seems to work with SR-IOV on A770

That would be huge because Intel stated multiple times that A770 doesn't support SR-IOV. Can someone else please confirm this?

[adapt-L]

The latest kernel tree from intel seems to work with SR-IOV on A770

That would be huge because Intel stated multiple times that A770 doesn't support SR-IOV. Can someone else please confirm this?

https://www.phoronix.com/news/Intel-Xe-Driver-SR-IOV-Plans

https://lore.kernel.org/dri-devel/20231110182231.1730-1-michal.wajdeczko@intel.com/

SR-IOV Platforms
================

Initially we plan to add SR-IOV functionality to the following SDV platforms
already supported by the Xe driver:

 - TGL (up to 7 VFs)
 - ADL (up to 7 VFs)
 - MTL (up to 7 VFs)
 - ATSM (up to 31 VFs)
 - PVC (up to 63 VFs)

Newer platforms will be supported later, but we hope that enabling will be
much faster, as majority of the driver changes are either platform agnostic
or are similar between earlier platforms (hence we start with SDVs).

Looks like Tigerlake, Alderlake, Meteorlake, Arctic Sound M, Ponte Vecchio respectively

[rhysperry111]

It'd be nice to have some sort of official statement about where everything's at with this.

I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible

[PerAstraAdDeum]

It'd be nice to have some sort of official statement about where everything's at with this.

I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible

I'd love to hear that too. Has anyone made it work? If so, how?

[Daniel15]

It'd be nice to have some sort of official statement about where everything's at with this.

I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible

As far as I know, they're targeting Linux 6.12. Someone earlier in the comments got it working with an A770 using 6.11.0-rc2.

Having said that, Raptor Lake is not in the initial support list that @adapt-L posted above.

[PerAstraAdDeum]

It'd be nice to have some sort of official statement about where everything's at with this.
I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible

As far as I know, they're targeting Linux 6.12. Someone earlier in the comments got it working with an A770 using 6.11.0-rc2.

Having said that, Raptor Lake is not in the initial support list that @adapt-L posted above.

Sorry, but doesn't these two statements conflict with each other? From what I've read, Intel Arc Alchemist dGPUs aren't on the list that @adapt-L posted either, or am I misreading that? And I'd assume that Raptor-Lake is closer to ADL (which is on the list) then Arc Alchemist dGPUs?

Other than that, I hope SR-IOV is finally mainlined with 6.12!

[rayzorben]

It'd be nice to have some sort of official statement about where everything's at with this.
I'm trying to get SRIOV working with the iGPU on my Raptor Lake laptop, am on kernel 6.10 and have manually switched over to the xe driver and yet I can't seem to find a way to get it to work. It'd at least be nice to know if (or when) it'll be possible

I'd love to hear that too. Has anyone made it work? If so, how?

I would like to know the answer to this question myself. I don't care 'if' it is supported but 'when' it is supported, how is this intended to be enabled? I am running the xe driver, I have enabled iommu I have set max_vfs on the kernel command line and I have tried setting sriov_numvfs to 7 in sysfs.conf as well as trying to write it to the file directly, but it is not working. Which is fine if it is not intending to work yet, but I want to know what the correct way of doing this is.

I am on 6.11.0-rc6-1

[adapt-L]

I think "Newer platforms will be supported later" suggests it may be possible, but unlikely considering that support is not listed on intel's site: https://www.intel.com/content/www/us/en/support/articles/000093216/graphics/processor-graphics.html

But Tigerlake is also listed as unsupported there and listed as planned to be supported in the patch note I just linked, so who knows.

But if you want a dedicated card today that supports SRIOV I think there are some old AMD workstation cards that do that.

I'm still waiting for it to be mainlined cause I don't really feel like adding experimental patches to my kernel and its been like 2 years already

[Daniel15]

cause I don't really feel like adding experimental patches to my kernel

@adapt-L As far as I know, the Intel LTS kernel (the one this issue is in) supports SR-IOV with the i915 driver. If you don't want to patch your kernel or use the DKMS module that was extracted from this kernel (https://github.com/strongtz/i915-sriov-dkms), you could just run this kernel directly.