Discussion:
PCI passthrough
Stephan von Krawczynski
2013-08-16 14:03:08 UTC
Permalink
Hello all,

since vbox has troubles with network latency we want to try pci passthrough of
a network card. The docs say we need to "VBoxManage modifyvm "VM name"
--pciattach 02:***@01:05.0", which means attaching the pci device in question
to the guest. But what about de-taching it from the host? The docs do not
mention this at all. Has anybody real experience with this? Do we need to
detach the device from the host or does vbox do that automagically in the
background when attaching it to a guest?
--
Regards,
Stephan
Fernando Cassia
2013-08-16 15:53:50 UTC
Permalink
On Fri, Aug 16, 2013 at 10:03 AM, Stephan von Krawczynski
Post by Stephan von Krawczynski
. But what about de-taching it from the host?
If the host is Windows, goign to Device Manager and "disabling" the device
would do the trick, I think. It essentially turns off the windows driver
for it.

FC
--
During times of Universal Deceit, telling the truth becomes a revolutionary
act
Durante épocas de Engaño Universal, decir la verdad se convierte en un Acto
Revolucionario
- George Orwell
Stephan von Krawczynski
2013-08-16 20:34:08 UTC
Permalink
On Fri, 16 Aug 2013 11:53:50 -0400
Post by Fernando Cassia
On Fri, Aug 16, 2013 at 10:03 AM, Stephan von Krawczynski
Post by Stephan von Krawczynski
. But what about de-taching it from the host?
If the host is Windows, goign to Device Manager and "disabling" the device
would do the trick, I think. It essentially turns off the windows driver
for it.
FC
Sorry Fernando,
since I am not linked to M* W*ndows in any manner I always forget to tell that
all things I write and ask are pure linux topics. In this case host as well as
guest are linux.
--
Regards,
Stephan
Fernando Cassia
2013-08-16 22:02:49 UTC
Permalink
On Fri, Aug 16, 2013 at 5:34 PM, Stephan von Krawczynski
Post by Stephan von Krawczynski
Sorry Fernando,
since I am not linked to M* W*ndows in any manner I always forget to tell that
all things I write and ask are pure linux topics. In this case host as well as
guest are linux.
This should help.
http://lwn.net/Articles/143397/

Let us know if you get it to work.
FC
--
During times of Universal Deceit, telling the truth becomes a revolutionary
act
Durante épocas de Engaño Universal, decir la verdad se convierte en un Acto
Revolucionario
- George Orwell
Stephan von Krawczynski
2013-08-18 12:08:22 UTC
Permalink
On Fri, 16 Aug 2013 19:02:49 -0300
Post by Fernando Cassia
On Fri, Aug 16, 2013 at 5:34 PM, Stephan von Krawczynski
Post by Stephan von Krawczynski
Sorry Fernando,
since I am not linked to M* W*ndows in any manner I always forget to tell that
all things I write and ask are pure linux topics. In this case host as well as
guest are linux.
This should help.
http://lwn.net/Articles/143397/
Let us know if you get it to work.
FC
Well, things are not really great:

# VBoxManage modifyvm "myVM" --pciattach 41:***@01:00.0
VBoxManage: error: Host PCI attachment only supported with ICH9 chipset
VBoxManage: error: Details: code NS_ERROR_INVALID_ARG (0x80070057), component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "AttachHostPCIDevice(iHostAddr, iGuestAddr, TRUE)" at line 2476 of file VBoxManageModifyVM.cpp

Does this really mean that you need a host with ICH9 chipset to use vbox pciattach at all?
That's really a no-go then, because my boxes have lots of xeon cores but no ICH9.
--
Regards,
Stephan
Matthew Hall
2013-08-18 16:09:51 UTC
Permalink
You got it backwards. I am 99% sure they mean the VM needs to be set into the non default ICH9 mode. Probably because the other older chipset doesn't have VT-D on it.
--
Sent from my mobile device.
Post by Stephan von Krawczynski
On Fri, 16 Aug 2013 19:02:49 -0300
Post by Fernando Cassia
On Fri, Aug 16, 2013 at 5:34 PM, Stephan von Krawczynski
Post by Stephan von Krawczynski
Sorry Fernando,
since I am not linked to M* W*ndows in any manner I always forget
to tell
Post by Fernando Cassia
Post by Stephan von Krawczynski
that
all things I write and ask are pure linux topics. In this case host
as
Post by Fernando Cassia
Post by Stephan von Krawczynski
well as
guest are linux.
This should help.
http://lwn.net/Articles/143397/
Let us know if you get it to work.
FC
VBoxManage: error: Host PCI attachment only supported with ICH9 chipset
VBoxManage: error: Details: code NS_ERROR_INVALID_ARG (0x80070057),
component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "AttachHostPCIDevice(iHostAddr, iGuestAddr,
TRUE)" at line 2476 of file VBoxManageModifyVM.cpp
Does this really mean that you need a host with ICH9 chipset to use vbox pciattach at all?
That's really a no-go then, because my boxes have lots of xeon cores but no ICH9.
Stephan von Krawczynski
2013-08-18 17:37:51 UTC
Permalink
On Sun, 18 Aug 2013 09:09:51 -0700
Post by Matthew Hall
You got it backwards. I am 99% sure they mean the VM needs to be set into the non default ICH9 mode. Probably because the other older chipset doesn't have VT-D on it.
--
Sent from my mobile device.
You are right, but unfortunately it did not really help either.
I switched the guest to ICH9, then I could pci-attach the two questionable
network cards. The guest booted and everything looked fine, but did not work.
The cards produced no interrupts at all. So I downed the guest and detached
the devices - only to find out that the original driver (igb) didn't want them
back... :-( (yes, I know how to bind via /sys)
I had to reboot the host to get the networks back and during the shutdown vbox
completely crashed and hung the box.
After coldboot everything is back to normal now - without pciattached networks
of course. I cannot recommend that to joe-average-user ...
--
Regards,
Stephan
Matthew Hall
2013-08-18 17:45:22 UTC
Permalink
Very strange. I only tried it on my machine a few months with a crypto accelerator board and it worked completely perfectly once I got the command line syntax figured out. The board worked fine and produced the expected level of acceleration of crypto performance in my extensive load testing.

Perhaps if you tried making sure to deactivate the interfaces, and unload the igb module, as a temporary test, to be 100% sure it got fully unbound from the host OS it might help isolate the cause of the issue. I had the advantage that my host OS did not have an integrated driver so I couldn't experience as many resource conflicts as your environment. Can you also try another peripheral in the passthrough to see if that change the result as well?

Matthew.
--
Sent from my mobile device.
Post by Stephan von Krawczynski
On Sun, 18 Aug 2013 09:09:51 -0700
Post by Matthew Hall
You got it backwards. I am 99% sure they mean the VM needs to be set
into the non default ICH9 mode. Probably because the other older
chipset doesn't have VT-D on it.
Post by Matthew Hall
--
Sent from my mobile device.
You are right, but unfortunately it did not really help either.
I switched the guest to ICH9, then I could pci-attach the two
questionable
network cards. The guest booted and everything looked fine, but did not work.
The cards produced no interrupts at all. So I downed the guest and detached
the devices - only to find out that the original driver (igb) didn't want them
back... :-( (yes, I know how to bind via /sys)
I had to reboot the host to get the networks back and during the shutdown vbox
completely crashed and hung the box.
After coldboot everything is back to normal now - without pciattached networks
of course. I cannot recommend that to joe-average-user ...
Fernando Cassia
2013-08-20 14:26:50 UTC
Permalink
Post by Matthew Hall
Very strange. I only tried it on my machine a few months with a crypto
accelerator board and it worked completely perfectly once I got the command
line syntax figured out
This thread reminds me. Is the same thing possible (PCI re-attach to guest
VM) possible with ISA cards?. I have an old video preprocessing card that
I'd like to use (it's just an analog-in analog-out video card with filter
settings on the card's chipset controlled from software. I don't think it
even needs or uses a IRQ, just an I/O address.

And unfortunately its manufacturer only supported up to Win2K, so no XP
drivers or beyond. Sounds like a good candidate to run in a VM...

*IF* ISA cards can be mapped to a VM...
FC
Fernando Cassia
2013-08-20 14:28:16 UTC
Permalink
Post by Fernando Cassia
And unfortunately its manufacturer only supported up to Win2K, so no XP
drivers or beyond. Sounds like a good candidate to run in a VM...
*IF* ISA cards can be mapped to a VM...
And before someone asks, YES, I do have a ISA slot in one of my systems. It
was one of the last motherboards produced with ISA on board, a 2.4 Ghz
Pentium 4 with 800 Mhz FSB.

FC
--
During times of Universal Deceit, telling the truth becomes a revolutionary
act
- George Orwell
Matthew Hall
2013-08-20 15:31:52 UTC
Permalink
Remapping peripherals requires IOMMU and VT-D support. I am 95% sure it won't work with any non PCI bus devices because they won't be routed through the necessary chips. Not to mention the VBox command to set it up requires specifically a PCI device ID which you won't have.

Matthew.
--
Sent from my mobile device.
Post by Fernando Cassia
Post by Fernando Cassia
And unfortunately its manufacturer only supported up to Win2K, so no
XP
Post by Fernando Cassia
drivers or beyond. Sounds like a good candidate to run in a VM...
*IF* ISA cards can be mapped to a VM...
And before someone asks, YES, I do have a ISA slot in one of my
systems. It
was one of the last motherboards produced with ISA on board, a 2.4 Ghz
Pentium 4 with 800 Mhz FSB.
FC
--
During times of Universal Deceit, telling the truth becomes a
revolutionary
act
- George Orwell
------------------------------------------------------------------------
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance
Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
------------------------------------------------------------------------
_______________________________________________
VBox-users-community mailing list
https://lists.sourceforge.net/lists/listinfo/vbox-users-community
_______________________________________________
Geoff Nordli
2013-08-16 19:24:11 UTC
Permalink
Post by Stephan von Krawczynski
Hello all,
since vbox has troubles with network latency we want to try pci passthrough of
a network card. The docs say we need to "VBoxManage modifyvm "VM name"
to the guest. But what about de-taching it from the host? The docs do not
mention this at all. Has anybody real experience with this? Do we need to
detach the device from the host or does vbox do that automagically in the
background when attaching it to a guest?
Hi Stephan.

I am really interested,, and I am sure I am not alone, in following what
you are doing with the networking performance side of things.

Please keep us posted on what you find.

thanks,

Geoff
Loading...