[SOLVED] Desktop freeze while starting vm

[quasar366]

I'm using an Ubuntu Desktop 14.04 System 64bit and latest kernel 3.19.2 with latest grsecurity patch (grsecurity-3.1-3.19.2-201503182219)

If I try to start a kvm vm, I get high cpu load and high i/o on my ssd and the destop freeze. I'm not able to do anything more.
The latest, which I can see on the vm is, that it is trying to reverse some interrupts.
The same happens, without pax and if I deactivate all grsec options. I tried the same with a unmodified 3.19.2 kernel, where I do not have this problem.

Is there anything, where I can help to sort out this problem? I have no log messages

On my server machines, all vm's are running fine and I do not have this problems and it wasn't a problem on desktop system with kernel 3,19.1 !

regards

[PaX Team]

what are the differences between the server and desktop configs? can you perhaps isolate the one that triggers the problem?

[quasar366]

Ther were more differences, but now I also tried the kernel config from my server with, but with following differences:
Preemption model -> Desktop
Grsecurity Usage Type -> Desktop

and the kvm version on my server is 2.0.0 and on desktop I'm using kvm release 2.1.2

It seems, that the desktop freezes, when the vm is trying to initialize the hardware at the first seconds.

What may be also interesting, that some small vm's are starting whithout problems. No freeze with a centos 6.6 system, but ubuntu 12.04 / 14.04 and Windows 7 vm's are not running.
It may be something with kernel release, because centos is running with 2.6 and some other systems with kernel 3.2.
The ubuntu systems, where system freezes are using 3.11 ubuntu original kernel and the other 3.19.x with grsecurity

And sorry, I don't know a possibility, to isolate one of the systems, but I'm thinking about a way to downgrade the kvm version.

I could send you my kernel configs per mail, if you need it.

in addition, I have a lcd display, where I can see, that the system is working about 1-2 seconds, when the desktop freeze

[PaX Team]

so i tried to reproduce this but everything seems to work fine for me (with qemu 2.2.1). it'd be really important to get some kernel logs out of the failing host. perhaps you could try to reproduce this using nested virtualization (pass nested=1 to kvm_intel to enable that mode), i.e., run a guest inside another grsec guest on a non-grsec host and log the outer guest's dmesg to a serial console.

[quasar366]

Sorry for my late response! I'm too busy at moment.

At first, thanks for trying to reproduce this issue! Perhaps I have news!

It may be hardware relevant. On my desktop system, I'm using a Intel Core i3 (4th gen) and on my server, I'm using a Core i5 (4th Gen)

I have also a laptop with a core i5 (3th gen) with a ubuntu desktop system, where I installed today an ubuntu system (like on my desktop system). The result was, no freeze on the desktop system.

Either it have to do with the iommu support (where I saw, it was not compiled as a kernel modul but integrated), or I have an hardware issue. Tomorrow should arrive a new core i5 cpu for my desktop system, then I'll be back to report.

I wasn't able to try the nested option, because I had no time yet. I'll do my tests on weekend!

edit: I saw, there is no possibility to load the intel_iommu support as module

[PaX Team]

another idea: you could try booting with nopcid passed on the kernel command line to see if it makes a difference (and perhaps post dmesg from a working/non-working kernel). also can you show me the qemu command line you use to start your guests (in particular i'm wondering about the -cpu option)?

[quasar366]

I found the culprit for!

I never had an eye catch on my memory, because I have 26GB RAM on my desktop system! So I'm using hugepages and I'm normaly using a 1GB hugepagesize and 4 hugepages. When I start the system with kernel 3.19.2 or 3.19.3 only 1,5 GB of RAM has been left and the rest is used by system/hugepage without running any extra application. (which is of course to less to run a vm and reserved 8GB ram )
If I'm booting the system with a hugepagesize of 256MB, I have 25GB free RAM and all my vm's are running fine now!


The desktop system never freezed completely. I was seeing on my lcd by booting a live cd in a vm today, that the system had a load average about 56.

edit: on my server I had always activated a smaller hugepagesize by default ....

[quasar366]

In addition, I saw the kernel 3.19.2 changelog on kernel.org and there were some changes on huge
If you want to reproduce that, I'm also using the overcommit option in systctl:

Code: Select all

vm.overcommit_memory = 2
vm.overcommit_ratio = 8000
vm.swappiness = 0

vm.hugetlb_shm_group = 2021
vm.nr_hugepages = 256
kernel.shmmax = 8589934592

[quasar366]

I would suggest to mark this thread as solved, because of misinterpreted vm.nr_hugepages in my sysctl which was a remnant, as I used the default hugepagesize of 2MB

If I use 1GB and 4 hugpages, the system will allocate 4GB RAM for the huge as expected.

So I have to say, sorry for my loudness and thanks for your awesome work!