Re: [Qemu-devel] QEMU patch to allow VM introspection via libvmi

[Valerio Aimale]

On 10/26/15 3:09 AM, Markus Armbruster wrote:
> [...]
>
> > Eduardo, I think it would be a common rule of politeness not to pass
> > any judgement on a person that you don't know, but for some texts in a
> > mailing list. I think I understand how mmap() works, and very well.
> >
> > Participating is this discussion has been a struggle for me. For the
> > good of the libvmi users, I have been trying to ignore the judgements,
> > the comments and so on. But, alas, I throw my hands up in the air, and
> > I surrender.
> I'm sorry we exceeded your tolerance for frustration.  This mailing list
> can be tough.  We try to be welcoming (believe it or not), but we too
> often fail (okay, that part is easily believable).
>
> To be honest, I had difficulties understanding your explanation, and
> ended up guessing.  I figure Eduardo did the same, and guessed
> incorrectly.  There but for the grace of God go I.
Well, I did scribble my C sample excerpt too fast. Participating in 
mailing list is not part of my job description - I was short on time, I 
admit to that. However, there is a big difference in saying "I do no 
understand your explanation, please try again" and saying "you're 
confused about mmap()"

I was trying to advocate the use of a shared mmap'ed region. The sharing 
would be two-ways (RW for both) between the QEMU virtualizer and the 
libvmi process. I envision that there could be a QEMU command line 
argument, such as "--mmap-guest-memory <filename>" Understand that Eric 
feels strongly the libvmi client should own the file name - I have not 
forgotten that. When that command line argument is given, as part of the 
guest initialization, QEMU creates a file of size equal to the size of 
the guest memory containing all zeros, mmaps that file to the guest 
memory with  PROT_READ|PROT_WRITE and MAP_FILE|MAP_SHARED, then starts 
the guest. And, if at all possible, makes the filename querable via qmp 
and/or hmp, so that the filename of the mmap would not need to be 
maintained in two different places, leading to maintenance nightmares.

Shared mmaped regions can be used as inter-process communication, here's 
a quick and dirty example:

p1.c
---
#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/file.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <errno.h>
#include <signal.h>

void handle_signal(int signal);

/* sorry, for ease of development I need these to be global */
int fh;
char *p;

void handle_signal(int signal) {
    const char *signal_name;
    sigset_t pending;

    switch (signal) {
        case SIGHUP:
            signal_name = "SIGHUP";
            fprintf(stdout, "Process 1 -- Map now contains: %s\n", p);
            munmap(p, sysconf(_SC_PAGE_SIZE) );
            close(fh);
            exit(0);
            break;
        default:
            fprintf(stderr, "Caught wrong signal: %d\n", signal);
            return;
    }
}


void main(int argc, char **argv) {
        struct sigaction sa;

        sa.sa_handler = &handle_signal;
        sa.sa_flags = SA_RESTART;
        sigfillset(&sa.sa_mask);
        if (sigaction(SIGHUP, &sa, NULL) == -1) {
                perror("Error: cannot handle SIGHUP");
                exit(1);
        }

        if ( (fh = open("shared.map", O_RDWR | O_CREAT, S_IRWXU)) ) {
                p = mmap(NULL, sysconf(_SC_PAGE_SIZE), 
PROT_READ|PROT_WRITE, MAP_FILE|MAP_SHARED, fh, (off_t) 0);

                if (p == MAP_FAILED) { printf("poop, didn't map: 
%s\n",strerror(errno)); close(fh); exit(1);}

                p[0] = 0xcc;

                fprintf(stdout, "Process 1 -- Writing to map:   All 
your bases are belong to us.\n");
                sprintf( (char*) p, "All your bases are belong to us.");

                while(1);

        }

}
---

p2.c
---
#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/file.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <errno.h>
#include <signal.h>

void main(int argc, char **argv) {
        int fh;
        void *p;
        int pid;

        pid = atoi(argv[1]);

        sleep(1);

        if ( (fh = open("shared.map", O_RDWR, S_IRWXU)) ) {
                p = mmap(NULL, sysconf(_SC_PAGE_SIZE), 
PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FILE,
                  fh, (off_t) 0);

                printf("Process 2 -- Map now contains: %s\n", (char*)p);
                printf("Process 2 -- Writing to map:   All your bases 
*NOW* belong to us.\n");
                fflush(stdout);
                sprintf( (char*) p, "All your bases *NOW* belong to us.");

                kill(pid, SIGHUP);

                sleep(3);

                munmap(p, sysconf(_SC_PAGE_SIZE));
                close(fh);
        }

}
---

if I run both, in bash, as:

rm -f shared.map ; \
gcc -o p1 p1.c ; \
gcc -o p2 p2.c ; \
for i in ` seq 1 `getconf PAGESIZE``; do echo -e -n "\0" > shared.map ; 
done ; \
./p1 & ./p2 $!

I get the following output:

$ rm -f shared.map ; \
> gcc -o p1 p1.c ; \
> gcc -o p2 p2.c ; \
> for i in ` seq 1 `getconf PAGESIZE``; do echo -e -n "\0" > shared.map 
; done ; \
> ./p1 & ./p2 $!
[1] 8223
Process 1 -- Writing to map:   All your bases are belong to us.
Process 2 -- Map now contains: All your bases are belong to us.
Process 2 -- Writing to map:   All your bases *NOW* belong to us.
Process 1 -- Map now contains: All your bases *NOW* belong to us.
[1]+  Done                    ./p1

To me, the example above shows the used of a mmaped region shared 
between two processes.

C code above was written too fast again! Some headers are redundant, I 
do not check for all error conditions, I don't do all the cleanup - it's 
kind of quick and dirty. I hope you can forgive me.
(it works on mac and linux unmodified, though quick and dirty)

I am fascinated by Daniel's suggestion of accessing the guest memory via 
the QMU process existing map, found through /dev/pid/mmaps. It is my 
understanding that Daniel's suggestion would require ptrace()'ing the 
QEMU process, and potentially stopping the QEMU process to access the 
guest mmap'ed memory.
> This thread is in reply to Valerio's attempt to upstream this patch.
> Good move.
>
> The usual questions for feature requests apply:
>
> 1. Is this a use case we want to serve?
>
>     Unreserved yes.  Supporting virtual machine introspection with LibVMI
>     makes sense.
>
> [...]

Markus that's a great description of  needs and potential approaches. It 
requires a bit of thinking. I reserve the right to provide comments 
later on.