Re: QEMU function trace

[Alex Bennée]

wanghw364  <wanghw364@163.com> writes:

> Thanks. I have several questions as below, please help, thanks.
>
> 1.What do you mean by "only have debug symbols available for linux-user so"? 
> What does the linux-user so
> refer to?
>  qemu_plugin_insn_symbol() can only see symbols from linux-user so?

The linux-user ELF loader will read the debug symbols (if they exist)
and populate the syminfos structures that lookup_symbol uses. It's
partially obscured by the ELF loaders heavy use of macros but see:

  static void glue(load_symbols, SZ)(struct elfhdr *ehdr, int fd, int must_swab,
                                     int clear_lsb, symbol_fn_t sym_cb)

in elf_ops.h

> 2.The purpose of teaching the linux kernel loader to understand and relocate 
> symbols from an ELF kernel
> image,
> or extract then and feed them directly to the plugin, is to solve the issue 
> that qemu_plugin_insn_symbol()
> can't see kernel symbol?

Yes. This is slightly complicated by the fact that the kernel loaders don't
expect to load pure ELF files but something that is wrapped up as a
Linux loader. For example:

  ➜  file vmlinux
  vmlinux: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), 
statically linked, BuildID[sha1]=21166458a10404e6157abf0da4a0921144c72675, with 
debug_info, not stripped
  🕙10:07:42 alex@zen:linux.git/builds/arm64.initramfs with 
arm64/aarch64-linux-gnu- on  linux-6.0.y [$!?] 
  ➜  file arch/arm64/boot/Image
  arch/arm64/boot/Image: Linux kernel ARM64 boot executable Image, 
little-endian, 4K pages

The second file is what is actually passed to -kernel in a typical boot.

The logic in arm_setup_direct_kernel_boot() implies you can load ELFs
directly and boot them but for some reason the Linux kernel doesn't work
if you try this way.

> 3.How to make the kernel loader understand and relocate symbols in QEMU? How 
> to feed the symbol table
> directly to the plugin?
> As I can see, cache plugin has used qemu_plugin_insn_symbol() and there is 
> function name info in the output
> result, 
> but it seems there is no symbol table feeding in the command, shown in
> https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/tcg-plugins.rst 
> . 
> $ qemu-x86_64 -plugin ./contrib/plugins/libcache.so -d plugin -D cache.log .
> /tests/tcg/x86_64-linux-user/float_convs
>
> So I was wondering how the symbol table was fed into the plugin? What is the 
> usage of para .
> /tests/tcg/x86_64-linux-user/float_convs?

It came directly from the debug symbols embedded in the ELF binary.

> 4.If we make kernel symbol visible to qemu_plugin_insn_symbol(), the only 
> thing we need to do is to make the
> core model identify which instruction
> is the start of one function and record the function trace by looking up 
> symbol table once the function-level
> start instruction was executed?
>
> Actually I have the kernel symbol table file named 'System.map' under the 
> kernel directory, I was wondering
> how to feed it to the plugin.

You could certainly write a System.map parser in your plugin and get the
addresses from that instead. It would probably be faster than working
out what to fix in the kernel load path.

>
> Thanks.
>
> At 2022-12-13 23:44:29, "Alex Bennée" <alex.bennee@linaro.org> wrote:
>>
>>wanghw364 <wanghw364@163.com> writes:
>>
>>> Hi all,
>>>
>>> Does qemu-system-riscv64 have any plugin or tools that can support target 
>>> program function trace feature?
>>>
>>> It seems there is no such feature under
>>> link:https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/tcg-plugins.rst
>>>  
>>>
>>> For example, we can use libexeclog.so plugin to trace target program 
>>> instruction trace.
>>>
>>> In my case, when I boot linux kernel with qemu, it hangs in the halfway, 
>>> but I don't know the hang position in
>>> the code, 
>>>
>>> so I want to trace the kernel function calling trace so that I can
>>> find out when and where execution diverges.
>>
>>Not currently but it wouldn't be super hard to write such a thing.
>>However currently we only have debug symbols available for linux-user so
>>that is all the helper qemu_plugin_insn_symbol() will see.
>>
>>You need to teach the linux kernel loader to understand and relocate
>>symbols from an ELF kernel image. Alternatively you could extract then
>>and feed them directly to the plugin. It would then be fairly trivial to
>>stick an execution callback at every function entrance.
>>
>>I suspect KASLR messes things up though.
>>
>>>
>>> Thanks. 
>>
>>
>>-- 
>>Alex Bennée


-- 
Alex Bennée