Re: [Qemu-devel] [RFC] tracing: per-CPU tracing state

[Lluís]

Stefan Hajnoczi writes:

> On Mon, Nov 8, 2010 at 2:35 PM, Lluís <address@hidden> wrote:
>> As more and more tracing backends appear (right now: nop, simple, ust
>> and the new dtrace), it is harder to provide a tracetool where these
>> backends can be efficiently used without being aware of per-CPU tracing
>> state.

> Per-CPU as in per-vcpu or thread-local in QEMU?

Sorry, per-vCPU. In fact, the problem has two dimensions:

* per-vCPU tracing event states (current events are QEMU-global)
* tracing events from guest code (such that the trace is produced from
  an auto-generated TCG call)

Right now, I use both together, but they could be seggregated into two
separate event properties.


>> What I have now is basically:
>> 
>> * trace.h : trace_##name
>>  Backend-specific declaration/definition of per-event tracing functions
>> 
>> * trace-cpu.h : trace_cpu_##name
>>  Generic "static inline" definitions of functions that check the
>>  per-CPU state and (if true) call the corresponding trace_##name
>> 
>> * trace-gen.h: trace_gen_##name
>>  Generic "static inline" definitions of functions that check the
>>  per-CPU state and (if true) generate TCG calls to a function
>>  redirecting the call to trace_##name
>> 
>> The last two are only generated for those events that have the "gen"
>> keyword in "trace-events".
>> 
>> The point is that the trace_##name functions do an extra check (the one
>> provided by the backend) that should not be necessary, as the per-CPU
>> state check already ensures that the trace must be produced.
>> 
>> Thus, I'd propose to provide an extra per-backend function that prodices
>> the trace without checking the tracing state.

> Is this already implemented in patches you've sent to the list?  I'm a
> little behind on the mailing list right now but would like to check
> out your work to understand your ideas better.

The trace-*.h changes have already been sent on some early stages, but
right now I'm overhauling those patch series as to provide a more clean
implementation (without having the per-backend tracing without state
checking).

But the idea is basically that of having traces of the code executed on
the guest (instructions executed, memory accesses, changes of privilege
level, changes of address-space, etc.).

Of course, I could still provide these events without the per-vCPU
states, but the latter allow me, for example, to trace only certaing
processess, only trace kernel code, or only user code, etc.


>> Another important change would be to provide a generic interface inside
>> QEMU that provides a programmatic control of the tracing state of all
>> events (both per-cpu and global), and let each backend provide a small
>> .c file implementing that functionality. This would provide a
>> centralized point through QEMU monitor for the control of tracing
>> events, plus any backend-specific control interface that might also be
>> available.

> Programmatic control can't be provided for all backends.

> If you're using a platform-specific trace backend then you probably
> want to use those tools.  For example, DTrace has a powerful language
> for aggregating trace data and reporting.  DTrace isn't about
> enabling/disabling probes, it does much more which we cannot usefully
> work into a common subset interface.  UST has its own control tool to
> query and configure tracepoints in a running process.

> I think it makes sense to use each backend's native tools.  If we try
> to come up with a common interface it will cost effort to maintain and
> users will be better off using the native tools.  Imagine a Fedora
> user wanting to instrument QEMU, why learn a QEMU-specific subset
> interface rather than using SystemTap in the same way you can
> instrument the kernel or other applications?

> The simple backend is built in to QEMU and therefore does need a
> HMP/QMP interface for control.

> Am I missing the cases where a common interface is useful?

I see. My current concern is how to integrate the per-vCPU state with
the backend-specific per-event state.

Right now I have a programmatic interface for controlling the per-vCPU
state, but that is oblivious of the global backend-specific per-event
state (which must be enabled beforehand), but I dont know how to provide
per-backend _and_ per-vCPU control of tracing state.

Maybe it's just not necessary, given that the trace events I'm inserting
are disabled by default, and then it's just a matter of flipping two
bits instead of one (although from separate tools).


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth