[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH for-2.8 v1 00/60] Modular build of trace event files
From: |
Daniel P. Berrange |
Subject: |
[Qemu-devel] [PATCH for-2.8 v1 00/60] Modular build of trace event files |
Date: |
Tue, 9 Aug 2016 16:31:28 +0100 |
I previously split the global trace-events file up into one file
per-subdirectory to avoid merge conflict hell.
This series builds on that to now actually generate the individual
trace files per-subdirectory too. The key benefit of doing this is
that a change in a trace-events file will no longer cause a rebuild
of the entire world. Instead only that one affected sub-directory
will have files rebuilt.
The key problem faced when creating this patch series was the fact
that the trace code had a global assumption that there was a single
enum declaring all events in a sequence with no gaps. Most of the
hard work in this series involves getting rid of that assumption
throughout the code. Instead we now have a notion of trace event
groups, each with their own distinct enum of events. As such the
individual event IDs are no longer globally unique. Most of the
code could be adapted to work with IDs that were unique only within
a group.
The first sticky part was the simpletrace binary event format which
includes the event ID. To deal with this I invented a notion of an
ID offset per trace-events file. So simpletrace format will now
use "(id-offset << 16) | event-id" in order to get a globally
unique ID value for its trace files. This means each trace-events
file can only have 65536 distinct events declared, but that ought
to be enough for a while :-)
The second sticky part was the per-VCPU events, due to the use of
a "trace_dstate" bitmap in the CPUState struct. The bitmap
inherantly assumes a global ID sequence, and would not work well
with the per-file id-offset concept, since it would require a
huge mostly-sparse bitmap. Rather than try to solve this problem
I took the easy way out and simply specified that all per-VCPU
events need to be declared in the top level trace-events file.
Since we have almost no per-vCPU events today, this works out
just fine.
I've build-tested with all targets enabled in both VPATH and
non-VPATH setups, on Fedora 24, with all 5 trace event backends
enabled at once. I've tested that simpletrace.py can still
extract events from generated traces. I've not functionally
tested dtrace/ftrace/ust at this time.
Daniel P. Berrange (60):
trace: move util/buffer.c trace points into correct file
trace: move util/qemu-coroutine*.c trace points into correct file
trace: move hw/mem/pc-dimm.c trace points into correct file
trace: move hw/virtio/virtio-balloon.c trace points into correct file
trace: add trace event iterator APIs
trace: convert code to use event iterators
trace: remove some now unused functions
trace: provide mechanism for registering trace events
trace: remove use of TRACE_VCPU_EVENT_COUNT in cpu.h
trace: remove fixed global event state arrays
trace: remove use of event ID enums from APIs
trace: use -1 instead of TRACE_VCPU_EVENT_COUNT as magic value
trace: remove generated-events.h from many includes
trace: get rid of generated-events.h/generated-events.c
trace: introduce a formal group name for trace events
trace: introduce ID range offsets per trace-events file
trace: introduce some Makefile rules for module code gen
trace: switch io/ directory to modular trace.h file
trace: switch util/ directory to modular trace.h file
trace: switch crypto/ directory to modular trace.h file
trace: switch migration/ directory to modular trace.h file
trace: switch block/ directory to modular trace.h file
trace: switch hw/block/ directory to modular trace.h file
trace: switch hw/char/ directory to modular trace.h file
trace: switch hw/intc/ directory to modular trace.h file
trace: switch hw/net/ directory to modular trace.h file
trace: switch hw/virtio/ directory to modular trace.h file
trace: switch hw/audio/ directory to modular trace.h file
trace: switch hw/misc/ directory to modular trace.h file
trace: switch hw/usb/ directory to modular trace.h file
trace: switch hw/scsi/ directory to modular trace.h file
trace: switch hw/nvram/ directory to modular trace.h file
trace: switch hw/display/ directory to modular trace.h file
trace: switch hw/input/ directory to modular trace.h file
trace: switch hw/timer/ directory to modular trace.h file
trace: switch hw/dma/ directory to modular trace.h file
trace: switch hw/sparc/ directory to modular trace.h file
trace: switch hw/sd/ directory to modular trace.h file
trace: switch hw/isa/ directory to modular trace.h file
trace: switch hw/mem/ directory to modular trace.h file
trace: switch hw/i386/ directory to modular trace.h file
trace: switch hw/9pfs/ directory to modular trace.h file
trace: switch hw/ppc/ directory to modular trace.h file
trace: switch hw/pci/ directory to modular trace.h file
trace: switch hw/s390x/ directory to modular trace.h file
trace: switch hw/vfio/ directory to modular trace.h file
trace: switch hw/acpi/ directory to modular trace.h file
trace: switch hw/arm/ directory to modular trace.h file
trace: switch hw/alpha/ directory to modular trace.h file
trace: switch ui/ directory to modular trace.h file
trace: switch audio/ directory to modular trace.h file
trace: switch net/ directory to modular trace.h file
trace: switch target-i386/ directory to modular trace.h file
trace: switch target-sparc/ directory to modular trace.h file
trace: switch target-s390x/ directory to modular trace.h file
trace: switch target-ppc/ directory to modular trace.h file
trace: switch qom/ directory to modular trace.h file
trace: switch linux-user/ directory to modular trace.h file
trace: remove the global include/trace.h file
trace: update docs to reflect new code generation approach
.gitignore | 15 +-
Makefile | 116 +++++++++++---
Makefile.objs | 89 ++++++-----
Makefile.target | 7 +-
audio/Makefile.objs | 2 +
audio/alsaaudio.c | 2 +-
audio/ossaudio.c | 2 +-
audio/trace-events | 2 +
block.c | 2 +-
block/Makefile.objs | 2 +
block/backup.c | 2 +-
block/block-backend.c | 2 +-
block/commit.c | 2 +-
block/dirty-bitmap.c | 2 +-
block/io.c | 2 +-
block/mirror.c | 2 +-
block/nfs.c | 2 +-
block/qcow2-cache.c | 2 +-
block/qcow2-cluster.c | 2 +-
block/qcow2.c | 2 +-
block/qed-l2-cache.c | 2 +-
block/qed-table.c | 2 +-
block/qed.c | 2 +-
block/raw-posix.c | 2 +-
block/raw-win32.c | 2 +-
block/stream.c | 2 +-
block/trace-events | 2 +
blockdev.c | 2 +-
crypto/Makefile.objs | 2 +
crypto/secret.c | 2 +-
crypto/tlscreds.c | 2 +-
crypto/tlscredsanon.c | 2 +-
crypto/tlscredsx509.c | 2 +-
crypto/tlssession.c | 2 +-
crypto/trace-events | 2 +
docs/tracing.txt | 61 ++++++--
hw/9pfs/9p.c | 2 +-
hw/9pfs/Makefile.objs | 2 +
hw/9pfs/trace-events | 2 +
hw/acpi/Makefile.objs | 2 +
hw/acpi/cpu.c | 2 +-
hw/acpi/memory_hotplug.c | 2 +-
hw/acpi/trace-events | 2 +
hw/alpha/Makefile.objs | 2 +
hw/alpha/pci.c | 2 +-
hw/alpha/trace-events | 2 +
hw/arm/Makefile.objs | 2 +
hw/arm/trace-events | 2 +
hw/arm/virt-acpi-build.c | 2 +-
hw/audio/Makefile.objs | 3 +
hw/audio/cs4231.c | 2 +-
hw/audio/milkymist-ac97.c | 2 +-
hw/audio/trace-events | 2 +
hw/block/Makefile.objs | 2 +
hw/block/dataplane/virtio-blk.c | 2 +-
hw/block/hd-geometry.c | 2 +-
hw/block/trace-events | 2 +
hw/block/virtio-blk.c | 2 +-
hw/char/Makefile.objs | 2 +
hw/char/escc.c | 2 +-
hw/char/grlib_apbuart.c | 2 +-
hw/char/lm32_juart.c | 2 +-
hw/char/lm32_uart.c | 2 +-
hw/char/milkymist-uart.c | 2 +-
hw/char/trace-events | 2 +
hw/char/virtio-console.c | 2 +-
hw/char/virtio-serial-bus.c | 2 +-
hw/display/Makefile.objs | 3 +
hw/display/g364fb.c | 2 +-
hw/display/jazz_led.c | 2 +-
hw/display/milkymist-tmu2.c | 2 +-
hw/display/milkymist-vgafb.c | 2 +-
hw/display/qxl-render.c | 2 +-
hw/display/qxl.c | 2 +-
hw/display/trace-events | 2 +
hw/display/vga.c | 2 +-
hw/display/virtio-gpu-3d.c | 2 +-
hw/display/virtio-gpu.c | 2 +-
hw/display/vmware_vga.c | 2 +-
hw/display/xenfb.c | 2 +-
hw/dma/Makefile.objs | 2 +
hw/dma/i8257.c | 2 +-
hw/dma/rc4030.c | 2 +-
hw/dma/sparc32_dma.c | 2 +-
hw/dma/sun4m_iommu.c | 2 +-
hw/dma/trace-events | 2 +
hw/i386/Makefile.objs | 2 +
hw/i386/trace-events | 6 +-
hw/i386/x86-iommu.c | 2 +-
hw/i386/xen/xen_platform.c | 1 +
hw/i386/xen/xen_pvdevice.c | 1 +
hw/input/Makefile.objs | 3 +
hw/input/hid.c | 2 +-
hw/input/milkymist-softusb.c | 2 +-
hw/input/ps2.c | 2 +-
hw/input/trace-events | 2 +
hw/input/virtio-input.c | 2 +-
hw/intc/Makefile.objs | 2 +
hw/intc/apic.c | 2 +-
hw/intc/apic_common.c | 2 +-
hw/intc/arm_gic.c | 2 +-
hw/intc/arm_gicv3_cpuif.c | 2 +-
hw/intc/arm_gicv3_dist.c | 2 +-
hw/intc/arm_gicv3_redist.c | 2 +-
hw/intc/aspeed_vic.c | 2 +-
hw/intc/grlib_irqmp.c | 2 +-
hw/intc/lm32_pic.c | 2 +-
hw/intc/s390_flic.c | 2 +-
hw/intc/s390_flic_kvm.c | 2 +-
hw/intc/slavio_intctl.c | 2 +-
hw/intc/trace-events | 2 +
hw/intc/xics.c | 2 +-
hw/intc/xics_kvm.c | 2 +-
hw/intc/xics_spapr.c | 2 +-
hw/isa/Makefile.objs | 2 +
hw/isa/pc87312.c | 2 +-
hw/isa/trace-events | 2 +
hw/mem/Makefile.objs | 3 +
hw/mem/pc-dimm.c | 2 +-
hw/mem/trace-events | 7 +
hw/misc/Makefile.objs | 3 +
hw/misc/aspeed_scu.c | 2 +-
hw/misc/eccmemctl.c | 2 +-
hw/misc/milkymist-hpdmc.c | 2 +-
hw/misc/milkymist-pfpu.c | 2 +-
hw/misc/slavio_misc.c | 2 +-
hw/misc/trace-events | 2 +
hw/net/Makefile.objs | 3 +
hw/net/e1000e.c | 2 +-
hw/net/e1000e_core.c | 2 +-
hw/net/e1000x_common.c | 2 +-
hw/net/lance.c | 2 +-
hw/net/milkymist-minimac2.c | 2 +-
hw/net/mipsnet.c | 2 +-
hw/net/net_rx_pkt.c | 2 +-
hw/net/opencores_eth.c | 2 +-
hw/net/pcnet-pci.c | 2 +-
hw/net/pcnet.c | 2 +-
hw/net/trace-events | 2 +
hw/nvram/Makefile.objs | 2 +
hw/nvram/ds1225y.c | 2 +-
hw/nvram/fw_cfg.c | 2 +-
hw/nvram/trace-events | 2 +
hw/pci/Makefile.objs | 2 +
hw/pci/pci.c | 2 +-
hw/pci/pci_host.c | 2 +-
hw/pci/trace-events | 2 +
hw/ppc/Makefile.objs | 3 +
hw/ppc/ppc.c | 2 +-
hw/ppc/prep.c | 2 +-
hw/ppc/spapr.c | 2 +-
hw/ppc/spapr_hcall.c | 2 +-
hw/ppc/spapr_iommu.c | 2 +-
hw/ppc/spapr_pci.c | 2 +-
hw/ppc/spapr_rtas_ddw.c | 2 +-
hw/ppc/trace-events | 2 +
hw/s390x/Makefile.objs | 3 +
hw/s390x/css.c | 2 +-
hw/s390x/trace-events | 2 +
hw/s390x/virtio-ccw.c | 2 +-
hw/scsi/Makefile.objs | 3 +
hw/scsi/esp-pci.c | 2 +-
hw/scsi/esp.c | 2 +-
hw/scsi/megasas.c | 2 +-
hw/scsi/mptconfig.c | 2 +-
hw/scsi/mptendian.c | 2 +-
hw/scsi/mptsas.c | 2 +-
hw/scsi/scsi-bus.c | 2 +-
hw/scsi/trace-events | 2 +
hw/scsi/vmw_pvscsi.c | 2 +-
hw/sd/Makefile.objs | 2 +
hw/sd/milkymist-memcard.c | 2 +-
hw/sd/trace-events | 2 +
hw/sparc/Makefile.objs | 2 +
hw/sparc/leon3.c | 2 +-
hw/sparc/sun4m.c | 2 +-
hw/sparc/trace-events | 2 +
hw/timer/Makefile.objs | 2 +
hw/timer/aspeed_timer.c | 2 +-
hw/timer/grlib_gptimer.c | 2 +-
hw/timer/lm32_timer.c | 2 +-
hw/timer/milkymist-sysctl.c | 2 +-
hw/timer/slavio_timer.c | 2 +-
hw/timer/trace-events | 2 +
hw/usb/Makefile.objs | 3 +
hw/usb/bus.c | 2 +-
hw/usb/combined-packet.c | 2 +-
hw/usb/core.c | 2 +-
hw/usb/desc.c | 2 +-
hw/usb/dev-hub.c | 2 +-
hw/usb/dev-mtp.c | 2 +-
hw/usb/dev-uas.c | 2 +-
hw/usb/hcd-ehci.c | 2 +-
hw/usb/hcd-ohci.c | 8 +-
hw/usb/hcd-uhci.c | 2 +-
hw/usb/hcd-xhci.c | 2 +-
hw/usb/host-libusb.c | 2 +-
hw/usb/trace-events | 2 +
hw/vfio/Makefile.objs | 2 +
hw/vfio/common.c | 2 +-
hw/vfio/pci-quirks.c | 2 +-
hw/vfio/pci.c | 2 +-
hw/vfio/platform.c | 2 +-
hw/vfio/spapr.c | 2 +-
hw/vfio/trace-events | 2 +
hw/virtio/Makefile.objs | 2 +
hw/virtio/trace-events | 7 +
hw/virtio/virtio-balloon.c | 2 +-
hw/virtio/virtio-rng.c | 2 +-
hw/virtio/virtio.c | 2 +-
include/qemu/module.h | 2 +
include/qom/cpu.h | 10 +-
include/trace-tcg.h | 1 -
include/trace.h | 7 -
io/Makefile.objs | 2 +
io/channel-buffer.c | 2 +-
io/channel-command.c | 2 +-
io/channel-file.c | 2 +-
io/channel-socket.c | 2 +-
io/channel-tls.c | 2 +-
io/channel-websock.c | 2 +-
io/task.c | 2 +-
io/trace-events | 6 +-
linux-user/Makefile.objs | 2 +
linux-user/signal.c | 2 +-
linux-user/trace-events | 2 +
migration/Makefile.objs | 2 +
migration/exec.c | 2 +-
migration/fd.c | 2 +-
migration/migration.c | 2 +-
migration/postcopy-ram.c | 2 +-
migration/qemu-file.c | 2 +-
migration/ram.c | 2 +-
migration/rdma.c | 2 +-
migration/savevm.c | 2 +-
migration/socket.c | 2 +-
migration/tls.c | 2 +-
migration/trace-events | 2 +
migration/vmstate.c | 2 +-
monitor.c | 16 +-
net/Makefile.objs | 2 +
net/filter-mirror.c | 2 +-
net/trace-events | 2 +
net/vhost-user.c | 2 +-
qemu-img.c | 1 +
qemu-io.c | 1 +
qemu-nbd.c | 1 +
qom/Makefile.objs | 2 +
qom/cpu.c | 2 +-
qom/object.c | 2 +-
qom/trace-events | 2 +
scripts/simpletrace.py | 7 +-
scripts/tracetool.py | 10 +-
scripts/tracetool/__init__.py | 59 +++++--
scripts/tracetool/backend/__init__.py | 12 +-
scripts/tracetool/backend/dtrace.py | 6 +-
scripts/tracetool/backend/ftrace.py | 7 +-
scripts/tracetool/backend/log.py | 6 +-
scripts/tracetool/backend/simple.py | 13 +-
scripts/tracetool/backend/ust.py | 6 +-
scripts/tracetool/format/__init__.py | 4 +-
scripts/tracetool/format/c.py | 48 +++++-
scripts/tracetool/format/d.py | 2 +-
scripts/tracetool/format/events_c.py | 44 ------
scripts/tracetool/format/events_h.py | 60 -------
scripts/tracetool/format/h.py | 51 +++++-
scripts/tracetool/format/simpletrace_stap.py | 4 +-
scripts/tracetool/format/stap.py | 2 +-
scripts/tracetool/format/tcg_h.py | 2 +-
scripts/tracetool/format/tcg_helper_c.py | 2 +-
scripts/tracetool/format/tcg_helper_h.py | 2 +-
scripts/tracetool/format/tcg_helper_wrapper_h.py | 2 +-
scripts/tracetool/format/ust_events_c.py | 4 +-
scripts/tracetool/format/ust_events_h.py | 15 +-
stubs/trace-control.c | 8 +-
target-i386/Makefile.objs | 2 +
target-i386/kvm.c | 2 +-
target-i386/trace-events | 2 +
target-ppc/Makefile.objs | 2 +
target-ppc/kvm.c | 2 +-
target-ppc/trace-events | 2 +
target-s390x/Makefile.objs | 2 +
target-s390x/cpu.c | 2 +-
target-s390x/ioinst.c | 2 +-
target-s390x/kvm.c | 2 +-
target-s390x/mmu_helper.c | 2 +-
target-s390x/trace-events | 2 +
target-sparc/Makefile.objs | 2 +
target-sparc/int32_helper.c | 2 +-
target-sparc/int64_helper.c | 2 +-
target-sparc/mmu_helper.c | 2 +-
target-sparc/trace-events | 2 +
target-sparc/win_helper.c | 2 +-
trace-events | 19 +--
trace/Makefile.objs | 98 ++----------
trace/control-internal.h | 39 ++---
trace/control-target.c | 20 +--
trace/control.c | 191 +++++++++++++++++------
trace/control.h | 108 +++----------
trace/event-internal.h | 7 +-
trace/qmp.c | 29 ++--
trace/simple.c | 6 +-
trace/simple.h | 6 +-
ui/Makefile.objs | 3 +
ui/console.c | 2 +-
ui/gtk-egl.c | 2 +-
ui/gtk-gl-area.c | 2 +-
ui/gtk.c | 2 +-
ui/input.c | 2 +-
ui/spice-display.c | 2 +-
ui/trace-events | 2 +
ui/vnc.c | 2 +-
util/Makefile.objs | 2 +
util/buffer.c | 2 +-
util/hbitmap.c | 2 +-
util/oslib-posix.c | 2 +-
util/oslib-win32.c | 2 +-
util/qemu-coroutine-lock.c | 2 +-
util/qemu-coroutine.c | 2 +-
util/trace-events | 21 +++
vl.c | 2 +
321 files changed, 1058 insertions(+), 768 deletions(-)
create mode 100644 hw/mem/trace-events
delete mode 100644 include/trace.h
delete mode 100644 scripts/tracetool/format/events_c.py
delete mode 100644 scripts/tracetool/format/events_h.py
--
2.7.4
- [Qemu-devel] [PATCH for-2.8 v1 00/60] Modular build of trace event files,
Daniel P. Berrange <=
- [Qemu-devel] [PATCH for-2.8 v1 02/60] trace: move util/qemu-coroutine*.c trace points into correct file, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 01/60] trace: move util/buffer.c trace points into correct file, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 07/60] trace: remove some now unused functions, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 04/60] trace: move hw/virtio/virtio-balloon.c trace points into correct file, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 05/60] trace: add trace event iterator APIs, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 03/60] trace: move hw/mem/pc-dimm.c trace points into correct file, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 06/60] trace: convert code to use event iterators, Daniel P. Berrange, 2016/08/09
- [Qemu-devel] [PATCH for-2.8 v1 08/60] trace: provide mechanism for registering trace events, Daniel P. Berrange, 2016/08/09