qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-devel] [PATCH for-2.8 v1 00/60] Modular build of trace event files


From: Daniel P. Berrange
Subject: [Qemu-devel] [PATCH for-2.8 v1 00/60] Modular build of trace event files
Date: Tue, 9 Aug 2016 16:31:28 +0100

I previously split the global trace-events file up into one file
per-subdirectory to avoid merge conflict hell.

This series builds on that to now actually generate the individual
trace files per-subdirectory too. The key benefit of doing this is
that a change in a trace-events file will no longer cause a rebuild
of the entire world. Instead only that one affected sub-directory
will have files rebuilt.

The key problem faced when creating this patch series was the fact
that the trace code had a global assumption that there was a single
enum declaring all events in a sequence with no gaps. Most of the
hard work in this series involves getting rid of that assumption
throughout the code. Instead we now have a notion of trace event
groups, each with their own distinct enum of events. As such the
individual event IDs are no longer globally unique. Most of the
code could be adapted to work with IDs that were unique only within
a group.

The first sticky part was the simpletrace binary event format which
includes the event ID. To deal with this I invented a notion of an
ID offset per trace-events file. So simpletrace format will now
use   "(id-offset << 16) | event-id" in order to get a globally
unique ID value for its trace files. This means each trace-events
file can only have 65536 distinct events declared, but that ought
to be enough for a while :-)

The second sticky part was the per-VCPU events, due to the use of
a "trace_dstate" bitmap in the CPUState struct. The bitmap
inherantly assumes a global ID sequence, and would not work well
with the per-file id-offset concept, since it would require a
huge mostly-sparse bitmap.  Rather than try to solve this problem
I took the easy way out and simply specified that all per-VCPU
events need to be declared in the top level trace-events file.
Since we have almost no per-vCPU events today, this works out
just fine.

I've build-tested with all targets enabled in both VPATH and
non-VPATH setups, on Fedora 24, with all 5 trace event backends
enabled at once. I've tested that simpletrace.py can still
extract events from generated traces. I've not functionally
tested dtrace/ftrace/ust at this time.

Daniel P. Berrange (60):
  trace: move util/buffer.c trace points into correct file
  trace: move util/qemu-coroutine*.c trace points into correct file
  trace: move hw/mem/pc-dimm.c trace points into correct file
  trace: move hw/virtio/virtio-balloon.c trace points into correct file
  trace: add trace event iterator APIs
  trace: convert code to use event iterators
  trace: remove some now unused functions
  trace: provide mechanism for registering trace events
  trace: remove use of TRACE_VCPU_EVENT_COUNT in cpu.h
  trace: remove fixed global event state arrays
  trace: remove use of event ID enums from APIs
  trace: use -1 instead of TRACE_VCPU_EVENT_COUNT as magic value
  trace: remove generated-events.h from many includes
  trace: get rid of generated-events.h/generated-events.c
  trace: introduce a formal group name for trace events
  trace: introduce ID range offsets per trace-events file
  trace: introduce some Makefile rules for module code gen
  trace: switch io/ directory to modular trace.h file
  trace: switch util/ directory to modular trace.h file
  trace: switch crypto/ directory to modular trace.h file
  trace: switch migration/ directory to modular trace.h file
  trace: switch block/ directory to modular trace.h file
  trace: switch hw/block/ directory to modular trace.h file
  trace: switch hw/char/ directory to modular trace.h file
  trace: switch hw/intc/ directory to modular trace.h file
  trace: switch hw/net/ directory to modular trace.h file
  trace: switch hw/virtio/ directory to modular trace.h file
  trace: switch hw/audio/ directory to modular trace.h file
  trace: switch hw/misc/ directory to modular trace.h file
  trace: switch hw/usb/ directory to modular trace.h file
  trace: switch hw/scsi/ directory to modular trace.h file
  trace: switch hw/nvram/ directory to modular trace.h file
  trace: switch hw/display/ directory to modular trace.h file
  trace: switch hw/input/ directory to modular trace.h file
  trace: switch hw/timer/ directory to modular trace.h file
  trace: switch hw/dma/ directory to modular trace.h file
  trace: switch hw/sparc/ directory to modular trace.h file
  trace: switch hw/sd/ directory to modular trace.h file
  trace: switch hw/isa/ directory to modular trace.h file
  trace: switch hw/mem/ directory to modular trace.h file
  trace: switch hw/i386/ directory to modular trace.h file
  trace: switch hw/9pfs/ directory to modular trace.h file
  trace: switch hw/ppc/ directory to modular trace.h file
  trace: switch hw/pci/ directory to modular trace.h file
  trace: switch hw/s390x/ directory to modular trace.h file
  trace: switch hw/vfio/ directory to modular trace.h file
  trace: switch hw/acpi/ directory to modular trace.h file
  trace: switch hw/arm/ directory to modular trace.h file
  trace: switch hw/alpha/ directory to modular trace.h file
  trace: switch ui/ directory to modular trace.h file
  trace: switch audio/ directory to modular trace.h file
  trace: switch net/ directory to modular trace.h file
  trace: switch target-i386/ directory to modular trace.h file
  trace: switch target-sparc/ directory to modular trace.h file
  trace: switch target-s390x/ directory to modular trace.h file
  trace: switch target-ppc/ directory to modular trace.h file
  trace: switch qom/ directory to modular trace.h file
  trace: switch linux-user/ directory to modular trace.h file
  trace: remove the global include/trace.h file
  trace: update docs to reflect new code generation approach

 .gitignore                                       |  15 +-
 Makefile                                         | 116 +++++++++++---
 Makefile.objs                                    |  89 ++++++-----
 Makefile.target                                  |   7 +-
 audio/Makefile.objs                              |   2 +
 audio/alsaaudio.c                                |   2 +-
 audio/ossaudio.c                                 |   2 +-
 audio/trace-events                               |   2 +
 block.c                                          |   2 +-
 block/Makefile.objs                              |   2 +
 block/backup.c                                   |   2 +-
 block/block-backend.c                            |   2 +-
 block/commit.c                                   |   2 +-
 block/dirty-bitmap.c                             |   2 +-
 block/io.c                                       |   2 +-
 block/mirror.c                                   |   2 +-
 block/nfs.c                                      |   2 +-
 block/qcow2-cache.c                              |   2 +-
 block/qcow2-cluster.c                            |   2 +-
 block/qcow2.c                                    |   2 +-
 block/qed-l2-cache.c                             |   2 +-
 block/qed-table.c                                |   2 +-
 block/qed.c                                      |   2 +-
 block/raw-posix.c                                |   2 +-
 block/raw-win32.c                                |   2 +-
 block/stream.c                                   |   2 +-
 block/trace-events                               |   2 +
 blockdev.c                                       |   2 +-
 crypto/Makefile.objs                             |   2 +
 crypto/secret.c                                  |   2 +-
 crypto/tlscreds.c                                |   2 +-
 crypto/tlscredsanon.c                            |   2 +-
 crypto/tlscredsx509.c                            |   2 +-
 crypto/tlssession.c                              |   2 +-
 crypto/trace-events                              |   2 +
 docs/tracing.txt                                 |  61 ++++++--
 hw/9pfs/9p.c                                     |   2 +-
 hw/9pfs/Makefile.objs                            |   2 +
 hw/9pfs/trace-events                             |   2 +
 hw/acpi/Makefile.objs                            |   2 +
 hw/acpi/cpu.c                                    |   2 +-
 hw/acpi/memory_hotplug.c                         |   2 +-
 hw/acpi/trace-events                             |   2 +
 hw/alpha/Makefile.objs                           |   2 +
 hw/alpha/pci.c                                   |   2 +-
 hw/alpha/trace-events                            |   2 +
 hw/arm/Makefile.objs                             |   2 +
 hw/arm/trace-events                              |   2 +
 hw/arm/virt-acpi-build.c                         |   2 +-
 hw/audio/Makefile.objs                           |   3 +
 hw/audio/cs4231.c                                |   2 +-
 hw/audio/milkymist-ac97.c                        |   2 +-
 hw/audio/trace-events                            |   2 +
 hw/block/Makefile.objs                           |   2 +
 hw/block/dataplane/virtio-blk.c                  |   2 +-
 hw/block/hd-geometry.c                           |   2 +-
 hw/block/trace-events                            |   2 +
 hw/block/virtio-blk.c                            |   2 +-
 hw/char/Makefile.objs                            |   2 +
 hw/char/escc.c                                   |   2 +-
 hw/char/grlib_apbuart.c                          |   2 +-
 hw/char/lm32_juart.c                             |   2 +-
 hw/char/lm32_uart.c                              |   2 +-
 hw/char/milkymist-uart.c                         |   2 +-
 hw/char/trace-events                             |   2 +
 hw/char/virtio-console.c                         |   2 +-
 hw/char/virtio-serial-bus.c                      |   2 +-
 hw/display/Makefile.objs                         |   3 +
 hw/display/g364fb.c                              |   2 +-
 hw/display/jazz_led.c                            |   2 +-
 hw/display/milkymist-tmu2.c                      |   2 +-
 hw/display/milkymist-vgafb.c                     |   2 +-
 hw/display/qxl-render.c                          |   2 +-
 hw/display/qxl.c                                 |   2 +-
 hw/display/trace-events                          |   2 +
 hw/display/vga.c                                 |   2 +-
 hw/display/virtio-gpu-3d.c                       |   2 +-
 hw/display/virtio-gpu.c                          |   2 +-
 hw/display/vmware_vga.c                          |   2 +-
 hw/display/xenfb.c                               |   2 +-
 hw/dma/Makefile.objs                             |   2 +
 hw/dma/i8257.c                                   |   2 +-
 hw/dma/rc4030.c                                  |   2 +-
 hw/dma/sparc32_dma.c                             |   2 +-
 hw/dma/sun4m_iommu.c                             |   2 +-
 hw/dma/trace-events                              |   2 +
 hw/i386/Makefile.objs                            |   2 +
 hw/i386/trace-events                             |   6 +-
 hw/i386/x86-iommu.c                              |   2 +-
 hw/i386/xen/xen_platform.c                       |   1 +
 hw/i386/xen/xen_pvdevice.c                       |   1 +
 hw/input/Makefile.objs                           |   3 +
 hw/input/hid.c                                   |   2 +-
 hw/input/milkymist-softusb.c                     |   2 +-
 hw/input/ps2.c                                   |   2 +-
 hw/input/trace-events                            |   2 +
 hw/input/virtio-input.c                          |   2 +-
 hw/intc/Makefile.objs                            |   2 +
 hw/intc/apic.c                                   |   2 +-
 hw/intc/apic_common.c                            |   2 +-
 hw/intc/arm_gic.c                                |   2 +-
 hw/intc/arm_gicv3_cpuif.c                        |   2 +-
 hw/intc/arm_gicv3_dist.c                         |   2 +-
 hw/intc/arm_gicv3_redist.c                       |   2 +-
 hw/intc/aspeed_vic.c                             |   2 +-
 hw/intc/grlib_irqmp.c                            |   2 +-
 hw/intc/lm32_pic.c                               |   2 +-
 hw/intc/s390_flic.c                              |   2 +-
 hw/intc/s390_flic_kvm.c                          |   2 +-
 hw/intc/slavio_intctl.c                          |   2 +-
 hw/intc/trace-events                             |   2 +
 hw/intc/xics.c                                   |   2 +-
 hw/intc/xics_kvm.c                               |   2 +-
 hw/intc/xics_spapr.c                             |   2 +-
 hw/isa/Makefile.objs                             |   2 +
 hw/isa/pc87312.c                                 |   2 +-
 hw/isa/trace-events                              |   2 +
 hw/mem/Makefile.objs                             |   3 +
 hw/mem/pc-dimm.c                                 |   2 +-
 hw/mem/trace-events                              |   7 +
 hw/misc/Makefile.objs                            |   3 +
 hw/misc/aspeed_scu.c                             |   2 +-
 hw/misc/eccmemctl.c                              |   2 +-
 hw/misc/milkymist-hpdmc.c                        |   2 +-
 hw/misc/milkymist-pfpu.c                         |   2 +-
 hw/misc/slavio_misc.c                            |   2 +-
 hw/misc/trace-events                             |   2 +
 hw/net/Makefile.objs                             |   3 +
 hw/net/e1000e.c                                  |   2 +-
 hw/net/e1000e_core.c                             |   2 +-
 hw/net/e1000x_common.c                           |   2 +-
 hw/net/lance.c                                   |   2 +-
 hw/net/milkymist-minimac2.c                      |   2 +-
 hw/net/mipsnet.c                                 |   2 +-
 hw/net/net_rx_pkt.c                              |   2 +-
 hw/net/opencores_eth.c                           |   2 +-
 hw/net/pcnet-pci.c                               |   2 +-
 hw/net/pcnet.c                                   |   2 +-
 hw/net/trace-events                              |   2 +
 hw/nvram/Makefile.objs                           |   2 +
 hw/nvram/ds1225y.c                               |   2 +-
 hw/nvram/fw_cfg.c                                |   2 +-
 hw/nvram/trace-events                            |   2 +
 hw/pci/Makefile.objs                             |   2 +
 hw/pci/pci.c                                     |   2 +-
 hw/pci/pci_host.c                                |   2 +-
 hw/pci/trace-events                              |   2 +
 hw/ppc/Makefile.objs                             |   3 +
 hw/ppc/ppc.c                                     |   2 +-
 hw/ppc/prep.c                                    |   2 +-
 hw/ppc/spapr.c                                   |   2 +-
 hw/ppc/spapr_hcall.c                             |   2 +-
 hw/ppc/spapr_iommu.c                             |   2 +-
 hw/ppc/spapr_pci.c                               |   2 +-
 hw/ppc/spapr_rtas_ddw.c                          |   2 +-
 hw/ppc/trace-events                              |   2 +
 hw/s390x/Makefile.objs                           |   3 +
 hw/s390x/css.c                                   |   2 +-
 hw/s390x/trace-events                            |   2 +
 hw/s390x/virtio-ccw.c                            |   2 +-
 hw/scsi/Makefile.objs                            |   3 +
 hw/scsi/esp-pci.c                                |   2 +-
 hw/scsi/esp.c                                    |   2 +-
 hw/scsi/megasas.c                                |   2 +-
 hw/scsi/mptconfig.c                              |   2 +-
 hw/scsi/mptendian.c                              |   2 +-
 hw/scsi/mptsas.c                                 |   2 +-
 hw/scsi/scsi-bus.c                               |   2 +-
 hw/scsi/trace-events                             |   2 +
 hw/scsi/vmw_pvscsi.c                             |   2 +-
 hw/sd/Makefile.objs                              |   2 +
 hw/sd/milkymist-memcard.c                        |   2 +-
 hw/sd/trace-events                               |   2 +
 hw/sparc/Makefile.objs                           |   2 +
 hw/sparc/leon3.c                                 |   2 +-
 hw/sparc/sun4m.c                                 |   2 +-
 hw/sparc/trace-events                            |   2 +
 hw/timer/Makefile.objs                           |   2 +
 hw/timer/aspeed_timer.c                          |   2 +-
 hw/timer/grlib_gptimer.c                         |   2 +-
 hw/timer/lm32_timer.c                            |   2 +-
 hw/timer/milkymist-sysctl.c                      |   2 +-
 hw/timer/slavio_timer.c                          |   2 +-
 hw/timer/trace-events                            |   2 +
 hw/usb/Makefile.objs                             |   3 +
 hw/usb/bus.c                                     |   2 +-
 hw/usb/combined-packet.c                         |   2 +-
 hw/usb/core.c                                    |   2 +-
 hw/usb/desc.c                                    |   2 +-
 hw/usb/dev-hub.c                                 |   2 +-
 hw/usb/dev-mtp.c                                 |   2 +-
 hw/usb/dev-uas.c                                 |   2 +-
 hw/usb/hcd-ehci.c                                |   2 +-
 hw/usb/hcd-ohci.c                                |   8 +-
 hw/usb/hcd-uhci.c                                |   2 +-
 hw/usb/hcd-xhci.c                                |   2 +-
 hw/usb/host-libusb.c                             |   2 +-
 hw/usb/trace-events                              |   2 +
 hw/vfio/Makefile.objs                            |   2 +
 hw/vfio/common.c                                 |   2 +-
 hw/vfio/pci-quirks.c                             |   2 +-
 hw/vfio/pci.c                                    |   2 +-
 hw/vfio/platform.c                               |   2 +-
 hw/vfio/spapr.c                                  |   2 +-
 hw/vfio/trace-events                             |   2 +
 hw/virtio/Makefile.objs                          |   2 +
 hw/virtio/trace-events                           |   7 +
 hw/virtio/virtio-balloon.c                       |   2 +-
 hw/virtio/virtio-rng.c                           |   2 +-
 hw/virtio/virtio.c                               |   2 +-
 include/qemu/module.h                            |   2 +
 include/qom/cpu.h                                |  10 +-
 include/trace-tcg.h                              |   1 -
 include/trace.h                                  |   7 -
 io/Makefile.objs                                 |   2 +
 io/channel-buffer.c                              |   2 +-
 io/channel-command.c                             |   2 +-
 io/channel-file.c                                |   2 +-
 io/channel-socket.c                              |   2 +-
 io/channel-tls.c                                 |   2 +-
 io/channel-websock.c                             |   2 +-
 io/task.c                                        |   2 +-
 io/trace-events                                  |   6 +-
 linux-user/Makefile.objs                         |   2 +
 linux-user/signal.c                              |   2 +-
 linux-user/trace-events                          |   2 +
 migration/Makefile.objs                          |   2 +
 migration/exec.c                                 |   2 +-
 migration/fd.c                                   |   2 +-
 migration/migration.c                            |   2 +-
 migration/postcopy-ram.c                         |   2 +-
 migration/qemu-file.c                            |   2 +-
 migration/ram.c                                  |   2 +-
 migration/rdma.c                                 |   2 +-
 migration/savevm.c                               |   2 +-
 migration/socket.c                               |   2 +-
 migration/tls.c                                  |   2 +-
 migration/trace-events                           |   2 +
 migration/vmstate.c                              |   2 +-
 monitor.c                                        |  16 +-
 net/Makefile.objs                                |   2 +
 net/filter-mirror.c                              |   2 +-
 net/trace-events                                 |   2 +
 net/vhost-user.c                                 |   2 +-
 qemu-img.c                                       |   1 +
 qemu-io.c                                        |   1 +
 qemu-nbd.c                                       |   1 +
 qom/Makefile.objs                                |   2 +
 qom/cpu.c                                        |   2 +-
 qom/object.c                                     |   2 +-
 qom/trace-events                                 |   2 +
 scripts/simpletrace.py                           |   7 +-
 scripts/tracetool.py                             |  10 +-
 scripts/tracetool/__init__.py                    |  59 +++++--
 scripts/tracetool/backend/__init__.py            |  12 +-
 scripts/tracetool/backend/dtrace.py              |   6 +-
 scripts/tracetool/backend/ftrace.py              |   7 +-
 scripts/tracetool/backend/log.py                 |   6 +-
 scripts/tracetool/backend/simple.py              |  13 +-
 scripts/tracetool/backend/ust.py                 |   6 +-
 scripts/tracetool/format/__init__.py             |   4 +-
 scripts/tracetool/format/c.py                    |  48 +++++-
 scripts/tracetool/format/d.py                    |   2 +-
 scripts/tracetool/format/events_c.py             |  44 ------
 scripts/tracetool/format/events_h.py             |  60 -------
 scripts/tracetool/format/h.py                    |  51 +++++-
 scripts/tracetool/format/simpletrace_stap.py     |   4 +-
 scripts/tracetool/format/stap.py                 |   2 +-
 scripts/tracetool/format/tcg_h.py                |   2 +-
 scripts/tracetool/format/tcg_helper_c.py         |   2 +-
 scripts/tracetool/format/tcg_helper_h.py         |   2 +-
 scripts/tracetool/format/tcg_helper_wrapper_h.py |   2 +-
 scripts/tracetool/format/ust_events_c.py         |   4 +-
 scripts/tracetool/format/ust_events_h.py         |  15 +-
 stubs/trace-control.c                            |   8 +-
 target-i386/Makefile.objs                        |   2 +
 target-i386/kvm.c                                |   2 +-
 target-i386/trace-events                         |   2 +
 target-ppc/Makefile.objs                         |   2 +
 target-ppc/kvm.c                                 |   2 +-
 target-ppc/trace-events                          |   2 +
 target-s390x/Makefile.objs                       |   2 +
 target-s390x/cpu.c                               |   2 +-
 target-s390x/ioinst.c                            |   2 +-
 target-s390x/kvm.c                               |   2 +-
 target-s390x/mmu_helper.c                        |   2 +-
 target-s390x/trace-events                        |   2 +
 target-sparc/Makefile.objs                       |   2 +
 target-sparc/int32_helper.c                      |   2 +-
 target-sparc/int64_helper.c                      |   2 +-
 target-sparc/mmu_helper.c                        |   2 +-
 target-sparc/trace-events                        |   2 +
 target-sparc/win_helper.c                        |   2 +-
 trace-events                                     |  19 +--
 trace/Makefile.objs                              |  98 ++----------
 trace/control-internal.h                         |  39 ++---
 trace/control-target.c                           |  20 +--
 trace/control.c                                  | 191 +++++++++++++++++------
 trace/control.h                                  | 108 +++----------
 trace/event-internal.h                           |   7 +-
 trace/qmp.c                                      |  29 ++--
 trace/simple.c                                   |   6 +-
 trace/simple.h                                   |   6 +-
 ui/Makefile.objs                                 |   3 +
 ui/console.c                                     |   2 +-
 ui/gtk-egl.c                                     |   2 +-
 ui/gtk-gl-area.c                                 |   2 +-
 ui/gtk.c                                         |   2 +-
 ui/input.c                                       |   2 +-
 ui/spice-display.c                               |   2 +-
 ui/trace-events                                  |   2 +
 ui/vnc.c                                         |   2 +-
 util/Makefile.objs                               |   2 +
 util/buffer.c                                    |   2 +-
 util/hbitmap.c                                   |   2 +-
 util/oslib-posix.c                               |   2 +-
 util/oslib-win32.c                               |   2 +-
 util/qemu-coroutine-lock.c                       |   2 +-
 util/qemu-coroutine.c                            |   2 +-
 util/trace-events                                |  21 +++
 vl.c                                             |   2 +
 321 files changed, 1058 insertions(+), 768 deletions(-)
 create mode 100644 hw/mem/trace-events
 delete mode 100644 include/trace.h
 delete mode 100644 scripts/tracetool/format/events_c.py
 delete mode 100644 scripts/tracetool/format/events_h.py

-- 
2.7.4




reply via email to

[Prev in Thread] Current Thread [Next in Thread]