[elpa] externals/beardbolt 7ff619c375 283/323: Add some benchmarks

[ELPA Syncer]

branch: externals/beardbolt
commit 7ff619c3757824b6d592bd4d79625ed19b17a90f
Author: João Távora <joaotavora@gmail.com>
Commit: João Távora <joaotavora@gmail.com>

    Add some benchmarks
---
 Makefile                                |   5 ++
 README.md                               | 101 ++++++++++++++++++++++++++++----
 beardbolt-benchmark.el                  |  57 ++++++++++++++++++
 starters/unordered-multimap-emplace.cpp |  28 +++++++++
 starters/vector-emplace-back.cpp        |  51 ++++++++++++++++
 5 files changed, 229 insertions(+), 13 deletions(-)

diff --git a/Makefile b/Makefile
index ced3bd1655..aeb139d5d9 100644
--- a/Makefile
+++ b/Makefile
@@ -13,4 +13,9 @@ compile: $(ELCFILES)
 clean:
        find . -iname '*.elc' -exec rm {} \;
 
+benchmark: compile
+       $(EMACS) -Q -L . --batch -l beardbolt-benchmark 
starters/slow-to-process.cpp
+       $(EMACS) -Q -L . --batch -l beardbolt-benchmark 
starters/vector-emplace-back.cpp
+       $(EMACS) -Q -L . --batch -l beardbolt-benchmark 
starters/unordered-multimap-emplace.cpp
+
 .PHONY: all compile clean check
diff --git a/README.md b/README.md
index 3c3af416b4..9a7a9d03b5 100644
--- a/README.md
+++ b/README.md
@@ -1,43 +1,118 @@
+# Beardbolt
 
-# beardbolt
-
-An experimental fork of [rmsbolt](https://gitlab.com/jgkamat/rmsbolt),
+An experimental fork of [RMSbolt](https://gitlab.com/jgkamat/rmsbolt),
 itself a supercharged implementation of [godbolt
 compiler-explorer](https://github.com/mattgodbolt/compiler-explorer)
 for Emacs.
 
-beardbolt tries to make it easy to see what your compiler is doing.
+Beardbolt tries to make it easy to see what your compiler is doing.
 It does this by showing you the assembly output of a given source code
 file.  It also highlights which source code a given assembly block
 corresponds to, and vice versa.
 
-### Why rmsbolt over beardbolt
+### Why RMSbolt over Beardbolt
 
-- Supports more languages/compilers. beardbolt only C++/C clang/gcc.
+- Supports more languages/compilers. Beardbolt only C++/C clang/gcc.
 - Has good documentation and a proper API.
-- Supports more Emacs versions.  beardbolt only 28+
+- Supports more Emacs versions.  Beardbolt only 28+
 - Support compile-commands.json
 
-### Why beardbolt over rmsbolt
+### Why Beardbolt over RMSbolt
 
 - Doesn't require file to be saved.
-- Faster (2x) and more responsive (TODO: show benchmarks)
-- Less buggy (TODO: show actual rmsbolt problems)
+- 2-5x faster on typical files.  See [here][#benchmarks])
+- Less buggy (TODO: show actual RMSbolt problems)
 - Has rainbows.
-- Simpler code (half the LOC)
+- Simpler code (less than half the LOC, but )
 
 ### Installation
 
 ```sh
-cd /path/to/beardbolt-clone
+cd /path/to/beardbolt/clone
 make
 ```
 
 ```lisp
-(add-to-list 'load-path "/path/to/beardbolt-clone")
+(add-to-list 'load-path "/path/to/beardbolt/clone")
 (require 'beardbolt)
 ```
 
 ```
 M-x beardbolt-starter
 ```
+
+<a name="benchmarks"></a>
+### Benchmarks vs RMSbolt
+
+First note that Beardbolt is highly hacky/experimental and may be
+providing incorrect results, in which case most/all of the following
+benchmarks/comparisons are probably invalid.
+
+Second, a word on what "fast" means: the performance metric to
+optimize is responsiveness.  Both Beardbolt and RMSbolt continuously
+analyze the program to present a "live" view of its assembly output.
+The goal is not only to provide this service as quickly as possible,
+but also to intrude as little as possible in the user's editing.
+
+Both extensions work in a two-step fashion.  Beardbolt tries to
+optimize step 2.
+
+1. The file is saved and partially compiled by an external program
+
+   This happens asynchronously.  It might takes several seconds and
+   spin up your CPU, but it does not generally harm the UX inside
+   Emacs.
+
+2. Some Elisp processing takes place on the assembly output
+
+   This happens inside Emacs, and it's generally bad if it takes a
+   long time, because Emacs is single-threaded and has no easily
+   accessible asynchronous mechanisms for this type of work.
+
+#### Results
+
+To run the benchmarks, have both RMSbolt and Beardbolt clones
+side-by-side, then:
+
+```
+$ cd /path/to/beardbolt/clone
+$ EMACS=~/Source/Emacs/emacs/src/emacs make benchmark
+/home/capitaomorte/Source/Emacs/emacs/src/emacs -Q -L . --batch -l 
beardbolt-benchmark starters/slow-to-process.cpp
+RMSbolt timings for slow-to-process.cpp
+  samples: (1.329s 1.316s 1.338s 1.345s 1.341s)
+  average: 1.334s
+Beardbolt timings for slow-to-process.cpp
+  samples: (0.324s 0.338s 0.334s 0.334s 0.342s)
+  average: 0.334s
+/home/capitaomorte/Source/Emacs/emacs/src/emacs -Q -L . --batch -l 
beardbolt-benchmark starters/vector-emplace-back.cpp
+RMSbolt timings for vector-emplace-back.cpp
+  samples: (0.234s 0.223s 0.223s 0.240s 0.224s)
+  average: 0.229s
+Beardbolt timings for vector-emplace-back.cpp
+  samples: (0.086s 0.074s 0.073s 0.074s 0.089s)
+  average: 0.079s
+/home/capitaomorte/Source/Emacs/emacs/src/emacs -Q -L . --batch -l 
beardbolt-benchmark starters/unordered-multimap-emplace.cpp
+RMSbolt timings for unordered-multimap-emplace.cpp
+  samples: (0.534s 0.523s 0.524s 0.523s 0.529s)
+  average: 0.527s
+Beardbolt timings for unordered-multimap-emplace.cpp
+  samples: (0.103s 0.123s 0.103s 0.102s 0.118s)
+  average: 0.110s
+```
+
+This ran `beardbolt-compile` and `rmsbolt-compile` 5 times on small
+two [cppreference.com][https://cppreference.com] examples
+([1][example1], [2][example2]) as well as a known "slow" file found in
+RMSbolt's bug tracker
+([here](https://gitlab.com/jgkamat/rmsbolt/-/issues/9)).
+
+I patched `rmsbolt.el` to generate slightly less useless (for our use
+case) debug information with `-g1` instead of `g`.  This makes RMSbolt
+run faster than it normally would, but since Beardbolt also uses this
+optimization, it is important to make the benchmarks fair(er?).
+
+The results were obtained on my Thinkpad T480 running Emacs 29
+(without native compilation).
+
+[example1]: https://en.cppreference.com/w/cpp/container/vector/emplace_back
+[example2]: 
https://en.cppreference.com/w/cpp/container/unordered_multimap/emplace
diff --git a/beardbolt-benchmark.el b/beardbolt-benchmark.el
new file mode 100644
index 0000000000..c5ea454753
--- /dev/null
+++ b/beardbolt-benchmark.el
@@ -0,0 +1,57 @@
+(require 'beardbolt)
+(require 'cl-lib)
+(require 'benchmark)
+
+(defvar beardbolt-benchmark-samples nil)
+(defvar beardbolt-benchmark-rmsbolt-samples nil)
+
+(advice-add (quote beardbolt--handle-finish-compile) :around
+            (lambda (oldfun &rest args)
+              (push (benchmark-elapse (apply oldfun args))
+                    beardbolt-benchmark-samples)))
+
+(defun beardbolt-benchmark-beardbolt (repeats)
+  (cl-loop
+   repeat repeats
+   do (beardbolt-compile (beardbolt--get-lang))
+   (while (process-live-p
+           (get-buffer-process (beardbolt--compilation-buffer)))
+     (accept-process-output)))
+  (message "Beardbolt timings for %s\n  samples: %s\n  average: %.3fs"
+           (file-name-nondirectory buffer-file-name)
+           (mapcar (lambda (s) (format "%.3fs" s))
+                   (reverse beardbolt-benchmark-samples))
+           (/ (cl-reduce #'+ beardbolt-benchmark-samples)
+              (length beardbolt-benchmark-samples) 1.0)))
+
+(defun beardbolt-benchmark-rmsbolt (repeats)
+  (add-to-list 'load-path (expand-file-name "../../rmsbolt" default-directory))
+  (cond ((require 'rmsbolt nil t)
+         (advice-add (quote rmsbolt--handle-finish-compile) :around
+                     (lambda (oldfun &rest args)
+                       (push (benchmark-elapse (apply oldfun args))
+                             beardbolt-benchmark-rmsbolt-samples)))
+         (cl-loop
+          repeat repeats
+          do
+          (sit-for 0.2) ;; unconfuse RMSbolt state cleanup
+          (rmsbolt-compile)
+          (while (process-live-p
+                  (get-buffer-process (get-buffer "*rmsbolt-compilation*")))
+            (accept-process-output)))
+         (message "RMSbolt timings for %s\n  samples: %s\n  average: %.3fs"
+                  (file-name-nondirectory buffer-file-name)
+                  (mapcar (lambda (s) (format "%.3fs" s)) (reverse
+                                                           
beardbolt-benchmark-rmsbolt-samples))
+                  (/ (cl-reduce #'+ beardbolt-benchmark-rmsbolt-samples)
+                     (length beardbolt-benchmark-rmsbolt-samples) 1.0)))
+        (t
+         (message "Can't find rmsbolt in load path %s" load-path))))
+
+(with-current-buffer (find-file (car argv))
+  (beardbolt-benchmark-rmsbolt 5)
+  (beardbolt-benchmark-beardbolt 5))
+
+
+
+
diff --git a/starters/unordered-multimap-emplace.cpp 
b/starters/unordered-multimap-emplace.cpp
new file mode 100644
index 0000000000..a059875be1
--- /dev/null
+++ b/starters/unordered-multimap-emplace.cpp
@@ -0,0 +1,28 @@
+#include <iostream>
+#include <utility>
+#include <string>
+#include <unordered_map>
+
+int main()
+{
+  std::unordered_multimap<std::string, std::string> m;
+
+  // uses pair's move constructor
+  m.emplace(std::make_pair(std::string("a"), std::string("a")));
+
+  // uses pair's converting move constructor
+  m.emplace(std::make_pair("b", "abcd"));
+
+  // uses pair's template constructor
+  m.emplace("d", "ddd");
+
+  // uses pair's piecewise constructor
+  m.emplace(std::piecewise_construct,
+            std::forward_as_tuple("c"),
+            std::forward_as_tuple(10, 'c'));
+
+
+  for (const auto &p : m) {
+    std::cout << p.first << " => " << p.second << '\n';
+  }
+}
diff --git a/starters/vector-emplace-back.cpp b/starters/vector-emplace-back.cpp
new file mode 100644
index 0000000000..7ee548c05e
--- /dev/null
+++ b/starters/vector-emplace-back.cpp
@@ -0,0 +1,51 @@
+#include <vector>
+#include <string>
+#include <cassert>
+#include <iostream>
+
+struct President
+{
+  std::string name;
+  std::string country;
+  int year;
+
+  President(std::string p_name, std::string p_country, int p_year)
+    : name(std::move(p_name)), country(std::move(p_country)), year(p_year)
+  {
+    std::cout << "I am being constructed.\n";
+  }
+  President(President&& other)
+    : name(std::move(other.name)), country(std::move(other.country)), 
year(other.year)
+  {
+    std::cout << "I am being moved.\n";
+  }
+  President& operator=(const President& other) = default;
+};
+
+int main()
+{
+  std::vector<President> elections;
+  std::cout << "emplace_back:\n";
+  auto& ref = elections.emplace_back("Nelson Mandela", "South Africa", 1994);
+  assert(ref.year == 1994 && "uses a reference to the created object (C++17)");
+
+  std::vector<President> reElections;
+  std::cout << "\npush_back:\n";
+  reElections.push_back(President("Franklin Delano Roosevelt", "the USA", 
1936));
+
+  std::cout << "\nContents:\n";
+  for (President const& president: elections) {
+    std::cout << president.name << " was elected president of "
+              << president.country << " in " << president.year << ".\n";
+  }
+  for (President const& president: reElections) {
+    std::cout << president.name << " was re-elected president of "
+              << president.country << " in " << president.year << ".\n";
+  }
+}
+
+// Local Variables:
+// beardbolt-preserve-directives: t
+// beardbolt-preserve-labels: t
+// beardbolt-command: "g++ -O3"
+// End: