[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[SCM] gawk branch, master, updated. gawk-4.1.0-4748-g2ae439f3
|
From: |
Arnold Robbins |
|
Subject: |
[SCM] gawk branch, master, updated. gawk-4.1.0-4748-g2ae439f3 |
|
Date: |
Fri, 24 Jun 2022 07:13:11 -0400 (EDT) |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "gawk".
The branch, master has been updated
via 2ae439f369f23c6ab3d2e671ce614fff32e32a38 (commit)
from d43b469f0b189cb0ed7d5aa88d1834249cc5f415 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://git.sv.gnu.org/cgit/gawk.git/commit/?id=2ae439f369f23c6ab3d2e671ce614fff32e32a38
commit 2ae439f369f23c6ab3d2e671ce614fff32e32a38
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 24 14:12:56 2022 +0300
Squashed commit of the following:
commit 61f2a5e0e48e050b8f73b127202af068b8cb574b
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 24 11:08:22 2022 +0300
Update awkcard.in.
commit 2b84c068b0568b4efaf93f9f6ffeaf2105295de7
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Wed Jun 22 22:02:59 2022 +0300
A few more small doc updates.
commit ce3bb88b7e71e1185c243bcc22cdcc7dbb250988
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Wed Jun 22 10:33:08 2022 +0300
Doc updates.
commit 0c101d168279d1400d3954a84a231ee70862ab61
Merge: 78fd9d6f d43b469f
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Wed Jun 22 10:19:58 2022 +0300
Merge branch 'master' into feature/pma2
commit 78fd9d6f1f45688d32e77ed7eb8cb18f9a140ec8
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Mon Jun 20 08:48:35 2022 +0300
Add --persist option.
commit a6253b1b2f3eba98a42c8b39d4af8916187d7249
Merge: bedd2e70 03f79dcb
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Mon Jun 20 08:43:28 2022 +0300
Merge branch 'master' into feature/pma2
commit bedd2e706af48057805b5221886912545c16f193
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 17 14:40:33 2022 +0300
Fix support/ChangeLog.
commit 8da00d7e8666808ffa50ae654b5498c4ca4b7d8c
Merge: 882a6b67 f2e71851
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 17 14:39:48 2022 +0300
Merge branch 'master' into feature/pma2
commit 882a6b67547204800ca865b530e3f8ef51ebd20c
Merge: 5c689dd7 ae879ed7
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 17 14:16:36 2022 +0300
Merge branch 'master' into feature/pma2
commit 5c689dd7da7f60228a78de3710e7d4f27c3a8eb5
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 17 14:10:34 2022 +0300
Doc fixes and small code fixes.
commit e3e5bc101641ceda429d8d98b1558dcc8aad5b0b
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 17 12:33:08 2022 +0300
Updates to pma for portability.
commit c187096f20b78631fc991898e4db62c5d6a6a441
Merge: de7bb7ee 6b97a4f7
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Wed Jun 15 18:11:00 2022 +0300
Merge branch 'master' into feature/pma2
commit de7bb7ee3e6d69c6c7985f43a0f06527f438e43d
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Wed Jun 15 14:16:07 2022 +0300
Add cygwin case to m4/pma.m4.
commit d78e8b44f8b470d894b46a0ff4cafc88ce8eed29
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Tue Jun 14 22:43:52 2022 +0300
Doc updates.
commit edc9ae773547c911d38b4a6fcb71946762df1e3d
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Tue Jun 14 22:40:58 2022 +0300
Add Mac OS support to configuration machinery.
commit 37dbf039eee81652d647aa2783f4ea4cb0266f52
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Tue Jun 14 22:01:23 2022 +0300
Add --disable-pma configure option and doc for it.
commit b0774ab145fe4828e54da22e6deb850fcceafcf4
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Tue Jun 14 15:13:38 2022 +0300
Use -no-pie only for gcc and clang.
commit fdab60d8bd6d78159571d2c4e1c4d08e0684a0ab
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Mon Jun 13 22:59:00 2022 +0300
Finish doc on persistent memory.
commit 17887261a7537f2e25f6cca63e1d67a93f3a3e36
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Mon Jun 13 13:41:19 2022 +0300
Doc and code improvements.
commit 24184e361b734aff4c7c9e96b0090cfaaf351e2d
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 10 14:58:26 2022 +0300
Fix for non-pma build.
commit f10f4af373c7ace425f2af9a68f5d6f4daa52bb3
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 10 14:54:54 2022 +0300
Get persistent stuff working!
commit 920119080d877186165506ba819a548a5ea4ae85
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Fri Jun 10 10:29:53 2022 +0300
Attempt pma symbol table integration. Works in pass through mode.
commit 36ba3cb86acd0cf8b70d78391ef0b77279b57b7c
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Thu Jun 9 22:00:21 2022 +0300
Add pma support, based on env var.
diff --git a/ChangeLog b/ChangeLog
index 85101e98..31d95383 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -4,6 +4,19 @@
* main.c (deprecate_mpfr): ifdef'ed out.
(parse_args): Comment out call to deprecate_mpfr().
+2022-06-20 Arnold D. Robbins <arnold@skeeve.com>
+
+ * main.c (optab): Add --persist.
+ (parse_args): For --persist, fatal message to use the
+ environment variable. Thanks to Terence Kelly for suggesting
+ the option, since it's mentioned in his paper.
+
+2022-06-17 Arnold D. Robbins <arnold@skeeve.com>
+
+ * NEWS: Typo fix.
+ * main.c (main): Correct the return value check in the call
+ to pma_init(). Thanks to Terence Kelly for the report.
+
2022-06-15 Andrew J. Schorr <aschorr@telemetry-investments.com>
* main.c (load_environ): Init ENVIRON_node before filling
@@ -11,6 +24,37 @@
in the environment. Thanks to David Arroyo <david@aqwari.net>
for the report.
+2022-06-14 Arnold D. Robbins <arnold@skeeve.com>
+
+ * main.c (main): Set gawk_name first thing, in case we fatal out
+ on pma_init.
+ * configure.ac: Add --disable-pma option.
+
+2022-06-13 Arnold D. Robbins <arnold@skeeve.com>
+
+ * main.c (main): If persistent memory not available and
+ GAWK_PERSIST_FILE supplied, issue a warnning. Similarly, don't
+ do mtrace if using persistent malloc.
+ (version): Include PMA version in the output.
+
+2022-06-10 Arnold D. Robbins <arnold@skeeve.com>
+
+ * awk.h (using_persistent_malloc): Declare new variable
+ * main.c (main): Check for error from pma_init. Set
+ using_persistent_malloc.
+ * symbol.c (init_symbol_table): Check using_persistent_malloc
+ appropriately. Adjust the flow for it.
+ * NEWS: Updated.
+ * Makefile.am (LDADD): Add $(LDFLAGS).
+ * custom.h (pma_init): Define to 0.
+
+2022-06-09 Arnold D. Robbins <arnold@skeeve.com>
+
+ * custom.h: Deal with use/non-use of persistent malloc.
+ * main.c (main): Bracket mtrace call in #ifndef. Call pma_init()
+ first thing.
+ * configure.ac (GAWK_USE_PERSISTENT_MALLOC): Add call.
+
2022-06-06 Andrew J. Schorr <aschorr@telemetry-investments.com>
* gawkapi.h (gawk_api_t): Add new api_destroy_array hook to clear
diff --git a/Makefile.am b/Makefile.am
index 5e38aac2..4808fdb5 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -115,7 +115,8 @@ gawk_SOURCES = $(base_sources)
# Get extra libs as needed, Automake will supply LIBINTL and SOCKET_LIBS.
LDADD = support/libsupport.a \
- $(LIBSIGSEGV) $(LIBINTL) $(SOCKET_LIBS) $(LIBREADLINE) $(LIBMPFR)
+ $(LIBSIGSEGV) $(LIBINTL) $(SOCKET_LIBS) $(LIBREADLINE) $(LIBMPFR) \
+ $(LDFLAGS)
# Directory for gawk's data files. Automake supplies datadir.
pkgdatadir = $(datadir)/awk
diff --git a/Makefile.in b/Makefile.in
index 5c77b9e8..597f3a8b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -128,8 +128,9 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
$(top_srcdir)/m4/lib-prefix.m4 $(top_srcdir)/m4/libsigsegv.m4 \
$(top_srcdir)/m4/longlong.m4 $(top_srcdir)/m4/mpfr.m4 \
$(top_srcdir)/m4/nls.m4 $(top_srcdir)/m4/noreturn.m4 \
- $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \
- $(top_srcdir)/m4/readline.m4 $(top_srcdir)/m4/socket.m4 \
+ $(top_srcdir)/m4/pma.m4 $(top_srcdir)/m4/po.m4 \
+ $(top_srcdir)/m4/progtest.m4 $(top_srcdir)/m4/readline.m4 \
+ $(top_srcdir)/m4/socket.m4 \
$(top_srcdir)/m4/triplet-transformation.m4 \
$(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
@@ -159,7 +160,8 @@ gawk_LDADD = $(LDADD)
am__DEPENDENCIES_1 =
gawk_DEPENDENCIES = support/libsupport.a $(am__DEPENDENCIES_1) \
$(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
- $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1)
+ $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
+ $(am__DEPENDENCIES_1)
am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`;
am__vpath_adj = case $$p in \
$(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \
@@ -557,7 +559,8 @@ gawk_SOURCES = $(base_sources)
# Get extra libs as needed, Automake will supply LIBINTL and SOCKET_LIBS.
LDADD = support/libsupport.a \
- $(LIBSIGSEGV) $(LIBINTL) $(SOCKET_LIBS) $(LIBREADLINE) $(LIBMPFR)
+ $(LIBSIGSEGV) $(LIBINTL) $(SOCKET_LIBS) $(LIBREADLINE) $(LIBMPFR) \
+ $(LDFLAGS)
# stuff for compiling gawk/pgawk
diff --git a/NEWS b/NEWS
index abd396e7..1f7f66e6 100644
--- a/NEWS
+++ b/NEWS
@@ -44,6 +44,11 @@ for saving / restoring all of gawk's variables and arrays.
Wherever possible, details were replaced with references to the online
copy of the manual.
+9. Gawk now supports Terence Kelly's "persistent malloc" (pma),
+allowing gawk to preserve the contents of its variables and arrays
+between runs. THIS IS AN EXPERIMENTAL FEATURE! For more information,
+see the manual.
+
Changes from 5.1.1 to 5.1.2
---------------------------
diff --git a/aclocal.m4 b/aclocal.m4
index 2c000089..0891c33d 100644
--- a/aclocal.m4
+++ b/aclocal.m4
@@ -1223,6 +1223,7 @@ m4_include([m4/longlong.m4])
m4_include([m4/mpfr.m4])
m4_include([m4/nls.m4])
m4_include([m4/noreturn.m4])
+m4_include([m4/pma.m4])
m4_include([m4/po.m4])
m4_include([m4/progtest.m4])
m4_include([m4/readline.m4])
diff --git a/awk.h b/awk.h
index 732aec04..f89b0866 100644
--- a/awk.h
+++ b/awk.h
@@ -1197,6 +1197,7 @@ extern enum do_flag_values {
extern bool do_optimize;
extern int use_lc_numeric;
extern int exit_val;
+extern bool using_persistent_malloc;
#ifdef NO_LINT
#define do_lint 0
diff --git a/awklib/ChangeLog b/awklib/ChangeLog
index da14522c..eefa0957 100644
--- a/awklib/ChangeLog
+++ b/awklib/ChangeLog
@@ -1,3 +1,8 @@
+2022-06-09 Arnold D. Robbins <arnold@skeeve.com>
+
+ * Makefile.am (AM_CPPFLAGS): Add -I.../support for inclusion
+ of pma.h.
+
2021-10-27 Arnold D. Robbins <arnold@skeeve.com>
* 5.1.1: Release tar ball made.
diff --git a/awklib/Makefile.am b/awklib/Makefile.am
index 20e9edba..c00990ef 100644
--- a/awklib/Makefile.am
+++ b/awklib/Makefile.am
@@ -35,7 +35,7 @@ AWKPROG = LC_ALL=C LANG=C "$(abs_top_builddir)/gawk$(EXEEXT)"
endif
# Get config.h from the build directory and custom.h from the source directory.
-AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir)
+AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir) -I$(top_srcdir)/support
pkgdatadir = $(datadir)/awk
pkglibexecdir = $(libexecdir)/awk
diff --git a/awklib/Makefile.in b/awklib/Makefile.in
index 59ff7cdb..b72210c2 100644
--- a/awklib/Makefile.in
+++ b/awklib/Makefile.in
@@ -120,8 +120,9 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
$(top_srcdir)/m4/lib-prefix.m4 $(top_srcdir)/m4/libsigsegv.m4 \
$(top_srcdir)/m4/longlong.m4 $(top_srcdir)/m4/mpfr.m4 \
$(top_srcdir)/m4/nls.m4 $(top_srcdir)/m4/noreturn.m4 \
- $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \
- $(top_srcdir)/m4/readline.m4 $(top_srcdir)/m4/socket.m4 \
+ $(top_srcdir)/m4/pma.m4 $(top_srcdir)/m4/po.m4 \
+ $(top_srcdir)/m4/progtest.m4 $(top_srcdir)/m4/readline.m4 \
+ $(top_srcdir)/m4/socket.m4 \
$(top_srcdir)/m4/triplet-transformation.m4 \
$(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
@@ -341,7 +342,7 @@ EXTRA_DIST = ChangeLog ChangeLog.0 ChangeLog.1 \
@TEST_CROSS_COMPILE_TRUE@AWKPROG = LC_ALL=C LANG=C awk$(EXEEXT)
# Get config.h from the build directory and custom.h from the source directory.
-AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir)
+AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir) -I$(top_srcdir)/support
AUXAWK = passwd.awk group.awk
nodist_grcat_SOURCES = grcat.c
nodist_pwcat_SOURCES = pwcat.c
diff --git a/configh.in b/configh.in
index 885dd006..283dcba2 100644
--- a/configh.in
+++ b/configh.in
@@ -406,6 +406,9 @@
/* Define to 1 if the character set is EBCDIC */
#undef USE_EBCDIC
+/* Define to 1 if we can use the pma allocator */
+#undef USE_PERSISTENT_MALLOC
+
/* Enable extensions on AIX 3, Interix. */
#ifndef _ALL_SOURCE
# undef _ALL_SOURCE
diff --git a/configure b/configure
index 648eabd0..f9e8affe 100755
--- a/configure
+++ b/configure
@@ -661,6 +661,8 @@ LIBREADLINE
SOCKET_LIBS
ENABLE_EXTENSIONS_FALSE
ENABLE_EXTENSIONS_TRUE
+USE_PERSISTENT_MALLOC_FALSE
+USE_PERSISTENT_MALLOC_TRUE
LIBSIGSEGV_PREFIX
LTLIBSIGSEGV
LIBSIGSEGV
@@ -799,6 +801,7 @@ enable_silent_rules
enable_lint
enable_builtin_intdiv0
enable_mpfr
+enable_pma
enable_versioned_extension_dir
enable_dependency_tracking
enable_largefile
@@ -1456,6 +1459,8 @@ Optional Features:
--enable-builtin-intdiv0
enable built-in intdiv0 function
--disable-mpfr do not check for MPFR
+ --disable-pma do not build gawk with the persistent memory
+ allocator
--enable-versioned-extension-dir
use a versioned directory for extensions
--enable-dependency-tracking
@@ -3661,6 +3666,18 @@ then :
fi
+SKIP_PERSIST_MALLOC=no
+# Check whether --enable-pma was given.
+if test ${enable_pma+y}
+then :
+ enableval=$enable_pma; if test "$enableval" = no
+ then
+ SKIP_PERSIST_MALLOC=yes
+ fi
+
+fi
+
+
EXTENSIONDIR=
# Check whether --enable-versioned-extension-dir was given.
if test ${enable_versioned_extension_dir+y}
@@ -12518,6 +12535,69 @@ printf "%s\n" "#define HAVE_MBRTOWC 1" >>confdefs.h
fi
+
+use_persistent_malloc=no
+if test "$SKIP_PERSIST_MALLOC" = no
+then
+ ac_fn_c_check_func "$LINENO" "mmap" "ac_cv_func_mmap"
+if test "x$ac_cv_func_mmap" = xyes
+then :
+
+fi
+
+ ac_fn_c_check_func "$LINENO" "munmap" "ac_cv_func_munmap"
+if test "x$ac_cv_func_munmap" = xyes
+then :
+
+fi
+
+ if test $ac_cv_func_mmap = yes && test $ac_cv_func_munmap = yes
+ then
+
+printf "%s\n" "#define USE_PERSISTENT_MALLOC 1" >>confdefs.h
+
+ use_persistent_malloc=yes
+ case $host_os in
+ linux-*)
+ case $CC in
+ gcc | clang)
+ LDFLAGS="${LDFLAGS} -no-pie"
+ export LDFLAGS
+ ;;
+ *)
+ # tinycc and pcc don't support -no-pie flag
+ # their executables are non-PIE automatically
+ # so no need to do anything
+ ;;
+ esac
+ ;;
+ *darwin*)
+ LDFLAGS="${LDFLAGS} -Xlinker -no_pie"
+ export LDFLAGS
+ ;;
+ *cygwin* | *CYGWIN*)
+ true # nothing do, Cygwin exes are not PIE
+ ;;
+ # Other OS's go here...
+ *)
+ # For now, play it safe
+ use_persistent_malloc=no
+ ;;
+ esac
+ else
+ use_persistent_malloc=no
+ fi
+fi
+ if test "$use_persistent_malloc" = "yes"; then
+ USE_PERSISTENT_MALLOC_TRUE=
+ USE_PERSISTENT_MALLOC_FALSE='#'
+else
+ USE_PERSISTENT_MALLOC_TRUE='#'
+ USE_PERSISTENT_MALLOC_FALSE=
+fi
+
+
+
# Check whether --enable-extensions was given.
if test ${enable_extensions+y}
then :
@@ -13832,6 +13912,10 @@ if test -z "${TEST_CROSS_COMPILE_TRUE}" && test -z
"${TEST_CROSS_COMPILE_FALSE}"
as_fn_error $? "conditional \"TEST_CROSS_COMPILE\" was never defined.
Usually this means the macro was only invoked conditionally." "$LINENO" 5
fi
+if test -z "${USE_PERSISTENT_MALLOC_TRUE}" && test -z
"${USE_PERSISTENT_MALLOC_FALSE}"; then
+ as_fn_error $? "conditional \"USE_PERSISTENT_MALLOC\" was never defined.
+Usually this means the macro was only invoked conditionally." "$LINENO" 5
+fi
if test -z "${ENABLE_EXTENSIONS_TRUE}" && test -z
"${ENABLE_EXTENSIONS_FALSE}"; then
as_fn_error $? "conditional \"ENABLE_EXTENSIONS\" was never defined.
Usually this means the macro was only invoked conditionally." "$LINENO" 5
diff --git a/configure.ac b/configure.ac
index 38197101..80cee254 100644
--- a/configure.ac
+++ b/configure.ac
@@ -74,6 +74,15 @@ AC_ARG_ENABLE([mpfr],
fi
)
+SKIP_PERSIST_MALLOC=no
+AC_ARG_ENABLE([pma],
+ [AS_HELP_STRING([--disable-pma],[do not build gawk with the persistent
memory allocator])],
+ if test "$enableval" = no
+ then
+ SKIP_PERSIST_MALLOC=yes
+ fi
+)
+
EXTENSIONDIR=
AC_ARG_ENABLE([versioned-extension-dir],
[AS_HELP_STRING([--enable-versioned-extension-dir], [use a versioned
directory for extensions])],
@@ -317,6 +326,9 @@ AC_CHECK_FUNCS(__etoa_l atexit btowc fmod gai_strerror \
dnl this check is for both mbrtowc and the mbstate_t type, which is good
AC_FUNC_MBRTOWC
+dnl check if we can use the persistent memory allocator
+GAWK_USE_PERSISTENT_MALLOC
+
dnl check for dynamic linking
dnl This is known to be very primitive
AC_ARG_ENABLE([extensions],
diff --git a/custom.h b/custom.h
index afd323b9..67e51df3 100644
--- a/custom.h
+++ b/custom.h
@@ -113,3 +113,16 @@ typedef unsigned long long uint_fast64_t;
#define ximalloc xmalloc
#include "mbsupport.h" /* defines stuff for DJGPP to fake MBS */
+
+#ifdef USE_PERSISTENT_MALLOC
+#include <stdlib.h>
+#include "pma.h"
+#define malloc pma_malloc
+#define calloc pma_calloc
+#define realloc pma_realloc
+#define free pma_free
+#else /* ! USE_PERSISTENT_MALLOC */
+#define pma_init(verbose, file) 0
+#define pma_get_root() NULL
+#define pma_set_root(rootptr) /* nothing */
+#endif /* ! USE_PERSISTENT_MALLOC */
diff --git a/doc/ChangeLog b/doc/ChangeLog
index eea6a919..926763b1 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,8 +1,37 @@
+2022-06-24 Arnold D. Robbins <arnold@skeeve.com>
+
+ * awkcard.in (Environment Variables): Shorten text and add
+ GAWK_PERSIST_FILE description.
+
+2022-06-22 Arnold D. Robbins <arnold@skeeve.com>
+
+ * gawktexi.in (Persistent Memory): Additional updates.
+
2022-06-20 Arnold D. Robbins <arnold@skeeve.com>
* gawk.1: MPFR is now "on parole" instead of deprecated.
* gawktexi.in: Ditto. Update copyright year.
+2022-06-17 Arnold D. Robbins <arnold@skeeve.com>
+
+ * gawk.1: Typo fix.
+ * wordlist, wordlist2, wordlist5: Updated.
+ * gawktexi.in (Persistent Memory): Updated after comments from
+ Terence Kelly.
+
+2022-06-14 Arnold D. Robbins <arnold@skeeve.com>
+
+ * gawktexi.in (Persistent Memory): Document --disable-pma.
+ Document that Mac OS works.
+
+2022-06-13 Arnold D. Robbins <arnold@skeeve.com>
+
+ * gawktexi.in (Persistent Memory): New section.
+
+2022-06-10 Arnold D. Robbins <arnold@skeeve.com>
+
+ * gawk.1: Document GAWK_PERSIST_FILE.
+
2022-06-06 Andrew J. Schorr <aschorr@telemetry-investments.com>
* gawktexi.in (Array Functions): Add new function destroy_array.
diff --git a/doc/Makefile.in b/doc/Makefile.in
index aa9f18e7..34212f65 100644
--- a/doc/Makefile.in
+++ b/doc/Makefile.in
@@ -122,8 +122,9 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
$(top_srcdir)/m4/lib-prefix.m4 $(top_srcdir)/m4/libsigsegv.m4 \
$(top_srcdir)/m4/longlong.m4 $(top_srcdir)/m4/mpfr.m4 \
$(top_srcdir)/m4/nls.m4 $(top_srcdir)/m4/noreturn.m4 \
- $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \
- $(top_srcdir)/m4/readline.m4 $(top_srcdir)/m4/socket.m4 \
+ $(top_srcdir)/m4/pma.m4 $(top_srcdir)/m4/po.m4 \
+ $(top_srcdir)/m4/progtest.m4 $(top_srcdir)/m4/readline.m4 \
+ $(top_srcdir)/m4/socket.m4 \
$(top_srcdir)/m4/triplet-transformation.m4 \
$(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
diff --git a/doc/awkcard.in b/doc/awkcard.in
index 9c1095de..64adab76 100644
--- a/doc/awkcard.in
+++ b/doc/awkcard.in
@@ -2,7 +2,7 @@
.\"
.\" Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
.\" 2005, 2007, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018,
-.\" 2019, 2020, 2021
+.\" 2019, 2020, 2021, 2022
.\" Free Software Foundation, Inc.
.\"
.\" Permission is granted to make and distribute verbatim copies of
@@ -100,7 +100,7 @@ Variables 5\*(CX
\*(CD
.SL
.nf
-\*(FRCopyright \(co 1996\(en2005, 2007, 2009\(en2021
+\*(FRCopyright \(co 1996\(en2005, 2007, 2009\(en2022
Free Software Foundation, Inc.
.nf
.BT
@@ -1154,15 +1154,11 @@ lp8 lp8 lp8 lp8.
.\" --- Environment Variables
.ES
.fi
-\*(CDThe environment variable \*(FCAWKPATH\fP specifies a search path to use
+\*(CD\*(FCAWKPATH\fP specifies a search path to use
when finding source files named with the \*(FC\-f\fP and \*(FC\-i\fP
options.
-The default path is
-\*(FC".:/usr/local/share/awk"\*(FR.
-If a source file name contains a ``/'' character,
-no path search is performed.
.sp .5
-The variable \*(FCAWKLIBPATH\fP
+\*(FCAWKLIBPATH\fP
specifies the search path for dynamic extensions to use
with \*(FC@load\fP and the \*(FC\-l\fP option.
.sp .5
@@ -1171,11 +1167,12 @@ For socket communication,
controls the number of connection retries, and
\*(FCGAWK_MSEC_SLEEP\fP controls
the interval between retries.
-The interval is in milliseconds. On systems that do not support
-\*(FIusleep\fP(3),
-the value is rounded up to an integral number of seconds.
+The interval is in milliseconds.
.sp .5
-The value of \*(FCGAWK_READ_TIMEOUT\fP specifies the time, in milliseconds,
+\*(FCGAWK_PERSIST_FILE\fP specifies a backing store file to use
+for persistent storage. See the manual for details.
+.sp .5
+\*(FCGAWK_READ_TIMEOUT\fP specifies the time, in milliseconds,
for \*(GK to
wait for input before returning with an error.
.sp .5
@@ -2014,7 +2011,7 @@ maintains it.\*(CX
.ES
.fi
\*(CDCopyright \(co 1996\(en2005,
-2007, 2009\(en2021 Free Software Foundation, Inc.
+2007, 2009\(en2022 Free Software Foundation, Inc.
.sp .5
Permission is granted to make and distribute verbatim copies of this
reference card provided the copyright notice and this permission notice
diff --git a/doc/gawk.1 b/doc/gawk.1
index 8e7b8794..cdb05c49 100644
--- a/doc/gawk.1
+++ b/doc/gawk.1
@@ -13,7 +13,8 @@
. if \w'\(rq' .ds rq "\(rq
. \}
.\}
-.TH GAWK 1 "Jun 08 2022" "Free Software Foundation" "Utility Commands"
+
+.TH GAWK 1 "Jun 09 2022" "Free Software Foundation" "Utility Commands"
.SH NAME
gawk \- pattern scanning and processing language
.SH SYNOPSIS
@@ -2296,6 +2297,13 @@ and
options.
.PP
The
+.B GAWK_PERSIST_FILE
+environment variable, if present, specifies a file to use as
+the backing store for persistent memory.
+.IR "This is an experimental feature" .
+See \*(EP for the details.
+.PP
+The
.B GAWK_READ_TIMEOUT
environment variable can be used to specify a timeout
in milliseconds for reading input from a terminal, pipe
diff --git a/doc/gawk.info b/doc/gawk.info
index 4d87e3e1..aa5c0189 100644
--- a/doc/gawk.info
+++ b/doc/gawk.info
@@ -504,6 +504,7 @@ in (a) below. A copy of the license is included in the
section entitled
* TCP/IP Networking:: Using 'gawk' for network
programming.
* Profiling:: Profiling your 'awk' programs.
+* Persistent Memory:: Preserving data between runs.
* Extension Philosophy:: What should be built-in and what
should not.
* Advanced Features Summary:: Summary of advanced features.
@@ -21048,8 +21049,11 @@ nondecimal numbers in input data, not just in 'awk'
programs. Then,
'gawk''s special features for sorting arrays are presented. Next,
two-way I/O, discussed briefly in earlier parts of this Info file, is
described in full detail, along with the basics of TCP/IP networking.
-Finally, we see how 'gawk' can "profile" an 'awk' program, making it
-possible to tune it for performance.
+We then see how 'gawk' can "profile" an 'awk' program, making it
+possible to tune it for performance. Next, we present an experimental
+feature that allows you to preserve the values of 'awk' variables and
+arrays between runs of 'gawk'. Finally, we discuss the philosophy
+behind 'gawk''s extension mechanism.
Additional advanced features are discussed in separate major nodes of
their own:
@@ -21076,6 +21080,7 @@ their own:
* Two-way I/O:: Two-way communications with another process.
* TCP/IP Networking:: Using 'gawk' for network programming.
* Profiling:: Profiling your 'awk' programs.
+* Persistent Memory:: Preserving data between runs.
* Extension Philosophy:: What should be built-in and what should not.
* Advanced Features Summary:: Summary of advanced features.
@@ -21854,7 +21859,7 @@ well as extensive examples.
this includes any web service whose URL starts with 'https://'.
-File: gawk.info, Node: Profiling, Next: Extension Philosophy, Prev: TCP/IP
Networking, Up: Advanced Features
+File: gawk.info, Node: Profiling, Next: Persistent Memory, Prev: TCP/IP
Networking, Up: Advanced Features
12.6 Profiling Your 'awk' Programs
==================================
@@ -22116,9 +22121,142 @@ numeric constants; if you used an octal or
hexadecimal value in your
source code, it will appear that way in the output.
-File: gawk.info, Node: Extension Philosophy, Next: Advanced Features
Summary, Prev: Profiling, Up: Advanced Features
+File: gawk.info, Node: Persistent Memory, Next: Extension Philosophy, Prev:
Profiling, Up: Advanced Features
-12.7 Builtin Features versus Extensions
+12.7 Preserving Data Between Runs
+=================================
+
+Starting with version 5.2, 'gawk' supports "persistent memory". This
+experimental feature stores the values of all of 'gawk''s variables and
+arrays in a persistent heap, which resides in a file in the filesystem.
+When persistent memory is not in use (the normal case), 'gawk''s data
+resides in ephemeral system memory.
+
+ Persistent memory is enabled on systems supporting the 'mmap()' and
+'munmap()' system calls. 'gawk' must be compiled as a non-PIE (Position
+Independent Executable) binary, since the persistent store ends up
+holding pointers to functions held within the 'gawk' executable. This
+also means that to use the persistent memory, you must use the same
+'gawk' executable from run to run.
+
+ As of this writing, persistent memory has only been tested on
+GNU/Linux and Mac OS systems. On others, your mileage may vary, and/or
+you may need to investigate how to make it work for you.
+
+ To use persistent memory, follow these steps:
+
+ 1. Create a new, empty sparse file of the desired size. For example,
+ four gigabytes. On a GNU/Linux system, you can use the 'truncate'
+ command:
+
+ $ truncate -s 4G data.pma
+
+ 2. Provide the path to the data file in the 'GAWK_PERSIST_FILE'
+ environment variable. This is best done by placing the value in
+ the environment just for the run of 'gawk', like so:
+
+ $ GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN { print ++i }'
+ 1
+
+ 3. Use the same data file in subsequent runs to use the preserved data
+ values:
+
+ $ GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN { print ++i }'
+ 2
+ $ GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN { print ++i }'
+ 3
+
+ As shown, in subsequent runs using the same data file, the values
+ of 'gawk''s variables are preserved.
+
+ Interestingly, the program that you execute need not be the same from
+run to run; the persistent store only maintains the values of variables
+and arrays, not the totality of 'gawk''s internal state. This lets you
+share data between unrelated programs, eliminating the need for scripts
+to communicate via text files.
+
+ Terence Kelly, the author of the persistent memory allocator 'gawk'
+uses, provides the following advice about the backing file:
+
+ Regarding backing file size, I recommend making it far larger than
+ all of the data that will ever reside in it, assuming that the file
+ system supports sparse files. The "pay only for what you use"
+ aspect of sparse files ensures that the actual storage resource
+ footprint of the backing file will meet the application's needs but
+ will be as small as possible. If the file system does _not_
+ support sparse files, there's a dilemma: Making the backing file
+ too large is wasteful, but making it too small risks memory
+ exhaustion, i.e., 'pma_malloc()' returns 'NULL'. But persistent
+ 'gawk' should still work even without sparse files.
+
+ You can disable the use of the persistent memory allocator in 'gawk'
+with the '--disable-pma' option to the 'configure' command at the time
+that you build 'gawk' (*note Unix Installation::).
+
+ Here are articles and web links that provide more information about
+persistent memory and why it's useful in a scripting language like
+'gawk'.
+
+<https://web.eecs.umich.edu/~tpkelly/pma/>
+ This is the canonical source for Terence Kelly's Persistent Memory
+ Allocator (PMA). Kelly may be reached directly at any of the
+ following email addresses: <tpkelly@acm.org>,
+ <tpkelly@cs.princeton.edu>, or <tpkelly@eecs.umich.edu>.
+
+'Persistent Memory Allocation'
+ Terence Kelly, Zi Fan Tan, Jianan Li, and Haris Volos, ACM 'Queue'
+ magazine, Vol. 20 No. 2 (March/April 2022), PDF
+ (https://dl.acm.org/doi/pdf/10.1145/3534855), HTML
+ (https://queue.acm.org/detail.cfm?id=3534855). This paper explains
+ the design of the PMA allocator used in persistent 'gawk'.
+
+'Persistent Scripting'
+ Zi Fan Tan, Jianan Li, Haris Volos, and Terence Kelly, Non-Volatile
+ Memory Workshop (NVMW) 2022, <http://nvmw.ucsd.edu/program/>. This
+ paper motivates and describes a research prototype of persistent
+ 'gawk' and presents performance evaluations on Intel Optane
+ non-volatile memory; note that the interface differs slightly.
+
+'Persistent Memory Programming on Conventional Hardware'
+ Terence Kelly, ACM 'Queue' magazine Vol. 17 No. 4 (July/Aug
+ 2019), PDF (https://dl.acm.org/doi/pdf/10.1145/3358955.3358957),
+ HTML (https://queue.acm.org/detail.cfm?id=3358957). This paper
+ describes simple techniques for persistent memory for C/C++ code on
+ conventional computers that lack non-volatile memory hardware.
+
+'Is Persistent Memory Persistent?'
+ Terence Kelly, ACM 'Queue' magazine Vol. 18 No. 2 (March/April
+ 2020), PDF (https://dl.acm.org/doi/pdf/10.1145/3400899.3400902),
+ HTML (https://queue.acm.org/detail.cfm?id=3400902). This paper
+ describes a simple and robust testbed for testing software against
+ real power failures.
+
+'Crashproofing the Original NoSQL Key/Value Store'
+ Terence Kelly, ACM 'Queue' magazine Vol. 19 No. 4 (July/Aug
+ 2021), PDF (https://dl.acm.org/doi/pdf/10.1145/3487019.3487353),
+ HTML (https://queue.acm.org/detail.cfm?id=3487353). This paper
+ describes a crash-tolerance feature added to GNU DBM' ('gdbm').
+
+ When Terence Kelly published his papers, his collaborators produced a
+prototype integration of PMA with 'gawk'. That version used a
+(mandatory!) option '--persist=FILE' to specify the file for storing
+the persistent heap. If this option is given to 'gawk', it produces a
+fatal error message instructing the user to use the 'GAWK_PERSIST_FILE'
+environment variable instead. Except for this paragraph, that option is
+otherwise undocumented.
+
+ As noted earlier, support for persistent memory is _experimental_.
+If it becomes burdensome,(1) then the feature will be removed.
+
+ ---------- Footnotes ----------
+
+ (1) Meaning, there are too many bug reports, or too many strange
+differences in behavior from when 'gawk' is run normally.
+
+
+File: gawk.info, Node: Extension Philosophy, Next: Advanced Features
Summary, Prev: Persistent Memory, Up: Advanced Features
+
+12.8 Builtin Features versus Extensions
=======================================
As this and subsequent major nodes show, 'gawk' has a large number of
@@ -22154,7 +22292,7 @@ or need.
File: gawk.info, Node: Advanced Features Summary, Prev: Extension
Philosophy, Up: Advanced Features
-12.8 Summary
+12.9 Summary
============
* The '--non-decimal-data' option causes 'gawk' to treat octal- and
@@ -22192,6 +22330,10 @@ File: gawk.info, Node: Advanced Features Summary,
Prev: Extension Philosophy,
* You can also just "pretty-print" the program.
+ * Persistent memory allows you to preserve the values of variables
+ and arrays between runs of 'gawk'. This feature is currently
+ experimental.
+
* New features should be developed using the extension mechanism if
possible; they should be added to the core interpreter only as a
last resort.
@@ -38852,615 +38994,617 @@ Index
Tag Table:
Node: Top1200
-Node: Foreword345720
-Node: Foreword450162
-Node: Preface51694
-Ref: Preface-Footnote-154553
-Ref: Preface-Footnote-254662
-Ref: Preface-Footnote-354896
-Node: History55038
-Node: Names57390
-Ref: Names-Footnote-158494
-Node: This Manual58641
-Ref: This Manual-Footnote-165280
-Node: Conventions65380
-Node: Manual History67749
-Ref: Manual History-Footnote-170746
-Ref: Manual History-Footnote-270787
-Node: How To Contribute70861
-Node: Acknowledgments71783
-Node: Getting Started76720
-Node: Running gawk79159
-Node: One-shot80349
-Node: Read Terminal81612
-Node: Long83605
-Node: Executable Scripts85118
-Ref: Executable Scripts-Footnote-187751
-Node: Comments87854
-Node: Quoting90338
-Node: DOS Quoting95864
-Node: Sample Data Files97920
-Node: Very Simple100515
-Node: Two Rules106617
-Node: More Complex108502
-Node: Statements/Lines110834
-Ref: Statements/Lines-Footnote-1115318
-Node: Other Features115583
-Node: When116519
-Ref: When-Footnote-1118273
-Node: Intro Summary118338
-Node: Invoking Gawk119222
-Node: Command Line120736
-Node: Options121534
-Ref: Options-Footnote-1139783
-Ref: Options-Footnote-2140014
-Node: Other Arguments140039
-Node: Naming Standard Input144050
-Node: Environment Variables145260
-Node: AWKPATH Variable145818
-Ref: AWKPATH Variable-Footnote-1149230
-Ref: AWKPATH Variable-Footnote-2149264
-Node: AWKLIBPATH Variable149635
-Ref: AWKLIBPATH Variable-Footnote-1151332
-Node: Other Environment Variables151707
-Node: Exit Status155659
-Node: Include Files156336
-Node: Loading Shared Libraries160026
-Node: Obsolete161454
-Node: Undocumented162074
-Node: Invoking Summary162371
-Node: Regexp165212
-Node: Regexp Usage166666
-Node: Escape Sequences168703
-Node: Regexp Operators174945
-Node: Regexp Operator Details175430
-Ref: Regexp Operator Details-Footnote-1182794
-Node: Interval Expressions182941
-Ref: Interval Expressions-Footnote-1185116
-Node: Bracket Expressions185214
-Ref: table-char-classes187690
-Node: Leftmost Longest191017
-Node: Computed Regexps192320
-Node: GNU Regexp Operators195747
-Node: Case-sensitivity199425
-Ref: Case-sensitivity-Footnote-1202291
-Ref: Case-sensitivity-Footnote-2202526
-Node: Regexp Summary202634
-Node: Reading Files204100
-Node: Records206369
-Node: awk split records207444
-Node: gawk split records212144
-Ref: gawk split records-Footnote-1217218
-Node: Fields217255
-Node: Nonconstant Fields219996
-Ref: Nonconstant Fields-Footnote-1222232
-Node: Changing Fields222436
-Node: Field Separators228467
-Node: Default Field Splitting231165
-Node: Regexp Field Splitting232283
-Node: Single Character Fields235960
-Node: Command Line Field Separator237020
-Node: Full Line Fields240238
-Ref: Full Line Fields-Footnote-1241760
-Ref: Full Line Fields-Footnote-2241806
-Node: Field Splitting Summary241907
-Node: Constant Size243981
-Node: Fixed width data244713
-Node: Skipping intervening248180
-Node: Allowing trailing data248978
-Node: Fields with fixed data250015
-Node: Splitting By Content251533
-Ref: Splitting By Content-Footnote-1255369
-Node: More CSV255532
-Node: FS versus FPAT257147
-Node: Testing field creation258307
-Node: Multiple Line259932
-Node: Getline266209
-Node: Plain Getline268678
-Node: Getline/Variable271251
-Node: Getline/File272402
-Node: Getline/Variable/File273790
-Ref: Getline/Variable/File-Footnote-1275395
-Node: Getline/Pipe275483
-Node: Getline/Variable/Pipe278187
-Node: Getline/Coprocess279322
-Node: Getline/Variable/Coprocess280589
-Node: Getline Notes281331
-Node: Getline Summary284128
-Ref: table-getline-variants284552
-Node: Read Timeout285301
-Ref: Read Timeout-Footnote-1289217
-Node: Retrying Input289275
-Node: Command-line directories290474
-Node: Input Summary291380
-Node: Input Exercises294552
-Node: Printing294986
-Node: Print296820
-Node: Print Examples298277
-Node: Output Separators301057
-Node: OFMT303074
-Node: Printf304430
-Node: Basic Printf305215
-Node: Control Letters306789
-Node: Format Modifiers311951
-Node: Printf Examples317966
-Node: Redirection320452
-Node: Special FD327293
-Ref: Special FD-Footnote-1330461
-Node: Special Files330535
-Node: Other Inherited Files331152
-Node: Special Network332153
-Node: Special Caveats333013
-Node: Close Files And Pipes333962
-Ref: table-close-pipe-return-values340869
-Ref: Close Files And Pipes-Footnote-1341683
-Ref: Close Files And Pipes-Footnote-2341831
-Node: Nonfatal341983
-Node: Output Summary344321
-Node: Output Exercises345543
-Node: Expressions346222
-Node: Values347410
-Node: Constants348088
-Node: Scalar Constants348779
-Ref: Scalar Constants-Footnote-1351289
-Node: Nondecimal-numbers351539
-Node: Regexp Constants354540
-Node: Using Constant Regexps355066
-Node: Standard Regexp Constants355688
-Node: Strong Regexp Constants358876
-Node: Variables362591
-Node: Using Variables363248
-Node: Assignment Options365158
-Node: Conversion367629
-Node: Strings And Numbers368153
-Ref: Strings And Numbers-Footnote-1371216
-Node: Locale influences conversions371325
-Ref: table-locale-affects374083
-Node: All Operators374702
-Node: Arithmetic Ops375331
-Node: Concatenation378047
-Ref: Concatenation-Footnote-1380894
-Node: Assignment Ops381001
-Ref: table-assign-ops385992
-Node: Increment Ops387306
-Node: Truth Values and Conditions390766
-Node: Truth Values391840
-Node: Typing and Comparison392888
-Node: Variable Typing393708
-Ref: Variable Typing-Footnote-1400171
-Ref: Variable Typing-Footnote-2400243
-Node: Comparison Operators400320
-Ref: table-relational-ops400739
-Node: POSIX String Comparison404235
-Ref: POSIX String Comparison-Footnote-1405930
-Ref: POSIX String Comparison-Footnote-2406069
-Node: Boolean Ops406153
-Ref: Boolean Ops-Footnote-1410635
-Node: Conditional Exp410727
-Node: Function Calls412463
-Node: Precedence416340
-Node: Locales419999
-Node: Expressions Summary421631
-Node: Patterns and Actions424204
-Node: Pattern Overview425324
-Node: Regexp Patterns427001
-Node: Expression Patterns427543
-Node: Ranges431324
-Node: BEGIN/END434432
-Node: Using BEGIN/END435193
-Ref: Using BEGIN/END-Footnote-1437947
-Node: I/O And BEGIN/END438053
-Node: BEGINFILE/ENDFILE440366
-Node: Empty443597
-Node: Using Shell Variables443914
-Node: Action Overview446188
-Node: Statements448513
-Node: If Statement450361
-Node: While Statement451856
-Node: Do Statement453884
-Node: For Statement455032
-Node: Switch Statement458287
-Node: Break Statement460728
-Node: Continue Statement462820
-Node: Next Statement464647
-Node: Nextfile Statement467030
-Node: Exit Statement469719
-Node: Built-in Variables472122
-Node: User-modified473255
-Node: Auto-set481022
-Ref: Auto-set-Footnote-1497829
-Ref: Auto-set-Footnote-2498035
-Node: ARGC and ARGV498091
-Node: Pattern Action Summary502304
-Node: Arrays504734
-Node: Array Basics506063
-Node: Array Intro506907
-Ref: figure-array-elements508882
-Ref: Array Intro-Footnote-1511587
-Node: Reference to Elements511715
-Node: Assigning Elements514179
-Node: Array Example514670
-Node: Scanning an Array516624
-Node: Controlling Scanning519646
-Ref: Controlling Scanning-Footnote-1526102
-Node: Numeric Array Subscripts526418
-Node: Uninitialized Subscripts528602
-Node: Delete530221
-Ref: Delete-Footnote-1532973
-Node: Multidimensional533030
-Node: Multiscanning536125
-Node: Arrays of Arrays537716
-Node: Arrays Summary542484
-Node: Functions544577
-Node: Built-in545615
-Node: Calling Built-in546768
-Node: Boolean Functions548764
-Node: Numeric Functions549318
-Ref: Numeric Functions-Footnote-1553345
-Ref: Numeric Functions-Footnote-2553993
-Ref: Numeric Functions-Footnote-3554041
-Node: String Functions554313
-Ref: String Functions-Footnote-1579155
-Ref: String Functions-Footnote-2579283
-Ref: String Functions-Footnote-3579531
-Node: Gory Details579618
-Ref: table-sub-escapes581409
-Ref: table-sub-proposed582929
-Ref: table-posix-sub584293
-Ref: table-gensub-escapes585835
-Ref: Gory Details-Footnote-1586659
-Node: I/O Functions586813
-Ref: table-system-return-values593267
-Ref: I/O Functions-Footnote-1595348
-Ref: I/O Functions-Footnote-2595496
-Node: Time Functions595616
-Ref: Time Functions-Footnote-1606287
-Ref: Time Functions-Footnote-2606355
-Ref: Time Functions-Footnote-3606513
-Ref: Time Functions-Footnote-4606624
-Ref: Time Functions-Footnote-5606736
-Ref: Time Functions-Footnote-6606963
-Node: Bitwise Functions607229
-Ref: table-bitwise-ops607823
-Ref: Bitwise Functions-Footnote-1613887
-Ref: Bitwise Functions-Footnote-2614060
-Node: Type Functions614251
-Node: I18N Functions617671
-Node: User-defined619322
-Node: Definition Syntax620134
-Ref: Definition Syntax-Footnote-1625828
-Node: Function Example625899
-Ref: Function Example-Footnote-1628821
-Node: Function Calling628843
-Node: Calling A Function629431
-Node: Variable Scope630389
-Node: Pass By Value/Reference633383
-Node: Function Caveats636027
-Ref: Function Caveats-Footnote-1638074
-Node: Return Statement638194
-Node: Dynamic Typing641173
-Node: Indirect Calls642103
-Node: Functions Summary653030
-Node: Library Functions655735
-Ref: Library Functions-Footnote-1659342
-Ref: Library Functions-Footnote-2659485
-Node: Library Names659656
-Ref: Library Names-Footnote-1663323
-Ref: Library Names-Footnote-2663546
-Node: General Functions663632
-Node: Strtonum Function664814
-Node: Assert Function667836
-Node: Round Function671162
-Node: Cliff Random Function672702
-Node: Ordinal Functions673718
-Ref: Ordinal Functions-Footnote-1676781
-Ref: Ordinal Functions-Footnote-2677033
-Node: Join Function677243
-Ref: Join Function-Footnote-1679013
-Node: Getlocaltime Function679213
-Node: Readfile Function682955
-Node: Shell Quoting684932
-Node: Isnumeric Function686360
-Node: Data File Management687748
-Node: Filetrans Function688380
-Node: Rewind Function692476
-Node: File Checking694385
-Ref: File Checking-Footnote-1695719
-Node: Empty Files695920
-Node: Ignoring Assigns697899
-Node: Getopt Function699449
-Ref: Getopt Function-Footnote-1714746
-Node: Passwd Functions714946
-Ref: Passwd Functions-Footnote-1723785
-Node: Group Functions723873
-Ref: Group Functions-Footnote-1731771
-Node: Walking Arrays731978
-Node: Library Functions Summary734986
-Node: Library Exercises736392
-Node: Sample Programs736857
-Node: Running Examples737627
-Node: Clones738355
-Node: Cut Program739579
-Node: Egrep Program749719
-Node: Id Program758720
-Node: Split Program768655
-Ref: Split Program-Footnote-1778548
-Node: Tee Program778721
-Node: Uniq Program781511
-Node: Wc Program789099
-Node: Bytes vs. Characters789486
-Node: Using extensions791034
-Node: wc program791788
-Node: Miscellaneous Programs796653
-Node: Dupword Program797866
-Node: Alarm Program799896
-Node: Translate Program804751
-Ref: Translate Program-Footnote-1809316
-Node: Labels Program809586
-Ref: Labels Program-Footnote-1812937
-Node: Word Sorting813021
-Node: History Sorting817093
-Node: Extract Program819318
-Node: Simple Sed827331
-Node: Igawk Program830405
-Ref: Igawk Program-Footnote-1844736
-Ref: Igawk Program-Footnote-2844938
-Ref: Igawk Program-Footnote-3845060
-Node: Anagram Program845175
-Node: Signature Program848237
-Node: Programs Summary849484
-Node: Programs Exercises850698
-Ref: Programs Exercises-Footnote-1854828
-Node: Advanced Features854914
-Node: Nondecimal Data857045
-Node: Boolean Typed Values858643
-Node: Array Sorting860524
-Node: Controlling Array Traversal861229
-Ref: Controlling Array Traversal-Footnote-1869597
-Node: Array Sorting Functions869715
-Ref: Array Sorting Functions-Footnote-1875626
-Node: Two-way I/O875822
-Ref: Two-way I/O-Footnote-1883548
-Ref: Two-way I/O-Footnote-2883735
-Node: TCP/IP Networking883817
-Node: Profiling886893
-Node: Extension Philosophy896202
-Node: Advanced Features Summary897681
-Node: Internationalization899696
-Node: I18N and L10N901370
-Node: Explaining gettext902057
-Ref: Explaining gettext-Footnote-1907949
-Ref: Explaining gettext-Footnote-2908134
-Node: Programmer i18n908299
-Ref: Programmer i18n-Footnote-1913248
-Node: Translator i18n913297
-Node: String Extraction914091
-Ref: String Extraction-Footnote-1915223
-Node: Printf Ordering915309
-Ref: Printf Ordering-Footnote-1918095
-Node: I18N Portability918159
-Ref: I18N Portability-Footnote-1920615
-Node: I18N Example920678
-Ref: I18N Example-Footnote-1923953
-Ref: I18N Example-Footnote-2924026
-Node: Gawk I18N924135
-Node: I18N Summary924757
-Node: Debugger926098
-Node: Debugging927098
-Node: Debugging Concepts927539
-Node: Debugging Terms929348
-Node: Awk Debugging931923
-Ref: Awk Debugging-Footnote-1932868
-Node: Sample Debugging Session933000
-Node: Debugger Invocation933534
-Node: Finding The Bug934920
-Node: List of Debugger Commands941394
-Node: Breakpoint Control942727
-Node: Debugger Execution Control946421
-Node: Viewing And Changing Data949783
-Node: Execution Stack953324
-Node: Debugger Info954961
-Node: Miscellaneous Debugger Commands959032
-Node: Readline Support964094
-Node: Limitations964990
-Node: Debugging Summary967544
-Node: Namespaces968823
-Node: Global Namespace969934
-Node: Qualified Names971332
-Node: Default Namespace972331
-Node: Changing The Namespace973072
-Node: Naming Rules974686
-Node: Internal Name Management976534
-Node: Namespace Example977576
-Node: Namespace And Features980138
-Node: Namespace Summary981573
-Node: Arbitrary Precision Arithmetic983050
-Node: Computer Arithmetic984537
-Ref: table-numeric-ranges988303
-Ref: table-floating-point-ranges988797
-Ref: Computer Arithmetic-Footnote-1989456
-Node: Math Definitions989513
-Ref: table-ieee-formats992489
-Node: MPFR features993057
-Node: MPFR On Parole993502
-Ref: MPFR On Parole-Footnote-1994331
-Node: MPFR Intro994486
-Node: FP Math Caution996125
-Ref: FP Math Caution-Footnote-1997197
-Node: Inexactness of computations997566
-Node: Inexact representation998597
-Node: Comparing FP Values999957
-Node: Errors accumulate1001198
-Node: Strange values1002654
-Ref: Strange values-Footnote-11005242
-Node: Getting Accuracy1005347
-Node: Try To Round1008057
-Node: Setting precision1008956
-Ref: table-predefined-precision-strings1009653
-Node: Setting the rounding mode1011484
-Ref: table-gawk-rounding-modes1011858
-Ref: Setting the rounding mode-Footnote-11015790
-Node: Arbitrary Precision Integers1015969
-Ref: Arbitrary Precision Integers-Footnote-11019144
-Node: Checking for MPFR1019293
-Node: POSIX Floating Point Problems1020767
-Ref: POSIX Floating Point Problems-Footnote-11025420
-Node: Floating point summary1025458
-Node: Dynamic Extensions1027648
-Node: Extension Intro1029201
-Node: Plugin License1030467
-Node: Extension Mechanism Outline1031264
-Ref: figure-load-extension1031703
-Ref: figure-register-new-function1033269
-Ref: figure-call-new-function1034362
-Node: Extension API Description1036425
-Node: Extension API Functions Introduction1038138
-Ref: table-api-std-headers1039974
-Node: General Data Types1044224
-Ref: General Data Types-Footnote-11052930
-Node: Memory Allocation Functions1053229
-Ref: Memory Allocation Functions-Footnote-11057730
-Node: Constructor Functions1057829
-Node: API Ownership of MPFR and GMP Values1061482
-Node: Registration Functions1063015
-Node: Extension Functions1063715
-Node: Exit Callback Functions1069037
-Node: Extension Version String1070287
-Node: Input Parsers1070950
-Node: Output Wrappers1083671
-Node: Two-way processors1088183
-Node: Printing Messages1090448
-Ref: Printing Messages-Footnote-11091619
-Node: Updating ERRNO1091772
-Node: Requesting Values1092511
-Ref: table-value-types-returned1093248
-Node: Accessing Parameters1094357
-Node: Symbol Table Access1095594
-Node: Symbol table by name1096106
-Ref: Symbol table by name-Footnote-11099131
-Node: Symbol table by cookie1099259
-Ref: Symbol table by cookie-Footnote-11103444
-Node: Cached values1103508
-Ref: Cached values-Footnote-11107044
-Node: Array Manipulation1107197
-Ref: Array Manipulation-Footnote-11108288
-Node: Array Data Types1108325
-Ref: Array Data Types-Footnote-11110983
-Node: Array Functions1111075
-Node: Flattening Arrays1115860
-Node: Creating Arrays1122836
-Node: Redirection API1127603
-Node: Extension API Variables1130436
-Node: Extension Versioning1131147
-Ref: gawk-api-version1131576
-Node: Extension GMP/MPFR Versioning1133308
-Node: Extension API Informational Variables1134936
-Node: Extension API Boilerplate1136009
-Node: Changes from API V11139983
-Node: Finding Extensions1141555
-Node: Extension Example1142114
-Node: Internal File Description1142912
-Node: Internal File Ops1146992
-Ref: Internal File Ops-Footnote-11158342
-Node: Using Internal File Ops1158482
-Ref: Using Internal File Ops-Footnote-11160865
-Node: Extension Samples1161139
-Node: Extension Sample File Functions1162668
-Node: Extension Sample Fnmatch1170317
-Node: Extension Sample Fork1171804
-Node: Extension Sample Inplace1173022
-Node: Extension Sample Ord1176648
-Node: Extension Sample Readdir1177484
-Ref: table-readdir-file-types1178373
-Node: Extension Sample Revout1179441
-Node: Extension Sample Rev2way1180030
-Node: Extension Sample Read write array1180770
-Node: Extension Sample Readfile1183935
-Node: Extension Sample Time1185030
-Node: Extension Sample API Tests1186782
-Node: gawkextlib1187274
-Node: Extension summary1190192
-Node: Extension Exercises1193894
-Node: Language History1195136
-Node: V7/SVR3.11196792
-Node: SVR41198944
-Node: POSIX1200378
-Node: BTL1201759
-Node: POSIX/GNU1202488
-Node: Feature History1208266
-Node: Common Extensions1225441
-Node: Ranges and Locales1226724
-Ref: Ranges and Locales-Footnote-11231340
-Ref: Ranges and Locales-Footnote-21231367
-Ref: Ranges and Locales-Footnote-31231602
-Node: Contributors1231825
-Node: History summary1237822
-Node: Installation1239202
-Node: Gawk Distribution1240146
-Node: Getting1240630
-Node: Extracting1241593
-Node: Distribution contents1243231
-Node: Unix Installation1250292
-Node: Quick Installation1251096
-Node: Compiling with MPFR1253516
-Node: Shell Startup Files1254206
-Node: Additional Configuration Options1255295
-Node: Configuration Philosophy1257610
-Node: Compiling from Git1260006
-Node: Building the Documentation1260561
-Node: Non-Unix Installation1261945
-Node: PC Installation1262405
-Node: PC Binary Installation1263243
-Node: PC Compiling1264116
-Node: PC Using1265233
-Node: Cygwin1268786
-Node: MSYS1270010
-Node: VMS Installation1270612
-Node: VMS Compilation1271331
-Ref: VMS Compilation-Footnote-11272560
-Node: VMS Dynamic Extensions1272618
-Node: VMS Installation Details1274303
-Node: VMS Running1276565
-Node: VMS GNV1280844
-Node: Bugs1281558
-Node: Bug definition1282470
-Node: Bug address1285406
-Node: Usenet1288594
-Node: Performance bugs1289783
-Node: Asking for help1292704
-Node: Maintainers1294671
-Node: Other Versions1295865
-Node: Installation summary1304135
-Node: Notes1305499
-Node: Compatibility Mode1306293
-Node: Additions1307075
-Node: Accessing The Source1308000
-Node: Adding Code1309437
-Node: New Ports1316252
-Node: Derived Files1320627
-Ref: Derived Files-Footnote-11326287
-Ref: Derived Files-Footnote-21326322
-Ref: Derived Files-Footnote-31326920
-Node: Future Extensions1327034
-Node: Implementation Limitations1327692
-Node: Extension Design1328902
-Node: Old Extension Problems1330046
-Ref: Old Extension Problems-Footnote-11331564
-Node: Extension New Mechanism Goals1331621
-Ref: Extension New Mechanism Goals-Footnote-11334985
-Node: Extension Other Design Decisions1335174
-Node: Extension Future Growth1337287
-Node: Notes summary1337893
-Node: Basic Concepts1339051
-Node: Basic High Level1339732
-Ref: figure-general-flow1340014
-Ref: figure-process-flow1340700
-Ref: Basic High Level-Footnote-11344002
-Node: Basic Data Typing1344187
-Node: Glossary1347515
-Node: Copying1379402
-Node: GNU Free Documentation License1416945
-Node: Index1442065
+Node: Foreword345790
+Node: Foreword450232
+Node: Preface51764
+Ref: Preface-Footnote-154623
+Ref: Preface-Footnote-254732
+Ref: Preface-Footnote-354966
+Node: History55108
+Node: Names57460
+Ref: Names-Footnote-158564
+Node: This Manual58711
+Ref: This Manual-Footnote-165350
+Node: Conventions65450
+Node: Manual History67819
+Ref: Manual History-Footnote-170816
+Ref: Manual History-Footnote-270857
+Node: How To Contribute70931
+Node: Acknowledgments71853
+Node: Getting Started76790
+Node: Running gawk79229
+Node: One-shot80419
+Node: Read Terminal81682
+Node: Long83675
+Node: Executable Scripts85188
+Ref: Executable Scripts-Footnote-187821
+Node: Comments87924
+Node: Quoting90408
+Node: DOS Quoting95934
+Node: Sample Data Files97990
+Node: Very Simple100585
+Node: Two Rules106687
+Node: More Complex108572
+Node: Statements/Lines110904
+Ref: Statements/Lines-Footnote-1115388
+Node: Other Features115653
+Node: When116589
+Ref: When-Footnote-1118343
+Node: Intro Summary118408
+Node: Invoking Gawk119292
+Node: Command Line120806
+Node: Options121604
+Ref: Options-Footnote-1139853
+Ref: Options-Footnote-2140084
+Node: Other Arguments140109
+Node: Naming Standard Input144120
+Node: Environment Variables145330
+Node: AWKPATH Variable145888
+Ref: AWKPATH Variable-Footnote-1149300
+Ref: AWKPATH Variable-Footnote-2149334
+Node: AWKLIBPATH Variable149705
+Ref: AWKLIBPATH Variable-Footnote-1151402
+Node: Other Environment Variables151777
+Node: Exit Status155729
+Node: Include Files156406
+Node: Loading Shared Libraries160096
+Node: Obsolete161524
+Node: Undocumented162144
+Node: Invoking Summary162441
+Node: Regexp165282
+Node: Regexp Usage166736
+Node: Escape Sequences168773
+Node: Regexp Operators175015
+Node: Regexp Operator Details175500
+Ref: Regexp Operator Details-Footnote-1182864
+Node: Interval Expressions183011
+Ref: Interval Expressions-Footnote-1185186
+Node: Bracket Expressions185284
+Ref: table-char-classes187760
+Node: Leftmost Longest191087
+Node: Computed Regexps192390
+Node: GNU Regexp Operators195817
+Node: Case-sensitivity199495
+Ref: Case-sensitivity-Footnote-1202361
+Ref: Case-sensitivity-Footnote-2202596
+Node: Regexp Summary202704
+Node: Reading Files204170
+Node: Records206439
+Node: awk split records207514
+Node: gawk split records212214
+Ref: gawk split records-Footnote-1217288
+Node: Fields217325
+Node: Nonconstant Fields220066
+Ref: Nonconstant Fields-Footnote-1222302
+Node: Changing Fields222506
+Node: Field Separators228537
+Node: Default Field Splitting231235
+Node: Regexp Field Splitting232353
+Node: Single Character Fields236030
+Node: Command Line Field Separator237090
+Node: Full Line Fields240308
+Ref: Full Line Fields-Footnote-1241830
+Ref: Full Line Fields-Footnote-2241876
+Node: Field Splitting Summary241977
+Node: Constant Size244051
+Node: Fixed width data244783
+Node: Skipping intervening248250
+Node: Allowing trailing data249048
+Node: Fields with fixed data250085
+Node: Splitting By Content251603
+Ref: Splitting By Content-Footnote-1255439
+Node: More CSV255602
+Node: FS versus FPAT257217
+Node: Testing field creation258377
+Node: Multiple Line260002
+Node: Getline266279
+Node: Plain Getline268748
+Node: Getline/Variable271321
+Node: Getline/File272472
+Node: Getline/Variable/File273860
+Ref: Getline/Variable/File-Footnote-1275465
+Node: Getline/Pipe275553
+Node: Getline/Variable/Pipe278257
+Node: Getline/Coprocess279392
+Node: Getline/Variable/Coprocess280659
+Node: Getline Notes281401
+Node: Getline Summary284198
+Ref: table-getline-variants284622
+Node: Read Timeout285371
+Ref: Read Timeout-Footnote-1289287
+Node: Retrying Input289345
+Node: Command-line directories290544
+Node: Input Summary291450
+Node: Input Exercises294622
+Node: Printing295056
+Node: Print296890
+Node: Print Examples298347
+Node: Output Separators301127
+Node: OFMT303144
+Node: Printf304500
+Node: Basic Printf305285
+Node: Control Letters306859
+Node: Format Modifiers312021
+Node: Printf Examples318036
+Node: Redirection320522
+Node: Special FD327363
+Ref: Special FD-Footnote-1330531
+Node: Special Files330605
+Node: Other Inherited Files331222
+Node: Special Network332223
+Node: Special Caveats333083
+Node: Close Files And Pipes334032
+Ref: table-close-pipe-return-values340939
+Ref: Close Files And Pipes-Footnote-1341753
+Ref: Close Files And Pipes-Footnote-2341901
+Node: Nonfatal342053
+Node: Output Summary344391
+Node: Output Exercises345613
+Node: Expressions346292
+Node: Values347480
+Node: Constants348158
+Node: Scalar Constants348849
+Ref: Scalar Constants-Footnote-1351359
+Node: Nondecimal-numbers351609
+Node: Regexp Constants354610
+Node: Using Constant Regexps355136
+Node: Standard Regexp Constants355758
+Node: Strong Regexp Constants358946
+Node: Variables362661
+Node: Using Variables363318
+Node: Assignment Options365228
+Node: Conversion367699
+Node: Strings And Numbers368223
+Ref: Strings And Numbers-Footnote-1371286
+Node: Locale influences conversions371395
+Ref: table-locale-affects374153
+Node: All Operators374772
+Node: Arithmetic Ops375401
+Node: Concatenation378117
+Ref: Concatenation-Footnote-1380964
+Node: Assignment Ops381071
+Ref: table-assign-ops386062
+Node: Increment Ops387376
+Node: Truth Values and Conditions390836
+Node: Truth Values391910
+Node: Typing and Comparison392958
+Node: Variable Typing393778
+Ref: Variable Typing-Footnote-1400241
+Ref: Variable Typing-Footnote-2400313
+Node: Comparison Operators400390
+Ref: table-relational-ops400809
+Node: POSIX String Comparison404305
+Ref: POSIX String Comparison-Footnote-1406000
+Ref: POSIX String Comparison-Footnote-2406139
+Node: Boolean Ops406223
+Ref: Boolean Ops-Footnote-1410705
+Node: Conditional Exp410797
+Node: Function Calls412533
+Node: Precedence416410
+Node: Locales420069
+Node: Expressions Summary421701
+Node: Patterns and Actions424274
+Node: Pattern Overview425394
+Node: Regexp Patterns427071
+Node: Expression Patterns427613
+Node: Ranges431394
+Node: BEGIN/END434502
+Node: Using BEGIN/END435263
+Ref: Using BEGIN/END-Footnote-1438017
+Node: I/O And BEGIN/END438123
+Node: BEGINFILE/ENDFILE440436
+Node: Empty443667
+Node: Using Shell Variables443984
+Node: Action Overview446258
+Node: Statements448583
+Node: If Statement450431
+Node: While Statement451926
+Node: Do Statement453954
+Node: For Statement455102
+Node: Switch Statement458357
+Node: Break Statement460798
+Node: Continue Statement462890
+Node: Next Statement464717
+Node: Nextfile Statement467100
+Node: Exit Statement469789
+Node: Built-in Variables472192
+Node: User-modified473325
+Node: Auto-set481092
+Ref: Auto-set-Footnote-1497899
+Ref: Auto-set-Footnote-2498105
+Node: ARGC and ARGV498161
+Node: Pattern Action Summary502374
+Node: Arrays504804
+Node: Array Basics506133
+Node: Array Intro506977
+Ref: figure-array-elements508952
+Ref: Array Intro-Footnote-1511657
+Node: Reference to Elements511785
+Node: Assigning Elements514249
+Node: Array Example514740
+Node: Scanning an Array516694
+Node: Controlling Scanning519716
+Ref: Controlling Scanning-Footnote-1526172
+Node: Numeric Array Subscripts526488
+Node: Uninitialized Subscripts528672
+Node: Delete530291
+Ref: Delete-Footnote-1533043
+Node: Multidimensional533100
+Node: Multiscanning536195
+Node: Arrays of Arrays537786
+Node: Arrays Summary542554
+Node: Functions544647
+Node: Built-in545685
+Node: Calling Built-in546838
+Node: Boolean Functions548834
+Node: Numeric Functions549388
+Ref: Numeric Functions-Footnote-1553415
+Ref: Numeric Functions-Footnote-2554063
+Ref: Numeric Functions-Footnote-3554111
+Node: String Functions554383
+Ref: String Functions-Footnote-1579225
+Ref: String Functions-Footnote-2579353
+Ref: String Functions-Footnote-3579601
+Node: Gory Details579688
+Ref: table-sub-escapes581479
+Ref: table-sub-proposed582999
+Ref: table-posix-sub584363
+Ref: table-gensub-escapes585905
+Ref: Gory Details-Footnote-1586729
+Node: I/O Functions586883
+Ref: table-system-return-values593337
+Ref: I/O Functions-Footnote-1595418
+Ref: I/O Functions-Footnote-2595566
+Node: Time Functions595686
+Ref: Time Functions-Footnote-1606357
+Ref: Time Functions-Footnote-2606425
+Ref: Time Functions-Footnote-3606583
+Ref: Time Functions-Footnote-4606694
+Ref: Time Functions-Footnote-5606806
+Ref: Time Functions-Footnote-6607033
+Node: Bitwise Functions607299
+Ref: table-bitwise-ops607893
+Ref: Bitwise Functions-Footnote-1613957
+Ref: Bitwise Functions-Footnote-2614130
+Node: Type Functions614321
+Node: I18N Functions617741
+Node: User-defined619392
+Node: Definition Syntax620204
+Ref: Definition Syntax-Footnote-1625898
+Node: Function Example625969
+Ref: Function Example-Footnote-1628891
+Node: Function Calling628913
+Node: Calling A Function629501
+Node: Variable Scope630459
+Node: Pass By Value/Reference633453
+Node: Function Caveats636097
+Ref: Function Caveats-Footnote-1638144
+Node: Return Statement638264
+Node: Dynamic Typing641243
+Node: Indirect Calls642173
+Node: Functions Summary653100
+Node: Library Functions655805
+Ref: Library Functions-Footnote-1659412
+Ref: Library Functions-Footnote-2659555
+Node: Library Names659726
+Ref: Library Names-Footnote-1663393
+Ref: Library Names-Footnote-2663616
+Node: General Functions663702
+Node: Strtonum Function664884
+Node: Assert Function667906
+Node: Round Function671232
+Node: Cliff Random Function672772
+Node: Ordinal Functions673788
+Ref: Ordinal Functions-Footnote-1676851
+Ref: Ordinal Functions-Footnote-2677103
+Node: Join Function677313
+Ref: Join Function-Footnote-1679083
+Node: Getlocaltime Function679283
+Node: Readfile Function683025
+Node: Shell Quoting685002
+Node: Isnumeric Function686430
+Node: Data File Management687818
+Node: Filetrans Function688450
+Node: Rewind Function692546
+Node: File Checking694455
+Ref: File Checking-Footnote-1695789
+Node: Empty Files695990
+Node: Ignoring Assigns697969
+Node: Getopt Function699519
+Ref: Getopt Function-Footnote-1714816
+Node: Passwd Functions715016
+Ref: Passwd Functions-Footnote-1723855
+Node: Group Functions723943
+Ref: Group Functions-Footnote-1731841
+Node: Walking Arrays732048
+Node: Library Functions Summary735056
+Node: Library Exercises736462
+Node: Sample Programs736927
+Node: Running Examples737697
+Node: Clones738425
+Node: Cut Program739649
+Node: Egrep Program749789
+Node: Id Program758790
+Node: Split Program768725
+Ref: Split Program-Footnote-1778618
+Node: Tee Program778791
+Node: Uniq Program781581
+Node: Wc Program789169
+Node: Bytes vs. Characters789556
+Node: Using extensions791104
+Node: wc program791858
+Node: Miscellaneous Programs796723
+Node: Dupword Program797936
+Node: Alarm Program799966
+Node: Translate Program804821
+Ref: Translate Program-Footnote-1809386
+Node: Labels Program809656
+Ref: Labels Program-Footnote-1813007
+Node: Word Sorting813091
+Node: History Sorting817163
+Node: Extract Program819388
+Node: Simple Sed827401
+Node: Igawk Program830475
+Ref: Igawk Program-Footnote-1844806
+Ref: Igawk Program-Footnote-2845008
+Ref: Igawk Program-Footnote-3845130
+Node: Anagram Program845245
+Node: Signature Program848307
+Node: Programs Summary849554
+Node: Programs Exercises850768
+Ref: Programs Exercises-Footnote-1854898
+Node: Advanced Features854984
+Node: Nondecimal Data857381
+Node: Boolean Typed Values858979
+Node: Array Sorting860860
+Node: Controlling Array Traversal861565
+Ref: Controlling Array Traversal-Footnote-1869933
+Node: Array Sorting Functions870051
+Ref: Array Sorting Functions-Footnote-1875962
+Node: Two-way I/O876158
+Ref: Two-way I/O-Footnote-1883884
+Ref: Two-way I/O-Footnote-2884071
+Node: TCP/IP Networking884153
+Node: Profiling887229
+Node: Persistent Memory896535
+Ref: Persistent Memory-Footnote-1902697
+Node: Extension Philosophy902824
+Node: Advanced Features Summary904311
+Node: Internationalization906483
+Node: I18N and L10N908157
+Node: Explaining gettext908844
+Ref: Explaining gettext-Footnote-1914736
+Ref: Explaining gettext-Footnote-2914921
+Node: Programmer i18n915086
+Ref: Programmer i18n-Footnote-1920035
+Node: Translator i18n920084
+Node: String Extraction920878
+Ref: String Extraction-Footnote-1922010
+Node: Printf Ordering922096
+Ref: Printf Ordering-Footnote-1924882
+Node: I18N Portability924946
+Ref: I18N Portability-Footnote-1927402
+Node: I18N Example927465
+Ref: I18N Example-Footnote-1930740
+Ref: I18N Example-Footnote-2930813
+Node: Gawk I18N930922
+Node: I18N Summary931544
+Node: Debugger932885
+Node: Debugging933885
+Node: Debugging Concepts934326
+Node: Debugging Terms936135
+Node: Awk Debugging938710
+Ref: Awk Debugging-Footnote-1939655
+Node: Sample Debugging Session939787
+Node: Debugger Invocation940321
+Node: Finding The Bug941707
+Node: List of Debugger Commands948181
+Node: Breakpoint Control949514
+Node: Debugger Execution Control953208
+Node: Viewing And Changing Data956570
+Node: Execution Stack960111
+Node: Debugger Info961748
+Node: Miscellaneous Debugger Commands965819
+Node: Readline Support970881
+Node: Limitations971777
+Node: Debugging Summary974331
+Node: Namespaces975610
+Node: Global Namespace976721
+Node: Qualified Names978119
+Node: Default Namespace979118
+Node: Changing The Namespace979859
+Node: Naming Rules981473
+Node: Internal Name Management983321
+Node: Namespace Example984363
+Node: Namespace And Features986925
+Node: Namespace Summary988360
+Node: Arbitrary Precision Arithmetic989837
+Node: Computer Arithmetic991324
+Ref: table-numeric-ranges995090
+Ref: table-floating-point-ranges995584
+Ref: Computer Arithmetic-Footnote-1996243
+Node: Math Definitions996300
+Ref: table-ieee-formats999276
+Node: MPFR features999844
+Node: MPFR On Parole1000289
+Ref: MPFR On Parole-Footnote-11001118
+Node: MPFR Intro1001273
+Node: FP Math Caution1002912
+Ref: FP Math Caution-Footnote-11003984
+Node: Inexactness of computations1004353
+Node: Inexact representation1005384
+Node: Comparing FP Values1006744
+Node: Errors accumulate1007985
+Node: Strange values1009441
+Ref: Strange values-Footnote-11012029
+Node: Getting Accuracy1012134
+Node: Try To Round1014844
+Node: Setting precision1015743
+Ref: table-predefined-precision-strings1016440
+Node: Setting the rounding mode1018271
+Ref: table-gawk-rounding-modes1018645
+Ref: Setting the rounding mode-Footnote-11022577
+Node: Arbitrary Precision Integers1022756
+Ref: Arbitrary Precision Integers-Footnote-11025931
+Node: Checking for MPFR1026080
+Node: POSIX Floating Point Problems1027554
+Ref: POSIX Floating Point Problems-Footnote-11032207
+Node: Floating point summary1032245
+Node: Dynamic Extensions1034435
+Node: Extension Intro1035988
+Node: Plugin License1037254
+Node: Extension Mechanism Outline1038051
+Ref: figure-load-extension1038490
+Ref: figure-register-new-function1040056
+Ref: figure-call-new-function1041149
+Node: Extension API Description1043212
+Node: Extension API Functions Introduction1044925
+Ref: table-api-std-headers1046761
+Node: General Data Types1051011
+Ref: General Data Types-Footnote-11059717
+Node: Memory Allocation Functions1060016
+Ref: Memory Allocation Functions-Footnote-11064517
+Node: Constructor Functions1064616
+Node: API Ownership of MPFR and GMP Values1068269
+Node: Registration Functions1069802
+Node: Extension Functions1070502
+Node: Exit Callback Functions1075824
+Node: Extension Version String1077074
+Node: Input Parsers1077737
+Node: Output Wrappers1090458
+Node: Two-way processors1094970
+Node: Printing Messages1097235
+Ref: Printing Messages-Footnote-11098406
+Node: Updating ERRNO1098559
+Node: Requesting Values1099298
+Ref: table-value-types-returned1100035
+Node: Accessing Parameters1101144
+Node: Symbol Table Access1102381
+Node: Symbol table by name1102893
+Ref: Symbol table by name-Footnote-11105918
+Node: Symbol table by cookie1106046
+Ref: Symbol table by cookie-Footnote-11110231
+Node: Cached values1110295
+Ref: Cached values-Footnote-11113831
+Node: Array Manipulation1113984
+Ref: Array Manipulation-Footnote-11115075
+Node: Array Data Types1115112
+Ref: Array Data Types-Footnote-11117770
+Node: Array Functions1117862
+Node: Flattening Arrays1122647
+Node: Creating Arrays1129623
+Node: Redirection API1134390
+Node: Extension API Variables1137223
+Node: Extension Versioning1137934
+Ref: gawk-api-version1138363
+Node: Extension GMP/MPFR Versioning1140095
+Node: Extension API Informational Variables1141723
+Node: Extension API Boilerplate1142796
+Node: Changes from API V11146770
+Node: Finding Extensions1148342
+Node: Extension Example1148901
+Node: Internal File Description1149699
+Node: Internal File Ops1153779
+Ref: Internal File Ops-Footnote-11165129
+Node: Using Internal File Ops1165269
+Ref: Using Internal File Ops-Footnote-11167652
+Node: Extension Samples1167926
+Node: Extension Sample File Functions1169455
+Node: Extension Sample Fnmatch1177104
+Node: Extension Sample Fork1178591
+Node: Extension Sample Inplace1179809
+Node: Extension Sample Ord1183435
+Node: Extension Sample Readdir1184271
+Ref: table-readdir-file-types1185160
+Node: Extension Sample Revout1186228
+Node: Extension Sample Rev2way1186817
+Node: Extension Sample Read write array1187557
+Node: Extension Sample Readfile1190722
+Node: Extension Sample Time1191817
+Node: Extension Sample API Tests1193569
+Node: gawkextlib1194061
+Node: Extension summary1196979
+Node: Extension Exercises1200681
+Node: Language History1201923
+Node: V7/SVR3.11203579
+Node: SVR41205731
+Node: POSIX1207165
+Node: BTL1208546
+Node: POSIX/GNU1209275
+Node: Feature History1215053
+Node: Common Extensions1232228
+Node: Ranges and Locales1233511
+Ref: Ranges and Locales-Footnote-11238127
+Ref: Ranges and Locales-Footnote-21238154
+Ref: Ranges and Locales-Footnote-31238389
+Node: Contributors1238612
+Node: History summary1244609
+Node: Installation1245989
+Node: Gawk Distribution1246933
+Node: Getting1247417
+Node: Extracting1248380
+Node: Distribution contents1250018
+Node: Unix Installation1257079
+Node: Quick Installation1257883
+Node: Compiling with MPFR1260303
+Node: Shell Startup Files1260993
+Node: Additional Configuration Options1262082
+Node: Configuration Philosophy1264397
+Node: Compiling from Git1266793
+Node: Building the Documentation1267348
+Node: Non-Unix Installation1268732
+Node: PC Installation1269192
+Node: PC Binary Installation1270030
+Node: PC Compiling1270903
+Node: PC Using1272020
+Node: Cygwin1275573
+Node: MSYS1276797
+Node: VMS Installation1277399
+Node: VMS Compilation1278118
+Ref: VMS Compilation-Footnote-11279347
+Node: VMS Dynamic Extensions1279405
+Node: VMS Installation Details1281090
+Node: VMS Running1283352
+Node: VMS GNV1287631
+Node: Bugs1288345
+Node: Bug definition1289257
+Node: Bug address1292193
+Node: Usenet1295381
+Node: Performance bugs1296570
+Node: Asking for help1299491
+Node: Maintainers1301458
+Node: Other Versions1302652
+Node: Installation summary1310922
+Node: Notes1312286
+Node: Compatibility Mode1313080
+Node: Additions1313862
+Node: Accessing The Source1314787
+Node: Adding Code1316224
+Node: New Ports1323039
+Node: Derived Files1327414
+Ref: Derived Files-Footnote-11333074
+Ref: Derived Files-Footnote-21333109
+Ref: Derived Files-Footnote-31333707
+Node: Future Extensions1333821
+Node: Implementation Limitations1334479
+Node: Extension Design1335689
+Node: Old Extension Problems1336833
+Ref: Old Extension Problems-Footnote-11338351
+Node: Extension New Mechanism Goals1338408
+Ref: Extension New Mechanism Goals-Footnote-11341772
+Node: Extension Other Design Decisions1341961
+Node: Extension Future Growth1344074
+Node: Notes summary1344680
+Node: Basic Concepts1345838
+Node: Basic High Level1346519
+Ref: figure-general-flow1346801
+Ref: figure-process-flow1347487
+Ref: Basic High Level-Footnote-11350789
+Node: Basic Data Typing1350974
+Node: Glossary1354302
+Node: Copying1386189
+Node: GNU Free Documentation License1423732
+Node: Index1448852
End Tag Table
diff --git a/doc/gawk.texi b/doc/gawk.texi
index 0bc75bcf..cc4bab53 100644
--- a/doc/gawk.texi
+++ b/doc/gawk.texi
@@ -878,6 +878,7 @@ particular records in a file and perform operations upon
them.
* TCP/IP Networking:: Using @command{gawk} for network
programming.
* Profiling:: Profiling your @command{awk} programs.
+* Persistent Memory:: Preserving data between runs.
* Extension Philosophy:: What should be built-in and what
should not.
* Advanced Features Summary:: Summary of advanced features.
@@ -29591,9 +29592,13 @@ programs.
Then, @command{gawk}'s special features for sorting arrays are presented.
Next, two-way I/O, discussed briefly in earlier parts of this
@value{DOCUMENT}, is described in full detail, along with the basics
-of TCP/IP networking. Finally, we see how @command{gawk}
+of TCP/IP networking. We then see how @command{gawk}
can @dfn{profile} an @command{awk} program, making it possible to tune
it for performance.
+Next, we present an experimental feature that allows you to preserve
+the values of @command{awk} variables and arrays between runs of
@command{gawk}.
+Finally, we discuss the philosophy behind @command{gawk}'s extension
+mechanism.
@c FULLXREF ON
Additional advanced features are discussed in separate @value{CHAPTER}s of
their
@@ -29628,6 +29633,7 @@ discusses the ability to dynamically add new built-in
functions to
* Two-way I/O:: Two-way communications with another process.
* TCP/IP Networking:: Using @command{gawk} for network programming.
* Profiling:: Profiling your @command{awk} programs.
+* Persistent Memory:: Preserving data between runs.
* Extension Philosophy:: What should be built-in and what should not.
* Advanced Features Summary:: Summary of advanced features.
@end menu
@@ -30962,6 +30968,169 @@ Profiling and pretty-printing also preserve the
original format of numeric
constants; if you used an octal or hexadecimal value in your source
code, it will appear that way in the output.
+@node Persistent Memory
+@section Preserving Data Between Runs
+
+Starting with @value{PVERSION} 5.2, @command{gawk} supports
+@dfn{persistent memory}. This experimental feature stores the values of
+all of @command{gawk}'s variables and arrays in a persistent heap,
+which resides in a file in
+the filesystem. When persistent memory is not in use (the normal case),
+@command{gawk}'s data resides in ephemeral system memory.
+
+Persistent memory is enabled on systems supporting the @code{mmap()}
+and @code{munmap()} system calls. @command{gawk} must be compiled as a
+non-PIE (Position Independent Executable) binary, since the persistent
+store ends up holding pointers to functions held within the @command{gawk}
+executable. This also means that to use the persistent memory, you must
+use the same @command{gawk} executable from run to run.
+
+As of this writing, persistent memory has only been tested on GNU/Linux
+and Mac OS systems. On others, your mileage may vary, and/or you may
+need to investigate how to make it work for you.
+
+To use persistent memory, follow these steps:
+
+@enumerate 1
+@item
+Create a new, empty sparse file of the desired size. For example, four
+gigabytes. On a GNU/Linux system, you can use the @command{truncate}
+command:
+
+@example
+$ @kbd{truncate -s 4G data.pma}
+@end example
+
+@item
+Provide the path to the data file in the @env{GAWK_PERSIST_FILE}
+environment variable. This is best done by placing the value in the
+environment just for the run of @command{gawk}, like so:
+
+@example
+$ @kbd{GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN @{ print ++i @}'}
+1
+@end example
+
+@item
+Use the same data file in subsequent runs to use the preserved data values:
+
+@example
+$ @kbd{GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN @{ print ++i @}'}
+2
+$ @kbd{GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN @{ print ++i @}'}
+3
+@end example
+
+@noindent
+As shown, in subsequent runs using the same data file, the values of
+@command{gawk}'s variables are preserved.
+
+@end enumerate
+
+Interestingly, the program that you execute need not be the same from
+run to run; the persistent store only maintains the values of variables
+and arrays, not the totality of @command{gawk}'s internal state.
+This lets you share data between unrelated programs, eliminating
+the need for scripts to communicate via text files.
+
+Terence Kelly, the author of the persistent memory allocator
+@command{gawk} uses, provides the following advice about the backing file:
+
+@quotation
+Regarding backing file size, I recommend making it far larger
+than all of the data that will ever reside in it, assuming
+that the file system supports sparse files. The ``pay only
+for what you use'' aspect of sparse files ensures that the
+actual storage resource footprint of the backing file will
+meet the application's needs but will be as small as
+possible. If the file system does @emph{not} support sparse
+files, there's a dilemma: Making the backing file too large
+is wasteful, but making it too small risks memory exhaustion,
+i.e., @code{pma_malloc()} returns @code{NULL}.
+But persistent @command{gawk} should
+still work even without sparse files.
+@ignore
+Note also that if the
+file system is encrypted, sparsity is no longer meaningful,
+even if the file system would otherwise support sparse files.
+At least this is true for some combinations of FS and FS
+encryption. Not sure if we need to say that here.
+@end ignore
+@end quotation
+
+You can disable the use of the persistent memory allocator in
+@command{gawk} with the @option{--disable-pma} option to the
@command{configure}
+command at the time that you build @command{gawk} (@pxref{Unix
+Installation}).
+
+Here are articles and web links that provide more information about
+persistent memory and why it's useful in a scripting language like
+@command{gawk}.
+
+@table @asis
+@item @uref{https://web.eecs.umich.edu/~tpkelly/pma/}
+This is the canonical source for Terence Kelly's Persistent Memory
+Allocator (PMA). Kelly may be reached directly at any of the following
+email addresses:
+@EMAIL{tpkelly@@acm.org, tpkelly AT acm.org},
+@EMAIL{tpkelly@@cs.princeton.edu, tpkelly AT cs.princeton.edu}, or
+@EMAIL{tpkelly@@eecs.umich.edu, tpkelly AT eecs.umich.edu}.
+
+@item @cite{Persistent Memory Allocation}
+Terence Kelly, Zi Fan Tan, Jianan Li, and Haris Volos,
+ACM @cite{Queue} magazine, Vol. 20 No. 2 (March/April 2022),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3534855, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3534855, HTML}.
+This paper explains the design of the PMA
+allocator used in persistent @command{gawk}.
+
+@item @cite{Persistent Scripting}
+Zi Fan Tan, Jianan Li, Haris Volos, and Terence Kelly,
+Non-Volatile Memory Workshop (NVMW) 2022,
+@uref{http://nvmw.ucsd.edu/program/}.
+This paper motivates and describes a research prototype of persistent
+@command{gawk} and presents performance evaluations on Intel Optane
+non-volatile memory; note that the interface differs slightly.
+
+@item @cite{Persistent Memory Programming on Conventional Hardware}
+Terence Kelly,
+ACM @cite{Queue} magazine Vol. 17 No. 4 (July/Aug 2019),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3358955.3358957, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3358957, HTML}.
+This paper describes simple techniques for persistent memory for C/C++
+code on conventional computers that lack non-volatile memory hardware.
+
+@item @cite{Is Persistent Memory Persistent?}
+Terence Kelly,
+ACM @cite{Queue} magazine Vol. 18 No. 2 (March/April 2020),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3400899.3400902, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3400902, HTML}.
+This paper describes a simple and robust testbed for testing software
+against real power failures.
+
+@item @cite{Crashproofing the Original NoSQL Key/Value Store}
+Terence Kelly,
+ACM @cite{Queue} magazine Vol. 19 No. 4 (July/Aug 2021),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3487019.3487353, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3487353, HTML}.
+This paper describes a crash-tolerance feature added to GNU DBM'
+(@code{gdbm}).
+@end table
+
+When Terence Kelly published his papers, his collaborators produced
+a prototype integration of PMA with @command{gawk}. That version used
+a (mandatory!) option @option{--persist=@var{file}} to specify the file
+for storing the persistent heap. If this option is given to @command{gawk},
+it produces a fatal error message instructing the user to use the
+@env{GAWK_PERSIST_FILE} environment variable instead. Except for this
+paragraph, that option is otherwise undocumented.
+
+As noted earlier, support for persistent memory is @emph{experimental}.
+If it becomes burdensome,@footnote{Meaning, there are too many
+bug reports, or too many strange differences in behavior from when
+@command{gawk} is run normally.} then the feature will be removed.
+
+
@node Extension Philosophy
@section Builtin Features versus Extensions
@@ -31049,6 +31218,10 @@ you tune them more easily. Sending the @code{USR1}
signal while profiling cause
@item
You can also just ``pretty-print'' the program.
+@item
+Persistent memory allows you to preserve the values of variables and arrays
+between runs of @command{gawk}. This feature is currently experimental.
+
@item
New features should be developed using the extension mechanism if possible;
they should be added to the core interpreter only as a last resort.
diff --git a/doc/gawktexi.in b/doc/gawktexi.in
index 7b4c2a8b..1728eb44 100644
--- a/doc/gawktexi.in
+++ b/doc/gawktexi.in
@@ -873,6 +873,7 @@ particular records in a file and perform operations upon
them.
* TCP/IP Networking:: Using @command{gawk} for network
programming.
* Profiling:: Profiling your @command{awk} programs.
+* Persistent Memory:: Preserving data between runs.
* Extension Philosophy:: What should be built-in and what
should not.
* Advanced Features Summary:: Summary of advanced features.
@@ -28473,9 +28474,13 @@ programs.
Then, @command{gawk}'s special features for sorting arrays are presented.
Next, two-way I/O, discussed briefly in earlier parts of this
@value{DOCUMENT}, is described in full detail, along with the basics
-of TCP/IP networking. Finally, we see how @command{gawk}
+of TCP/IP networking. We then see how @command{gawk}
can @dfn{profile} an @command{awk} program, making it possible to tune
it for performance.
+Next, we present an experimental feature that allows you to preserve
+the values of @command{awk} variables and arrays between runs of
@command{gawk}.
+Finally, we discuss the philosophy behind @command{gawk}'s extension
+mechanism.
@c FULLXREF ON
Additional advanced features are discussed in separate @value{CHAPTER}s of
their
@@ -28510,6 +28515,7 @@ discusses the ability to dynamically add new built-in
functions to
* Two-way I/O:: Two-way communications with another process.
* TCP/IP Networking:: Using @command{gawk} for network programming.
* Profiling:: Profiling your @command{awk} programs.
+* Persistent Memory:: Preserving data between runs.
* Extension Philosophy:: What should be built-in and what should not.
* Advanced Features Summary:: Summary of advanced features.
@end menu
@@ -29844,6 +29850,169 @@ Profiling and pretty-printing also preserve the
original format of numeric
constants; if you used an octal or hexadecimal value in your source
code, it will appear that way in the output.
+@node Persistent Memory
+@section Preserving Data Between Runs
+
+Starting with @value{PVERSION} 5.2, @command{gawk} supports
+@dfn{persistent memory}. This experimental feature stores the values of
+all of @command{gawk}'s variables and arrays in a persistent heap,
+which resides in a file in
+the filesystem. When persistent memory is not in use (the normal case),
+@command{gawk}'s data resides in ephemeral system memory.
+
+Persistent memory is enabled on systems supporting the @code{mmap()}
+and @code{munmap()} system calls. @command{gawk} must be compiled as a
+non-PIE (Position Independent Executable) binary, since the persistent
+store ends up holding pointers to functions held within the @command{gawk}
+executable. This also means that to use the persistent memory, you must
+use the same @command{gawk} executable from run to run.
+
+As of this writing, persistent memory has only been tested on GNU/Linux
+and Mac OS systems. On others, your mileage may vary, and/or you may
+need to investigate how to make it work for you.
+
+To use persistent memory, follow these steps:
+
+@enumerate 1
+@item
+Create a new, empty sparse file of the desired size. For example, four
+gigabytes. On a GNU/Linux system, you can use the @command{truncate}
+command:
+
+@example
+$ @kbd{truncate -s 4G data.pma}
+@end example
+
+@item
+Provide the path to the data file in the @env{GAWK_PERSIST_FILE}
+environment variable. This is best done by placing the value in the
+environment just for the run of @command{gawk}, like so:
+
+@example
+$ @kbd{GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN @{ print ++i @}'}
+1
+@end example
+
+@item
+Use the same data file in subsequent runs to use the preserved data values:
+
+@example
+$ @kbd{GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN @{ print ++i @}'}
+2
+$ @kbd{GAWK_PERSIST_FILE=data.pma ./gawk 'BEGIN @{ print ++i @}'}
+3
+@end example
+
+@noindent
+As shown, in subsequent runs using the same data file, the values of
+@command{gawk}'s variables are preserved.
+
+@end enumerate
+
+Interestingly, the program that you execute need not be the same from
+run to run; the persistent store only maintains the values of variables
+and arrays, not the totality of @command{gawk}'s internal state.
+This lets you share data between unrelated programs, eliminating
+the need for scripts to communicate via text files.
+
+Terence Kelly, the author of the persistent memory allocator
+@command{gawk} uses, provides the following advice about the backing file:
+
+@quotation
+Regarding backing file size, I recommend making it far larger
+than all of the data that will ever reside in it, assuming
+that the file system supports sparse files. The ``pay only
+for what you use'' aspect of sparse files ensures that the
+actual storage resource footprint of the backing file will
+meet the application's needs but will be as small as
+possible. If the file system does @emph{not} support sparse
+files, there's a dilemma: Making the backing file too large
+is wasteful, but making it too small risks memory exhaustion,
+i.e., @code{pma_malloc()} returns @code{NULL}.
+But persistent @command{gawk} should
+still work even without sparse files.
+@ignore
+Note also that if the
+file system is encrypted, sparsity is no longer meaningful,
+even if the file system would otherwise support sparse files.
+At least this is true for some combinations of FS and FS
+encryption. Not sure if we need to say that here.
+@end ignore
+@end quotation
+
+You can disable the use of the persistent memory allocator in
+@command{gawk} with the @option{--disable-pma} option to the
@command{configure}
+command at the time that you build @command{gawk} (@pxref{Unix
+Installation}).
+
+Here are articles and web links that provide more information about
+persistent memory and why it's useful in a scripting language like
+@command{gawk}.
+
+@table @asis
+@item @uref{https://web.eecs.umich.edu/~tpkelly/pma/}
+This is the canonical source for Terence Kelly's Persistent Memory
+Allocator (PMA). Kelly may be reached directly at any of the following
+email addresses:
+@EMAIL{tpkelly@@acm.org, tpkelly AT acm.org},
+@EMAIL{tpkelly@@cs.princeton.edu, tpkelly AT cs.princeton.edu}, or
+@EMAIL{tpkelly@@eecs.umich.edu, tpkelly AT eecs.umich.edu}.
+
+@item @cite{Persistent Memory Allocation}
+Terence Kelly, Zi Fan Tan, Jianan Li, and Haris Volos,
+ACM @cite{Queue} magazine, Vol. 20 No. 2 (March/April 2022),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3534855, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3534855, HTML}.
+This paper explains the design of the PMA
+allocator used in persistent @command{gawk}.
+
+@item @cite{Persistent Scripting}
+Zi Fan Tan, Jianan Li, Haris Volos, and Terence Kelly,
+Non-Volatile Memory Workshop (NVMW) 2022,
+@uref{http://nvmw.ucsd.edu/program/}.
+This paper motivates and describes a research prototype of persistent
+@command{gawk} and presents performance evaluations on Intel Optane
+non-volatile memory; note that the interface differs slightly.
+
+@item @cite{Persistent Memory Programming on Conventional Hardware}
+Terence Kelly,
+ACM @cite{Queue} magazine Vol. 17 No. 4 (July/Aug 2019),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3358955.3358957, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3358957, HTML}.
+This paper describes simple techniques for persistent memory for C/C++
+code on conventional computers that lack non-volatile memory hardware.
+
+@item @cite{Is Persistent Memory Persistent?}
+Terence Kelly,
+ACM @cite{Queue} magazine Vol. 18 No. 2 (March/April 2020),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3400899.3400902, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3400902, HTML}.
+This paper describes a simple and robust testbed for testing software
+against real power failures.
+
+@item @cite{Crashproofing the Original NoSQL Key/Value Store}
+Terence Kelly,
+ACM @cite{Queue} magazine Vol. 19 No. 4 (July/Aug 2021),
+@uref{https://dl.acm.org/doi/pdf/10.1145/3487019.3487353, PDF},
+@uref{https://queue.acm.org/detail.cfm?id=3487353, HTML}.
+This paper describes a crash-tolerance feature added to GNU DBM'
+(@code{gdbm}).
+@end table
+
+When Terence Kelly published his papers, his collaborators produced
+a prototype integration of PMA with @command{gawk}. That version used
+a (mandatory!) option @option{--persist=@var{file}} to specify the file
+for storing the persistent heap. If this option is given to @command{gawk},
+it produces a fatal error message instructing the user to use the
+@env{GAWK_PERSIST_FILE} environment variable instead. Except for this
+paragraph, that option is otherwise undocumented.
+
+As noted earlier, support for persistent memory is @emph{experimental}.
+If it becomes burdensome,@footnote{Meaning, there are too many
+bug reports, or too many strange differences in behavior from when
+@command{gawk} is run normally.} then the feature will be removed.
+
+
@node Extension Philosophy
@section Builtin Features versus Extensions
@@ -29931,6 +30100,10 @@ you tune them more easily. Sending the @code{USR1}
signal while profiling cause
@item
You can also just ``pretty-print'' the program.
+@item
+Persistent memory allows you to preserve the values of variables and arrays
+between runs of @command{gawk}. This feature is currently experimental.
+
@item
New features should be developed using the extension mechanism if possible;
they should be added to the core interpreter only as a last resort.
diff --git a/doc/wordlist b/doc/wordlist
index 90bc5aec..abb631eb 100644
--- a/doc/wordlist
+++ b/doc/wordlist
@@ -102,10 +102,12 @@ Coprocess
Coprocesses
Coreutils
Corinna
+Crashproofing
Ctrl
Curreli
Cygwin
DARKCORNER
+DBM
DCL
DCLTABLES
DD
@@ -128,6 +130,7 @@ Deifik
Demaille
Dinline
Drepper
+DrillBits
Duman
Dupword
EAGAIN
@@ -210,6 +213,7 @@ HURD
Haible
Hankerson
Haque
+Haris
Hartholz
Hasegawa
Hermann
@@ -257,6 +261,7 @@ Jaegermann
Jannick
Jawk
Jeroen
+Jianan
Johansen's
Jul
Jun
@@ -331,6 +336,7 @@ NOMATCH
NR
NT
NUMCUR
+NVMW
NaN
NaNs
Nachum
@@ -340,6 +346,7 @@ NetBSD
Newsreader
Nextfile
Nlines
+NoSQL
Nof
Nonalphabetic
Nonconstant
@@ -367,6 +374,7 @@ OpenBSD
OpenSolaris
OpenVMS
Ops
+Optane
Optarg
Opterr
Optind
@@ -383,6 +391,7 @@ PCPU
PCSI
PDF
PEBKAC
+PMA
PNG
POS
PQgetvalue
@@ -500,6 +509,7 @@ VER
VT
Versioning
Vinschen
+Volos
WIPO
WONTFIX
Walamazoo
@@ -532,6 +542,7 @@ Zaretskii
Zaretskii's
Zeichen
Zeichenkette
+Zi
Zoulas
aB
aCa
@@ -565,6 +576,7 @@ ac
aca
acbfoo
aclocal
+acm
addrs
adm
adminprog
@@ -730,6 +742,7 @@ cd
cde
cdot
ce
+cfm
cgi
cgit
cgiwrap
@@ -855,6 +868,7 @@ dlopen
dlsym
dn
docbook
+doi
dom
dont
dorking
@@ -873,6 +887,7 @@ dx
ebcdic
eca
edu
+eecs
eg
egid
egrep
@@ -1010,6 +1025,7 @@ gawktexi
gawkworkflow
gcc
gdb
+gdbm
gdef
ge
gecos
@@ -1272,6 +1288,7 @@ mkbool
mkdir
mkinstalldirs
mktime
+mmap
mmk
mms
mo
@@ -1296,6 +1313,7 @@ multiline
multispan
multitable
multitranslation
+munmap
murphy
mv
mydata
@@ -1388,6 +1406,7 @@ numfields
nums
nusers
nvalue
+nvmw
nwords
ny
nyah
@@ -1472,6 +1491,7 @@ pival
pkgdatadir
pkgextensiondir
pm
+pma
pmode
png
po
@@ -1486,6 +1506,7 @@ preformatted
preopened
prepinfo
prev
+princeton
printf
printindex
printpage
@@ -1663,6 +1684,7 @@ strnums
strtod
strtonum
struct
+su
subarray
subarrays
subentry
@@ -1726,6 +1748,7 @@ tmy
tolower
toolset
toupper
+tpkelly
tr
tracy
transfile
@@ -1742,6 +1765,7 @@ txt
typeof
tyw
ucb
+ucsd
udc
udcf
udp
diff --git a/doc/wordlist2 b/doc/wordlist2
index 2441b5ea..7be6dcf1 100644
--- a/doc/wordlist2
+++ b/doc/wordlist2
@@ -10,6 +10,7 @@ FIXME
GCC
Gettext
ISBN
+Jun
Kahrs
Libtool
PCC
diff --git a/doc/wordlist5 b/doc/wordlist5
index 63b99038..2a927dcb 100644
--- a/doc/wordlist5
+++ b/doc/wordlist5
@@ -1,6 +1,7 @@
Awk
Chet
GAWKBUG
+IDT
IST
Ramey
TMPDIR
diff --git a/extras/Makefile.in b/extras/Makefile.in
index 7cb82e0b..9ee6e9cd 100644
--- a/extras/Makefile.in
+++ b/extras/Makefile.in
@@ -121,8 +121,9 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
$(top_srcdir)/m4/lib-prefix.m4 $(top_srcdir)/m4/libsigsegv.m4 \
$(top_srcdir)/m4/longlong.m4 $(top_srcdir)/m4/mpfr.m4 \
$(top_srcdir)/m4/nls.m4 $(top_srcdir)/m4/noreturn.m4 \
- $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \
- $(top_srcdir)/m4/readline.m4 $(top_srcdir)/m4/socket.m4 \
+ $(top_srcdir)/m4/pma.m4 $(top_srcdir)/m4/po.m4 \
+ $(top_srcdir)/m4/progtest.m4 $(top_srcdir)/m4/readline.m4 \
+ $(top_srcdir)/m4/socket.m4 \
$(top_srcdir)/m4/triplet-transformation.m4 \
$(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
diff --git a/m4/ChangeLog b/m4/ChangeLog
index fb2467fb..aaac3b88 100644
--- a/m4/ChangeLog
+++ b/m4/ChangeLog
@@ -1,3 +1,25 @@
+2022-06-15 Arnold D. Robbins <arnold@skeeve.com>
+
+ * pma.m4: Add case for Cygwin.
+
+2022-06-14 Arnold D. Robbins <arnold@skeeve.com>
+
+ * pma.m4: Put -no-pie inside additional check for gcc and clang.
+ Tinycc and PCC are no-PIE anyway. Add support for --disable-pma.
+ Add the right magic for Mac OS.
+
+2022-06-13 Arnold D. Robbins <arnold@skeeve.com>
+
+ * pma.m4: Put -no-pie inside check for Linux.
+
+2022-06-10 Arnold D. Robbins <arnold@skeeve.com>
+
+ * pma.m4: Add -no-pie to LDFLAGS for persistent malloc.
+
+2022-06-09 Arnold D. Robbins <arnold@skeeve.com>
+
+ * pma.m4: New file.
+
2021-10-27 Arnold D. Robbins <arnold@skeeve.com>
* 5.1.1: Release tar ball made.
diff --git a/m4/pma.m4 b/m4/pma.m4
new file mode 100644
index 00000000..f6f6e969
--- /dev/null
+++ b/m4/pma.m4
@@ -0,0 +1,51 @@
+dnl Decide whether or not to use the persistent memory allocator
+
+# Copyright (C) 2022 Free Software Foundation, Inc.
+# This file is free software; the Free Software Foundation
+# gives unlimited permission to copy and/or distribute it,
+# with or without modifications, as long as this notice is preserved.
+
+AC_DEFUN([GAWK_USE_PERSISTENT_MALLOC],
+[
+use_persistent_malloc=no
+if test "$SKIP_PERSIST_MALLOC" = no
+then
+ AC_CHECK_FUNC([mmap])
+ AC_CHECK_FUNC([munmap])
+ if test $ac_cv_func_mmap = yes && test $ac_cv_func_munmap = yes
+ then
+ AC_DEFINE(USE_PERSISTENT_MALLOC, 1, [Define to 1 if we can use
the pma allocator])
+ use_persistent_malloc=yes
+ case $host_os in
+ linux-*)
+ case $CC in
+ gcc | clang)
+ LDFLAGS="${LDFLAGS} -no-pie"
+ export LDFLAGS
+ ;;
+ *)
+ # tinycc and pcc don't support -no-pie flag
+ # their executables are non-PIE automatically
+ # so no need to do anything
+ ;;
+ esac
+ ;;
+ *darwin*)
+ LDFLAGS="${LDFLAGS} -Xlinker -no_pie"
+ export LDFLAGS
+ ;;
+ *cygwin* | *CYGWIN*)
+ true # nothing do, Cygwin exes are not PIE
+ ;;
+ # Other OS's go here...
+ *)
+ # For now, play it safe
+ use_persistent_malloc=no
+ ;;
+ esac
+ else
+ use_persistent_malloc=no
+ fi
+fi
+AM_CONDITIONAL([USE_PERSISTENT_MALLOC], [test "$use_persistent_malloc" =
"yes"])
+])
diff --git a/main.c b/main.c
index da8bf515..1a7f642c 100644
--- a/main.c
+++ b/main.c
@@ -149,6 +149,7 @@ static void parse_args(int argc, char **argv);
static void set_locale_stuff(void);
static bool stopped_early = false;
+bool using_persistent_malloc = false;
enum do_flag_values do_flags = DO_FLAG_NONE;
bool do_itrace = false; /* provide simple instruction
trace */
bool do_optimize = true; /* apply default optimizations */
@@ -204,6 +205,7 @@ static const struct option optab[] = {
#if defined(YYDEBUG) || defined(GAWKDEBUG)
{ "parsedebug", no_argument, NULL, 'Y' },
#endif
+ { "persist", required_argument, NULL, 'T' },
{ "posix", no_argument, NULL, 'P' },
{ "pretty-print", optional_argument, NULL, 'o' },
{ "profile", optional_argument, NULL, 'p' },
@@ -227,22 +229,34 @@ main(int argc, char **argv)
bool have_srcfile = false;
SRCFILE *s;
char *cp;
+ const char *persist_file = getenv("GAWK_PERSIST_FILE"); /* backing file
+for PMA */
#if defined(LOCALEDEBUG)
const char *initial_locale;
#endif
+ myname = gawk_name(argv[0]);
+
+ if (pma_init(1, persist_file) != 0) {
+ fatal(_("persistent memory allocator failed to initialize"));
+ }
+
+ using_persistent_malloc = (persist_file != NULL);
+#ifndef USE_PERSISTENT_MALLOC
+ if (using_persistent_malloc)
+ warning(_("persistent memory is not supported"));
+#endif
+
/* do these checks early */
if (getenv("TIDYMEM") != NULL)
do_flags |= DO_TIDY_MEM;
#ifdef HAVE_MCHECK_H
#ifdef HAVE_MTRACE
- if (do_tidy_mem)
+ if (! using_persistent_malloc && do_tidy_mem)
mtrace();
#endif /* HAVE_MTRACE */
#endif /* HAVE_MCHECK_H */
-
- myname = gawk_name(argv[0]);
os_arg_fixup(&argc, &argv); /* emulate redirection, expand wildcards */
if (argc < 2)
@@ -1356,8 +1370,11 @@ version()
#ifdef DYNAMIC
printf(", API: %d.%d", GAWK_API_MAJOR_VERSION, GAWK_API_MINOR_VERSION);
#endif
+#ifdef USE_PERSISTENT_MALLOC
+ printf(", PMA: %s", pma_version);
+#endif
#ifdef HAVE_MPFR
- printf(" (GNU MPFR %s, GNU MP %s)", mpfr_get_version(), gmp_version);
+ printf(", (GNU MPFR %s, GNU MP %s)", mpfr_get_version(), gmp_version);
#endif
printf("\n");
print_ext_versions();
@@ -1711,6 +1728,14 @@ parse_args(int argc, char **argv)
do_flags |= DO_SANDBOX;
break;
+ case 'T': // --persist=file
+#ifdef USE_PERSISTENT_MALLOC
+ fatal(_("Use `GAWK_PERSIST_FILE=%s gawk ...' instead of
--persist."), optarg);
+#else
+ warning(_("Persistent memory is not supported."));
+#endif /* USE_PERSISTENT_MALLOC */
+ break;
+
case 'V':
do_version = true;
break;
diff --git a/support/ChangeLog b/support/ChangeLog
index 408137b1..2815c594 100644
--- a/support/ChangeLog
+++ b/support/ChangeLog
@@ -2,6 +2,21 @@
* dfa.c, dfa.h: Sync with GNULIB.
+2022-06-17 Arnold D. Robbins <arnold@skeeve.com>
+
+ * pma.h (PMA_H_VERSION): Add "+ gawk" to the version string.
+ * pma.c (pma_version): Ditto.
+ (PGSZ): Now a variable, assigned equal to sysconf(_SC_PAGESIZE).
+ Allows portability to Cygwin.
+ (pma_init): Remove no-longer-used `ps' variable. Remove check
+ of PGSZ against sysconf(_SC_PAGESIZE).
+
+2022-06-09 Arnold D. Robbins <arnold@skeeve.com>
+
+ * Makefile.am (libsupport_a_SOURCES): Add pma.c if we
+ have USE_PERSISTENT_MALLOC defined.
+ * pma.h, pma.c, agpl-3.0.txt: New files.
+
2022-06-03 Arnold D. Robbins <arnold@skeeve.com>
* dfa.c, dfa.h: Sync with GNULIB.
diff --git a/support/Makefile.am b/support/Makefile.am
index f209092a..9afadbd2 100644
--- a/support/Makefile.am
+++ b/support/Makefile.am
@@ -34,6 +34,8 @@ EXTRA_DIST = \
ChangeLog.0 \
Makefile.am \
Makefile.in \
+ pma.h \
+ pma.c \
regcomp.c \
regex_internal.c \
regex_internal.h \
@@ -71,6 +73,10 @@ libsupport_a_SOURCES = \
malloc/dynarray_resize.c \
malloc/dynarray_resize_clear.c
+if USE_PERSISTENT_MALLOC
+libsupport_a_SOURCES += pma.c pma.h
+endif
+
# For some make's, e.g. OpenBSD, that don't define this
RM = rm -f
diff --git a/support/Makefile.in b/support/Makefile.in
index 476e7edf..5cbf33e7 100644
--- a/support/Makefile.in
+++ b/support/Makefile.in
@@ -111,6 +111,7 @@ PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
host_triplet = @host@
+@USE_PERSISTENT_MALLOC_TRUE@am__append_1 = pma.c pma.h
subdir = support
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
@@ -121,8 +122,9 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
$(top_srcdir)/m4/lib-prefix.m4 $(top_srcdir)/m4/libsigsegv.m4 \
$(top_srcdir)/m4/longlong.m4 $(top_srcdir)/m4/mpfr.m4 \
$(top_srcdir)/m4/nls.m4 $(top_srcdir)/m4/noreturn.m4 \
- $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \
- $(top_srcdir)/m4/readline.m4 $(top_srcdir)/m4/socket.m4 \
+ $(top_srcdir)/m4/pma.m4 $(top_srcdir)/m4/po.m4 \
+ $(top_srcdir)/m4/progtest.m4 $(top_srcdir)/m4/readline.m4 \
+ $(top_srcdir)/m4/socket.m4 \
$(top_srcdir)/m4/triplet-transformation.m4 \
$(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
@@ -140,14 +142,23 @@ am__v_AR_0 = @echo " AR " $@;
am__v_AR_1 =
libsupport_a_AR = $(AR) $(ARFLAGS)
libsupport_a_LIBADD =
+am__libsupport_a_SOURCES_DIST = attribute.h cdefs.h dfa.c dfa.h \
+ dynarray.h flexmember.h getopt.c getopt.h getopt1.c \
+ getopt_int.h idx.h intprops.h libc-config.h localeinfo.c \
+ localeinfo.h random.c random.h regex.c regex.h verify.h \
+ xalloc.h malloc/dynarray.h malloc/dynarray_at_failure.c \
+ malloc/dynarray_emplace_enlarge.c malloc/dynarray_finalize.c \
+ malloc/dynarray_resize.c malloc/dynarray_resize_clear.c pma.c \
+ pma.h
am__dirstamp = $(am__leading_dot)dirstamp
+@USE_PERSISTENT_MALLOC_TRUE@am__objects_1 = pma.$(OBJEXT)
am_libsupport_a_OBJECTS = dfa.$(OBJEXT) getopt.$(OBJEXT) \
getopt1.$(OBJEXT) localeinfo.$(OBJEXT) random.$(OBJEXT) \
regex.$(OBJEXT) malloc/dynarray_at_failure.$(OBJEXT) \
malloc/dynarray_emplace_enlarge.$(OBJEXT) \
malloc/dynarray_finalize.$(OBJEXT) \
malloc/dynarray_resize.$(OBJEXT) \
- malloc/dynarray_resize_clear.$(OBJEXT)
+ malloc/dynarray_resize_clear.$(OBJEXT) $(am__objects_1)
libsupport_a_OBJECTS = $(am_libsupport_a_OBJECTS)
AM_V_P = $(am__v_P_@AM_V@)
am__v_P_ = $(am__v_P_@AM_DEFAULT_V@)
@@ -166,7 +177,7 @@ depcomp = $(SHELL) $(top_srcdir)/build-aux/depcomp
am__maybe_remake_depfiles = depfiles
am__depfiles_remade = ./$(DEPDIR)/dfa.Po ./$(DEPDIR)/getopt.Po \
./$(DEPDIR)/getopt1.Po ./$(DEPDIR)/localeinfo.Po \
- ./$(DEPDIR)/random.Po ./$(DEPDIR)/regex.Po \
+ ./$(DEPDIR)/pma.Po ./$(DEPDIR)/random.Po ./$(DEPDIR)/regex.Po \
malloc/$(DEPDIR)/dynarray_at_failure.Po \
malloc/$(DEPDIR)/dynarray_emplace_enlarge.Po \
malloc/$(DEPDIR)/dynarray_finalize.Po \
@@ -186,7 +197,7 @@ am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@)
am__v_CCLD_0 = @echo " CCLD " $@;
am__v_CCLD_1 =
SOURCES = $(libsupport_a_SOURCES)
-DIST_SOURCES = $(libsupport_a_SOURCES)
+DIST_SOURCES = $(am__libsupport_a_SOURCES_DIST)
am__can_run_installinfo = \
case $$AM_UPDATE_INFO_DIR in \
n|no|NO) false;; \
@@ -356,6 +367,8 @@ EXTRA_DIST = \
ChangeLog.0 \
Makefile.am \
Makefile.in \
+ pma.h \
+ pma.c \
regcomp.c \
regex_internal.c \
regex_internal.h \
@@ -365,35 +378,13 @@ EXTRA_DIST = \
# what to make and install
noinst_LIBRARIES = libsupport.a
-libsupport_a_SOURCES = \
- attribute.h \
- cdefs.h \
- dfa.c \
- dfa.h \
- dynarray.h \
- flexmember.h \
- getopt.c \
- getopt.h \
- getopt1.c \
- getopt_int.h \
- idx.h \
- intprops.h \
- libc-config.h \
- localeinfo.c \
- localeinfo.h \
- random.c \
- random.h \
- regex.c \
- regex.h \
- verify.h \
- xalloc.h \
- malloc/dynarray.h \
- malloc/dynarray_at_failure.c \
- malloc/dynarray_emplace_enlarge.c \
- malloc/dynarray_finalize.c \
- malloc/dynarray_resize.c \
- malloc/dynarray_resize_clear.c
-
+libsupport_a_SOURCES = attribute.h cdefs.h dfa.c dfa.h dynarray.h \
+ flexmember.h getopt.c getopt.h getopt1.c getopt_int.h idx.h \
+ intprops.h libc-config.h localeinfo.c localeinfo.h random.c \
+ random.h regex.c regex.h verify.h xalloc.h malloc/dynarray.h \
+ malloc/dynarray_at_failure.c malloc/dynarray_emplace_enlarge.c \
+ malloc/dynarray_finalize.c malloc/dynarray_resize.c \
+ malloc/dynarray_resize_clear.c $(am__append_1)
# For some make's, e.g. OpenBSD, that don't define this
RM = rm -f
@@ -466,6 +457,7 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/getopt.Po@am__quote@ #
am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/getopt1.Po@am__quote@ #
am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/localeinfo.Po@am__quote@ #
am--include-marker
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pma.Po@am__quote@ #
am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/random.Po@am__quote@ #
am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/regex.Po@am__quote@ #
am--include-marker
@AMDEP_TRUE@@am__include@
@am__quote@malloc/$(DEPDIR)/dynarray_at_failure.Po@am__quote@ #
am--include-marker
@@ -625,6 +617,7 @@ distclean: distclean-am
-rm -f ./$(DEPDIR)/getopt.Po
-rm -f ./$(DEPDIR)/getopt1.Po
-rm -f ./$(DEPDIR)/localeinfo.Po
+ -rm -f ./$(DEPDIR)/pma.Po
-rm -f ./$(DEPDIR)/random.Po
-rm -f ./$(DEPDIR)/regex.Po
-rm -f malloc/$(DEPDIR)/dynarray_at_failure.Po
@@ -681,6 +674,7 @@ maintainer-clean: maintainer-clean-am
-rm -f ./$(DEPDIR)/getopt.Po
-rm -f ./$(DEPDIR)/getopt1.Po
-rm -f ./$(DEPDIR)/localeinfo.Po
+ -rm -f ./$(DEPDIR)/pma.Po
-rm -f ./$(DEPDIR)/random.Po
-rm -f ./$(DEPDIR)/regex.Po
-rm -f malloc/$(DEPDIR)/dynarray_at_failure.Po
diff --git a/support/agpl-3.0.txt b/support/agpl-3.0.txt
new file mode 100644
index 00000000..be3f7b28
--- /dev/null
+++ b/support/agpl-3.0.txt
@@ -0,0 +1,661 @@
+ GNU AFFERO GENERAL PUBLIC LICENSE
+ Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+ The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+ Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+ A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+ The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+ An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ TERMS AND CONDITIONS
+
+ 0. Definitions.
+
+ "This License" refers to version 3 of the GNU Affero General Public License.
+
+ "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+ "The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+ To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy. The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+ A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+ To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+ To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+ An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+ 1. Source Code.
+
+ The "source code" for a work means the preferred form of the work
+for making modifications to it. "Object code" means any non-source
+form of a work.
+
+ A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+ The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+ The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+ The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+ The Corresponding Source for a work in source code form is that
+same work.
+
+ 2. Basic Permissions.
+
+ All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+ You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force. You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright. Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+ Conveying under any other circumstances is permitted solely under
+the conditions stated below. Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+ No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+ When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+ 4. Conveying Verbatim Copies.
+
+ You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+ You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+ 5. Conveying Modified Source Versions.
+
+ You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+ a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+
+ b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under section
+ 7. This requirement modifies the requirement in section 4 to
+ "keep intact all notices".
+
+ c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+
+ d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+ A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+ 6. Conveying Non-Source Forms.
+
+ You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+ a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+
+ b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the
+ Corresponding Source from a network server at no charge.
+
+ c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+
+ d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+
+ e) Convey the object code using peer-to-peer transmission, provided
+ you inform other peers where the object code and Corresponding
+ Source of the work are being offered to the general public at no
+ charge under subsection 6d.
+
+ A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+ A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling. In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage. For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product. A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+ "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source. The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+ If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+ The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed. Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+ Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+ 7. Additional Terms.
+
+ "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+ When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+ Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+ a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+
+ b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+
+ c) Prohibiting misrepresentation of the origin of that material, or
+ requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+
+ d) Limiting the use for publicity purposes of names of licensors or
+ authors of the material; or
+
+ e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+
+ f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions of
+ it) with contractual assumptions of liability to the recipient, for
+ any liability that these contractual assumptions directly impose on
+ those licensors and authors.
+
+ All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+ If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+ Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+ 8. Termination.
+
+ You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+ However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+ Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+ Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+ 9. Acceptance Not Required for Having Copies.
+
+ You are not required to accept this License in order to receive or
+run a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+ 10. Automatic Licensing of Downstream Recipients.
+
+ Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+ An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+ You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+ 11. Patents.
+
+ A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+ A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+ Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+ In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+ If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+ If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+ A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License. You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+ Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+ 12. No Surrender of Others' Freedom.
+
+ If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all. For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+ 13. Remote Network Interaction; Use with the GNU General Public License.
+
+ Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software. This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+ Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+ 14. Revised Versions of this License.
+
+ The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time. Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+ If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+ Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+ 15. Disclaimer of Warranty.
+
+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. Limitation of Liability.
+
+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+ 17. Interpretation of Sections 15 and 16.
+
+ If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+ If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+ You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<https://www.gnu.org/licenses/>.
diff --git a/support/pma.c b/support/pma.c
new file mode 100644
index 00000000..da4d0b9a
--- /dev/null
+++ b/support/pma.c
@@ -0,0 +1,663 @@
+/* "pma", a persistent memory allocator (implementation)
+ Copyright (C) 2019, 2022 Terence Kelly
+ Contact: tpkelly @ { acm.org, cs.princeton.edu, eecs.umich.edu }
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+ Code is intended to be (mostly) C99 / POSIX 2017.
+
+ Compile and link with your programs in the obvious way. If
+ assertions are enabled (the default) extensive integrity checks
+ are frequently performed; these checks may be slow, depending on
+ the size and state of the persistent heap.
+*/
+
+#define _DEFAULT_SOURCE // for MAP_ANONYMOUS
+#include <assert.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <limits.h>
+#include <math.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include "pma.h"
+
+// Software version; not the same as backing file format version.
+const char pma_version[] = "2022.05May.01 (Avon 5) + gawk";
+
+#define S(s) #s
+#define S2(s) S(s)
+#define COORDS __FILE__ ":" S2(__LINE__) ": "
+#define FP(...) (void)fprintf(stderr, COORDS __VA_ARGS__)
+#define ERN " errno => '%s'\n", strerror(errno)
+#define ERR(...) do { if (0 < state.vrb) FP("ERROR: " __VA_ARGS__); } while
(0)
+#define WRN(...) do { if (1 < state.vrb) FP("Warning: " __VA_ARGS__); } while
(0)
+#define FYI(...) do { if (2 < state.vrb) FP("FYI: " __VA_ARGS__); } while
(0)
+
+int pma_errno;
+#define SE pma_errno = __LINE__
+#define RL return __LINE__
+#define RN return NULL
+#define SERL do { SE; RL; } while (0)
+#define SERN do { SE; RN; } while (0)
+
+enum {
+ VERS = 2, // backing file format version number
+ WDSZ = 8, // word size (bytes)
+ NFL = 422 // number of free lists / size classes
+};
+
+static int PGSZ; // can vary per system and even dynamically
+
+typedef struct ao { // alloc object
+ struct ao *anext, // header; singly linked list of all aos in addr
order
+ *fprev, *fnext; // for doubly linked free list
+} ao_t;
+
+// We stash three flags in the least significant bits of a header:
+// bit 0: is this ao in use (i.e., live, as opposed to free)?
+// bit 1: is the previous ao on the state.anext list in use?
+// bit 2: has this ao ever grown via realloc?
+
+// Some older compilers gratuitously reject the himask definition on
+// the first line below despite it being perfectly legal C99. The
+// workaround on the second line below appears correct by inspection
+// but it has not yet been subjected to the same extensive tests as
+// the original.
+// static const uintptr_t lomask = 0x7, himask = ~lomask; // requires recent
compiler
+static const uintptr_t lomask = 0x7, himask = ~ ( (uintptr_t) 0x7 ); // not
extensively tested as of Fri 27 May 2022
+
+// extract bits from header
+#define HIBH(ao_t_ptr) ((ao_t *)((uintptr_t)(ao_t_ptr) & himask))
+#define LOBH(ao_t_ptr) ((ao_t *)((uintptr_t)(ao_t_ptr) & lomask))
+
+// size of an ao is its memory footprint; capacity of ao is number of bytes
+// available to caller; difference is header overhead of live (in-use) ao
+#define AOSZ(ao_t_ptr) ((size_t)((uintptr_t)HIBH((ao_t_ptr)->anext) -
(uintptr_t)HIBH(ao_t_ptr)))
+#define AOCAP(ao_t_ptr) (AOSZ(ao_t_ptr) - WDSZ)
+
+#define Z1(i) assert(0 == (i) || 1 == (i))
+static void slobh(ao_t *p, int iu, int piu, int grown) { // set low bits of
header to which p points
+ uintptr_t h;
+ assert(p); Z1(iu); Z1(piu); Z1(grown);
+ h = (uintptr_t)p->anext;
+ h &= himask; // clear low bits
+ h |= (uintptr_t)(iu | piu << 1 | grown << 2);
+ p->anext = (ao_t *)h;
+}
+
+// getters below could be re-written as simpler macros
+
+static void globh(const ao_t *p, int *iu, int *piu, int *grown) { // get low
bits of header to which p points
+ uintptr_t h;
+ assert(p && iu && piu && grown);
+ h = (uintptr_t)p->anext;
+ *iu = !!(h & 0x1); Z1(*iu);
+ *piu = !!(h & 0x2); Z1(*piu);
+ *grown = !!(h & 0x4); Z1(*grown);
+}
+
+typedef struct { // header of backing file; contains allocator metadata
+ void * mapaddr; // virtual address where backing file must be mapped
+ uint64_t bf_vers, // version number of backing file format
+ nallocs, // number of allocations
+ nfrees, // number of de-allocations
+ res_0; // reserved for possible future use
+ void * root; // live persistent data must be reachable from root
+ ao_t * afirst, // list of all alloc objects, both live & free, in addr
order
+ * abound; // one beyond end of allocatable area
+ ao_t free[NFL]; // free lists; each has dummy header
+} pma_hdr_t;
+
+static struct {
+ int init, // has persistent heap been initialized?
+ vrb; // verbsity level
+ const char * file; // name of backing file
+ pma_hdr_t * hdr; // addr where backing file is mapped
+} state;
+
+#define ASI assert(1 == state.init || 2 == state.init)
+
+enum { IU = 0, PIU = 1, GROWN = 2 };
+
+static int getbit(ao_t *p, int bit) {
+ int iu, piu, grown;
+ globh(p, &iu, &piu, &grown);
+ switch (bit) {
+ case IU: return iu;
+ case PIU: return piu;
+ case GROWN: return grown;
+ default:
+ ERR("bad bit: %d\n", bit);
+ assert(0);
+ return INT_MIN;
+ }
+}
+
+#define DP(...) (void)fprintf(stderr, __VA_ARGS__)
+#define VS (void *)
+
+#ifndef NDEBUG
+static int valid_footer(ao_t *p) {
+ if (!getbit(p, IU)) {
+ ao_t **ftr = (ao_t **)HIBH(p->anext) - 1;
+ return *ftr == p; // footer should point to header
+ }
+ else
+ return 1;
+}
+// valid ao ptr:
+#define VAO(p) (VS state.hdr->afirst <= VS (p) && VS state.hdr->abound > VS
(p))
+#define VAF(p) (VAO(p) && valid_footer(p))
+#endif // NDEBUG
+
+static void pao(ao_t *p) { // print ao
+ int iu, piu, grown;
+ ao_t **footer = (ao_t **) ((char *)p + AOSZ(p)) - 1; // TODO: squelch
alignment warning?
+ assert(VAO(p));
+ assert(0 == AOSZ(p) % WDSZ);
+ assert(0 == LOBH(p));
+ globh(p, &iu, &piu, &grown);
+ DP(" AO at %p: size %lu B / %lu w\n"
+ " hdr %p (H 0%lo L 0%lo) iu %d piu %d grown %d\n"
+ " fp %p%s\n"
+ " fn %p%s\n"
+ " ft %p%s\n",
+ VS p, AOSZ(p), AOSZ(p) / WDSZ,
+ VS p->anext, (uintptr_t)HIBH(p->anext), (uintptr_t)LOBH(p->anext), iu,
piu, grown,
+ VS p->fprev, iu ? " (ignore)" : "",
+ VS p->fnext, iu ? " (ignore)" : "",
+ VS *footer, iu ? " (ignore)" : "");
+}
+
+static size_t UB[NFL]; // UB[c] is upper bound of size class c in machine
words
+
+#define IC integrity_check(__LINE__)
+static int integrity_check(int line) { // can be slow; intended for debugging
small heaps
+ pma_hdr_t *h = state.hdr;
+ int nadd = 0, naddfree = 0, tiu = 0, tpiu = 0, tf = 0; // beware overflow
if lists are long
+ FYI("integrity_check called from line %d\n", line);
+#ifdef NDEBUG
+ WRN("integrity check relies on assertions, which are disabled (call at line
%d)\n", line);
+#endif
+ assert(h && h == h->mapaddr);
+ // check addr-ordered list
+ for (ao_t *next, *a = h->afirst; a < h->abound; a = next) {
+ next = HIBH(a->anext);
+ assert(VAF(a));
+ assert(32 <= AOSZ(a));
+ assert(next > a && next <= h->abound);
+ if (1000000 < ++nadd) {
+ WRN("integrity check discontinued; anext list too long (call at line
%d)\n", line);
+ SE; // if persistent heap contains too many aos/blocks,
+ return 0; // integrity check will be too slow; therefore discontinue
+ }
+ if (0 == getbit(a, IU))
+ ++naddfree;
+ tiu += getbit(a, IU); // total number of aos with in-use bit set
+ tpiu += getbit(a, PIU); // total number of aos with previous-ao-is-in-use
bit set
+ }
+ assert(tpiu == tiu || 1 + tpiu == tiu);
+ FYI("anext list length: %d tiu %d tpiu %d nallocs %lu nfrees %lu\n",
+ nadd, tiu, tpiu, h->nallocs, h->nfrees);
+ assert(h->nallocs >= h->nfrees);
+ assert(h->nallocs - h->nfrees == (uint64_t)tiu);
+ // check free lists
+ for (int i = 0; i < NFL; i++) {
+ ao_t *p, *f = &(h->free[i]);
+ if (f->fprev == f) // empty list
+ assert(f->fnext == f);
+ else {
+ int nfwd = 0, nrev = 0; // count how many we find going forward and
reverse
+ for (p = f->fnext; p != f; p = p->fnext) { nfwd++; assert(VAF(p));
assert(0 == getbit(p, IU)); }
+ for (p = f->fprev; p != f; p = p->fprev) { nrev++; assert(VAF(p));
assert(0 == getbit(p, IU)); }
+ assert(nfwd == nrev); // count should be the same in both directions
+ tf += nfwd;
+ // check ao sizes against UB
+ for (p = f->fnext; p != f; p = p->fnext) {
+ #ifndef NDEBUG
+ size_t capwords = AOCAP(p) / WDSZ;
+ #endif
+ assert(capwords <= UB[i]);
+ if (0 < i)
+ assert(capwords > UB[i-1]);
+ }
+ }
+ }
+ FYI("total free aos: %d naddfree %d integrity check line %d\n", tf,
naddfree, line);
+ assert(tf == naddfree);
+ return 0;
+}
+
+#define NM " not meaningful in fallback mode\n"
+
+void pma_check_and_dump(void) {
+ pma_hdr_t *h = state.hdr;
+ ASI;
+ if (2 == state.init) { ERR("check_and_dump" NM); assert(0); SE; return; }
+ if (IC) ERR("integrity check failed\n");
+ DP(COORDS "check data structures and dump\n");
+ DP("header version: %s\n", PMA_H_VERSION);
+ DP("software version: %s\n", pma_version);
+ DP("sizeof state: %lu\n", sizeof state);
+ DP("sizeof header: %lu\n", sizeof(pma_hdr_t));
+ DP("state:\n");
+ DP(" init: %d\n", state.init); assert(0 == state.init || 1 == state.init
|| 2 == state.init);
+ DP(" vrb: %d\n", state.vrb); assert(0 <= state.vrb && 3 >= state.vrb);
+ DP(" file: %p \"%s\"\n", (const void *)state.file, state.file);
+ DP(" hdr: %p\n", VS h);
+ if (NULL != h) {
+ DP("header:\n"); // nallocs & nfrees not printed; they'd add noise to
initial/final state diff
+ DP(" mapaddr: %p\n", h->mapaddr);
+ DP(" bf_vers: %" PRIu64 "\n", h->bf_vers);
+ DP(" root: %p\n", h->root); assert(NULL == h->root || (h->root >= VS
h->afirst && h->root < VS h->abound));
+ DP(" afirst: %p\n", VS h->afirst); assert(VS h->afirst > h->mapaddr);
+ DP(" abound: %p\n", VS h->abound); assert(h->abound > h->afirst);
+ DP(" all allocated objects in addr order:\n");
+ for (ao_t *a = h->afirst; a < h->abound; a = HIBH(a->anext)) {
+ assert(HIBH(a->anext) > a);
+ pao(a);
+ }
+ for (int i = 0; i < NFL; i++) {
+ ao_t *f = &(h->free[i]);
+ if (f->fprev == f) // empty list
+ assert(f->fnext == f);
+ else {
+ DP(" free list of size class %d UB %lu (prev %lu) list head %p:\n",
i, UB[i], i > 0 ? UB[i-1] : 0, VS f);
+ for (ao_t *p = f->fnext; p != f; p = p->fnext)
+ pao(p);
+ }
+ }
+ }
+}
+#undef DP
+
+static int sc(size_t nbytes) { // compute size class; nbytes is malloc() arg
+ static int init;
+ const size_t maxwords = (size_t)0x1 << 61; // 2^61 words == 2^64 bytes
+ size_t nwords;
+ int c;
+ if (! init) {
+ FYI("initializing UB[]\n");
+ UB[0] = 3; // smallest allocation has size (AOSZ) 4 words and capacity
(AOCAP) 3 words
+ for (c = 1; ; c++) {
+ long double hi = floorl((long double)(1 + UB[c-1]) * 1.1L);
+ assert(NFL > c);
+ if (hi >= (long double)maxwords) { UB[c] = maxwords; break; }
+ else UB[c] = (size_t)hi;
+ }
+ assert(NFL - 1 == c);
+ init = 1;
+ }
+ nwords = (nbytes / WDSZ) + !!(nbytes % WDSZ);
+ assert(nwords <= maxwords);
+ for (c = 0; NFL > c; c++)
+ if (nwords <= UB[c])
+ break;
+ assert(NFL > c);
+ return c;
+}
+
+static void fli(ao_t *p) { // insert given ao at head of appropriate free list
+ ao_t *h;
+ int fl;
+ assert(NULL != p);
+ assert(VAO(p));
+ fl = sc(AOCAP(p));
+ assert(0 <= fl && NFL > fl);
+ h = &(state.hdr->free[fl]); // head of free list
+ FYI("fli(%p) h == %p h->fn %p h->fp %p\n", VS p, VS h, VS h->fnext, VS
h->fprev);
+ p->fprev = h;
+ p->fnext = h->fnext;
+ h->fnext->fprev = p;
+ h->fnext = p;
+}
+
+static void flr(ao_t *p) { // remove ao from free list
+ p->fnext->fprev = p->fprev;
+ p->fprev->fnext = p->fnext;
+ p->fprev = p->fnext = NULL;
+}
+
+#define MMAP(N) mmap(NULL, (N), PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS |
MAP_NORESERVE, -1, 0)
+#define MUNMAP(A, N) do { if (0 != munmap((A), (N))) ERR("mmap()" ERN); }
while (0)
+static void * addrgap(off_t n) { // find big gap in address space to map n
bytes
+ void *A, *Amax = NULL; size_t L = 0, U, Max = 0, N = (size_t)n; char *r;
+ FYI("addrgap(%ld)\n", n);
+ if (N % (size_t)PGSZ) { ERR("file size %lu not multiple of PGSZ\n", N);
SERN; }
+ if (N < sizeof(pma_hdr_t) + 10 * PGSZ) { ERR("file size %lu too small\n",
N); SERN; }
+ for (U = 1; ; U *= 2) // double upper bound until failure
+ if (MAP_FAILED == (A = MMAP(U))) break;
+ else MUNMAP(A, U);
+ while (1 + L < U) { // binary search between bounds
+ size_t M = L + (U - L) / 2; // avoid overflow
+ if (MAP_FAILED == (A = MMAP(M))) { U = M; }
+ else { Amax = A; Max = M; MUNMAP(A, M); L = M; }
+ }
+ FYI("max gap: %lu bytes at %p\n", Max, Amax);
+ if (Max < N + (size_t)PGSZ * 2) { // safety margin
+ ERR("max gap %lu too small for required %lu\n", Max, N);
+ SERN;
+ }
+ r = (char *)Amax + (Max - N)/2;
+ while ((uintptr_t)r % PGSZ) // page align
+ r++;
+ FYI("addrgap returns %p == %lu\n", VS r, (uintptr_t)r);
+ return r;
+}
+#undef MMAP
+#undef MUNMAP
+
+#define MM(a,s,f) mmap((a), (size_t)(s), PROT_READ | PROT_WRITE, MAP_SHARED,
(f), 0)
+int pma_init(int verbose, const char *file) {
+ int fd; void *a1, *a2; size_t as = sizeof(a1); struct stat s;
+ pma_hdr_t *h;
+ assert(0 <= verbose && 3 >= verbose);
+ state.vrb = verbose;
+ if (state.init) { ERR("already initialized\n"); assert(0); SERL; }
+ FYI("pma software version \"%s\" header version \"%s\", expects backing file
format version %d\n",
+ pma_version, PMA_H_VERSION, VERS);
+ assert(0 == strcmp(pma_version, PMA_H_VERSION));
+ FYI("pma_init(%d,\"%s\")\n", verbose, file);
+ // check assumptions
+ assert(WDSZ == sizeof(void *)); // in C11 we'd static_assert()
+ assert(WDSZ == sizeof(size_t));
+ assert(WDSZ == sizeof(unsigned long));
+ assert(0 == sizeof(pma_hdr_t) % WDSZ);
+
+ PGSZ = sysconf(_SC_PAGESIZE);
+ if (NULL == file) {
+ WRN("no backing file provided; falling back on standard malloc\n");
+ state.init = 2;
+ state.file = NULL;
+ state.hdr = NULL;
+ return 0;
+ }
+ // map backing file containing persistent heap
+ if (0 > (fd = open(file, O_RDWR))) { ERR("open()" ERN); SERL; }
+ if (0 != fstat(fd, &s)) { ERR("fstat()" ERN); SERL; }
+ if (!S_ISREG(s.st_mode)) { ERR("not reg\n" ); SERL; }
+ if ((ssize_t)as != read(fd, &a1, as)) { ERR("read()" ERN); SERL; }
+ if (NULL == a1) a1 = addrgap(s.st_size);
+ if (NULL == a1) { ERR("addrgap()" ERN); RL; }
+ FYI("map at %p\n", a1);
+ if (a1 != (a2 = MM(a1, s.st_size, fd))) { ERR("mmap()" ERN); SERL; }
+ if (0 != close(fd)) { ERR("close()" ERN); SE; } //
carry on
+ state.init = 1;
+ state.file = file;
+ state.hdr = h = (pma_hdr_t *)a2;
+ if (NULL == h->mapaddr) {
+ int i;
+ ao_t *w, **ftr; // initial "wilderness", footer
+ FYI("initializing persistent heap\n");
+ assert( 0 == h->bf_vers
+ && 0 == h->nallocs
+ && 0 == h->nfrees
+ && 0 == h->res_0
+ && NULL == h->root
+ && NULL == h->afirst
+ && NULL == h->abound);
+ for (i = 0; i < NFL; i++) {
+ assert( NULL == h->free[i].anext
+ && NULL == h->free[i].fprev
+ && NULL == h->free[i].fnext);
+ // free[i] is dummy node in doubly-linked free list; this simplifies
code to splice aos into & out of free list
+ h->free[i].fprev = h->free[i].fnext = &(h->free[i]);
+ }
+ h->mapaddr = h;
+ h->bf_vers = VERS;
+ h->nallocs = 0;
+ h->nfrees = 0;
+ h->res_0 = 0;
+ h->afirst = (ao_t *)(1 + h);
+ h->abound = (ao_t *)((char *)h + s.st_size); // TODO: squelch alignment
warning?
+ // install "wilderness" free ao on all-object list; every free ao has a
+ // footer pointing to its header, to facilitate coalescing adjacent frees;
+ // header on final ao points beyond allocatable area
+ w = h->afirst;
+ w->anext = h->abound;
+ assert(0 == AOSZ(w) % WDSZ && WDSZ < AOSZ(w));
+ ftr = (ao_t **)h->abound - 1;
+ *ftr = w;
+ fli(w);
+ }
+ else {
+ FYI("persistent heap already initialized\n");
+ if (VERS != h->bf_vers) {
+ ERR("backing file version mismatch: %d vs. %" PRIu64 "\n", VERS,
h->bf_vers);
+ SERL;
+ }
+ (void)sc(1); // to populate UB[]
+ }
+ if (IC) { ERR("integrity check failed\n"); SERL; }
+ return 0;
+}
+#undef MM
+
+// bite off adequate prefix if ao is too large, return remainder to appropriate
+// free list; must handle two cases: given ao is free, versus given ao is live
+// and is being shrunk by realloc
+static ao_t * split_ao(ao_t *p, size_t s) {
+ size_t c = AOCAP(p), Cw = c / WDSZ, Sw; int iu, piu, grown; ao_t *n;
+ if (s < 24) s = 24; // increase request size to minimum allocation size if
necessary; TODO: magic number here
+ Sw = s / WDSZ + !!(s % WDSZ);
+ assert(NULL == LOBH(p)); // lo bits of header (p->anext) might be set, but
not lo bits of p
+ assert(NULL == p->fprev && NULL == p->fnext); // *p should already be
spliced out of free lists
+ assert(c >= s && 0 == c % WDSZ);
+ FYI("split_ao(%p,%lu) AOCAP %lu words req %lu words cap %lu\n", VS p, s, c,
Sw, Cw);
+ globh(p, &iu, &piu, &grown);
+ if (4 <= Cw - Sw) { // split ao if remainder is large enough to be
allocatable
+ ao_t *rem = (ao_t *)(&(p->fprev) + Sw), // remainder
+ **rft = (ao_t **)HIBH(p->anext) - 1; // footer of remainder
+ FYI("splitting at %p\n", VS rem);
+ rem->anext = HIBH(p->anext); // set header of remainder
+ *rft = rem; // set footer of remainder
+ fli(rem);
+ p->anext = rem; // set header of *p
+ }
+ assert(AOCAP(p) >= s);
+ slobh(p, 1, piu, grown); // in-use bit is set; values of prev-in-use and
grown bits are preserved
+ // set prev-in-use bit of next ao on anext list (if such exists), preserving
iu and grown bits
+ n = HIBH(p->anext);
+ assert(n <= state.hdr->abound);
+ if (n < state.hdr->abound) {
+ globh(n, &iu, &piu, &grown);
+ slobh(n, iu, 1, grown);
+ }
+ return p;
+}
+
+void * pma_malloc(size_t size) {
+ ao_t *r = NULL;
+ FYI("malloc(%lu)\n", size);
+ ASI;
+ if (2 == state.init) return malloc(size);
+ assert(!IC);
+ if (0 >= size) {
+ WRN("malloc(%lu) argument <= zero\n", size); SERN; }
+ for (int c = sc(size); c < NFL; c++) {
+ ao_t *h = &(state.hdr->free[c]);
+ // FYI("check size class %d UB %lu\n", c, UB[c]);
+ for (ao_t *f = h->fnext; f != h; f = f->fnext) {
+ // FYI("free list contains ao with capacity %lu\n", AOCAP(f));
+ if (AOCAP(f) >= size) {
+ r = f;
+ goto end;
+ }
+ }
+ }
+ end:
+ if (NULL != r) {
+ flr(r);
+ r = split_ao(r, size);
+ FYI("malloc returning %p\n", VS &(r->fprev));
+ ++(state.hdr->nallocs);
+ assert(!IC);
+ return &(r->fprev);
+ }
+ else {
+ WRN("malloc(%lu) cannot satisfy request at this time\n", size);
+ SERN; // conflates ENOMEM / EAGAIN (request might succeed after frees)
+ }
+}
+
+void * pma_calloc(size_t nmemb, size_t size) {
+ void *p;
+ FYI("calloc(%lu,%lu)\n", nmemb, size);
+ ASI;
+ if (2 == state.init) return calloc(nmemb, size);
+ if (0 >= nmemb || 0 >= size) {
+ WRN("calloc(%lu,%lu) argument <= zero\n", nmemb, size); SERN; }
+ // SSIZE_MAX exists but SIZE_MAX apparently doesn't; sheesh
+ if (nmemb > UINT64_MAX / size) {
+ WRN("calloc(%lu,%lu) arguments overflow\n", nmemb, size); SERN; }
+ if (NULL != (p = pma_malloc(nmemb * size)))
+ (void)memset(p, 0, nmemb * size);
+ return p;
+}
+
+void * pma_realloc(void *ptr, size_t size) {
+ ao_t *p; void *nu; // "new" is a C++ keyword
+ FYI("realloc(%p,%lu)\n", ptr, size);
+ ASI;
+ if (2 == state.init) return realloc(ptr, size);
+ if (NULL == ptr) return pma_malloc(size);
+ if (0 >= size) { pma_free(ptr); RN; }
+ p = (ao_t *)((ao_t **)ptr - 1);
+ if (AOCAP(p) >= size) // no growth needed
+ return ptr;
+ nu = pma_malloc(size);
+ if (NULL == nu)
+ SERN;
+ (void)memcpy(nu, ptr, AOCAP(p));
+ pma_free(ptr);
+ return nu;
+}
+
+// Merge *p with next ao on anext list; returns 1 if coalesces, 0 otherwise.
+// Flag indicates which of the two aos is on the free list and must be removed.
+static int coalesce(ao_t *p, int flr_lo_hi) {
+ ao_t *n = HIBH(p->anext), **ftr; int piu;
+ assert(n > p && n <= state.hdr->abound);
+ FYI("coalesce(%p)\n", VS p);
+ if (n >= state.hdr->abound) // *p is final ao on anext list
+ return 0;
+ if (0 == getbit(n, IU)) { // next ao is not in use, therefore coalesce
+ flr(flr_lo_hi ? n : p); // remove appropriate ao from free list
+ piu = getbit(p, PIU);
+ p->anext = HIBH(n->anext);
+ ftr = (ao_t **)p->anext - 1;
+ *ftr = p;
+ slobh(p, 0, piu, 0); // preserve prev-in-use bit of header of *p
+ return 1;
+ }
+ else // next ao is in use, therefore do not coalesce
+ return 0;
+}
+
+void pma_free(void *ptr) {
+ ao_t *p, *n, **ftr; int r;
+ FYI("free(%p)\n", ptr);
+ ASI;
+ if (2 == state.init) { free(ptr); return; }
+ assert(!IC);
+ if (NULL == ptr) return; // allowed by C & POSIX
+ if (! (VS state.hdr->afirst <= ptr && VS state.hdr->abound > ptr)) { //
e.g., p=strdup("foo") ... pma_free(p);
+ ERR("freed ptr %p outside allocatable area bounds %p %p\n",
+ ptr, VS state.hdr->afirst, VS state.hdr->abound);
+ assert(0);
+ SE;
+ return;
+ }
+ assert(0 == (uintptr_t)ptr % WDSZ);
+ p = (ao_t *)((ao_t **)ptr - 1);
+ assert(VAO(p));
+ n = HIBH(p->anext);
+ assert(p < n && n <= state.hdr->abound);
+ assert(1 == getbit(p, IU));
+ slobh(p, 0, getbit(p, PIU), 0); // clear iu and grown bits of *p header
+ if (n < state.hdr->abound)
+ assert(1 == getbit(n, PIU));
+ FYI("merge with right/higher ao\n");
+ r = coalesce(p, 1);
+ FYI("%s\n", r ? "yes" : "no");
+ if (0 == getbit(p, PIU) && p > state.hdr->afirst) { // if ao is not
leftmost/lowest and previous/lower ao is not in use, merge
+ ao_t *prev = *((ao_t **)p - 1);
+ assert(prev < p);
+ FYI("merge with left/lower ao\n");
+ r = coalesce(prev, 0);
+ assert(r);
+ p = prev;
+ }
+#ifndef NDEBUG
+ (void)memset(&(p->fprev), 0, AOCAP(p)); // for near-complete "reversibility"
+#endif
+ n = HIBH(p->anext);
+ assert(p < n && n <= state.hdr->abound);
+ ftr = (ao_t **)n - 1;
+ *ftr = p;
+ if (n < state.hdr->abound)
+ slobh(n, getbit(n, IU), 0, getbit(n, GROWN)); // clear prev-in-use bit of
next
+ fli(p);
+ ++(state.hdr->nfrees);
+ assert(!IC);
+ }
+
+void pma_set_root(void *p) {
+ FYI("set_root(%p)\n", p);
+ ASI;
+ if (2 == state.init) { ERR("set_root" NM); assert(0); SE; return; }
+ assert(NULL == p || VAO(p)); // could also check that p "looks like"
pointer returned by malloc,
+ state.hdr->root = p; // e.g., header's in-use bit should be set and
HIBH should be reasonable
+}
+
+void * pma_get_root(void) {
+ void *p;
+ FYI("get_root()\n");
+ ASI;
+ if (2 == state.init) { ERR("get_root" NM); assert(0); SERN; }
+ p = state.hdr->root;
+ assert(NULL == p || VAO(p));
+ return p;
+}
+
+typedef unsigned long ul_t;
+void pma_set_avail_mem(const ul_t v) {
+ FYI("set_avail_mem(0x%lx)\n", v);
+ ASI;
+ if (2 == state.init) { ERR("set_avail_mem" NM); assert(0); SE; return; }
+ assert(!IC);
+ for (int i = 0; i < NFL; i++) {
+ ao_t *p, *f = &(state.hdr->free[i]);
+ if (f->fprev != f)
+ for (p = f->fnext; p != f; p = p->fnext) {
+ ul_t *q = (ul_t *)(1 + p),
+ *e = (ul_t *)(HIBH(p->anext)) - 1;
+ for ( ; q != e; q++)
+ if (*q != v) // avoid over-writing same value; gratuitous
modification is a Bad Thing for persistent memory
+ *q = v;
+ for ( ; q != e; q++)
+ assert(*q == v);
+ }
+ }
+ assert(!IC);
+}
+
diff --git a/support/pma.h b/support/pma.h
new file mode 100644
index 00000000..f5851d96
--- /dev/null
+++ b/support/pma.h
@@ -0,0 +1,73 @@
+/* "pma", a persistent memory allocator (interface)
+ Copyright (C) 2022 Terence Kelly
+ Contact: tpkelly @ { acm.org, cs.princeton.edu, eecs.umich.edu }
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
+*/
+
+#ifndef PMA_H_INCLUDED
+#define PMA_H_INCLUDED
+
+// version strings of interface and implementation should match
+#define PMA_H_VERSION "2022.05May.01 (Avon 5) + gawk"
+extern const char pma_version[];
+
+/* May contain line number in pma.c where something went wrong if one
+ of the other interfaces fails. "Use the Source, Luke." */
+extern int pma_errno;
+
+/* Must be called exactly once before any other pma_* function is
+ called, otherwise behavior is undefined. Available verbosity
+ levels are: 0 => no diagnostic printouts, 1 => errors only printed
+ to stderr, 2 => also warnings, 3 => also very verbose "FYI"
+ messages. Use verbosity level 2 by default. File argument
+ specifies backing file containing persistent heap, initially an
+ empty sparse file whose size is a multiple of the system page
+ size. If file argument is NULL, fall back on standard memory
+ allocator: pma_malloc passes the buck to ordinary malloc, pma_free
+ calls ordinary free, etc. In fallback-to-standard-malloc mode,
+ only pma_malloc/calloc/realloc/free may be used; other functions
+ such as pma_get/set_root must not be used. The implementation may
+ store a copy of the file argument, i.e., the pointer, so both this
+ pointer and the memory to which it points (*file) must not
+ change. */
+extern int pma_init(int verbose, const char *file);
+
+/* The following four functions may be used like their standard
+ counterparts. See notes on pma_init for fallback to standard
+ allocator. See notes on fork in README. */
+extern void * pma_malloc(size_t size);
+extern void * pma_calloc(size_t nmemb, size_t size);
+extern void * pma_realloc(void *ptr, size_t size);
+extern void pma_free(void *ptr);
+
+/* The following two functions access the persistent heap's root
+ pointer, which must either be NULL or must point within a block of
+ persistent memory. If the pointer passed to pma_set_root is not a
+ pointer returned by pma_malloc/calloc/realloc, that is likely a
+ bug, though the implementation is not obliged to check for such
+ bugs. */
+extern void pma_set_root(void *ptr);
+extern void * pma_get_root(void);
+
+/* Prints to stderr details of internal data structures, e.g., free
+ lists, and performs integrity checks, which may be very slow. */
+extern void pma_check_and_dump(void);
+
+/* Set non-metadata words of free blocks to given value. Useful for
+ debugging (v == 0xdeadDEADbeefBEEF) and for preparing backing file
+ to be re-sparsified with fallocate (v == 0x0). */
+extern void pma_set_avail_mem(const unsigned long v);
+
+#endif // PMA_H_INCLUDED
diff --git a/symbol.c b/symbol.c
index 78b29bba..5ac5656d 100644
--- a/symbol.c
+++ b/symbol.c
@@ -55,9 +55,20 @@ static bool installing_specials = false;
void
init_symbol_table()
{
- getnode(global_table);
- memset(global_table, '\0', sizeof(NODE));
- null_array(global_table);
+ NODE *pma_root_node = NULL;
+
+ if (using_persistent_malloc) {
+ pma_root_node = (NODE *) pma_get_root();
+ if (pma_root_node != NULL) {
+ global_table = pma_root_node;
+ }
+ }
+
+ if (global_table == NULL) {
+ getnode(global_table);
+ memset(global_table, '\0', sizeof(NODE));
+ null_array(global_table);
+ }
getnode(param_table);
memset(param_table, '\0', sizeof(NODE));
@@ -66,8 +77,15 @@ init_symbol_table()
installing_specials = true;
func_table = install_symbol(estrdup("FUNCTAB", 7), Node_var_array);
- symbol_table = install_symbol(estrdup("SYMTAB", 6), Node_var_array);
+ if (using_persistent_malloc && pma_root_node != NULL)
+ symbol_table = lookup("SYMTAB");
+ else
+ symbol_table = install_symbol(estrdup("SYMTAB", 6),
Node_var_array);
+
installing_specials = false;
+
+ if (using_persistent_malloc && pma_root_node == 0)
+ pma_set_root(global_table);
}
/*
diff --git a/test/Makefile.in b/test/Makefile.in
index c031daf7..2673a17b 100644
--- a/test/Makefile.in
+++ b/test/Makefile.in
@@ -120,8 +120,9 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \
$(top_srcdir)/m4/lib-prefix.m4 $(top_srcdir)/m4/libsigsegv.m4 \
$(top_srcdir)/m4/longlong.m4 $(top_srcdir)/m4/mpfr.m4 \
$(top_srcdir)/m4/nls.m4 $(top_srcdir)/m4/noreturn.m4 \
- $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \
- $(top_srcdir)/m4/readline.m4 $(top_srcdir)/m4/socket.m4 \
+ $(top_srcdir)/m4/pma.m4 $(top_srcdir)/m4/po.m4 \
+ $(top_srcdir)/m4/progtest.m4 $(top_srcdir)/m4/readline.m4 \
+ $(top_srcdir)/m4/socket.m4 \
$(top_srcdir)/m4/triplet-transformation.m4 \
$(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 44 +
Makefile.am | 3 +-
Makefile.in | 11 +-
NEWS | 5 +
aclocal.m4 | 1 +
awk.h | 1 +
awklib/ChangeLog | 5 +
awklib/Makefile.am | 2 +-
awklib/Makefile.in | 7 +-
configh.in | 3 +
configure | 84 ++
configure.ac | 12 +
custom.h | 13 +
doc/ChangeLog | 29 +
doc/Makefile.in | 5 +-
doc/awkcard.in | 23 +-
doc/gawk.1 | 10 +-
doc/gawk.info | 1374 ++++++++++++++++-------------
doc/gawk.texi | 175 +++-
doc/gawktexi.in | 175 +++-
doc/wordlist | 24 +
doc/wordlist2 | 1 +
doc/wordlist5 | 1 +
extras/Makefile.in | 5 +-
m4/ChangeLog | 22 +
m4/pma.m4 | 51 ++
main.c | 33 +-
support/ChangeLog | 15 +
support/Makefile.am | 6 +
support/Makefile.in | 62 +-
extension/COPYING => support/agpl-3.0.txt | 153 ++--
support/pma.c | 663 ++++++++++++++
support/pma.h | 73 ++
symbol.c | 26 +-
test/Makefile.in | 5 +-
35 files changed, 2351 insertions(+), 771 deletions(-)
create mode 100644 m4/pma.m4
copy extension/COPYING => support/agpl-3.0.txt (86%)
create mode 100644 support/pma.c
create mode 100644 support/pma.h
hooks/post-receive
--
gawk
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [SCM] gawk branch, master, updated. gawk-4.1.0-4748-g2ae439f3,
Arnold Robbins <=