[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
strstr speedup
|
From: |
Eric Blake |
|
Subject: |
strstr speedup |
|
Date: |
Thu, 10 Jan 2008 21:44:57 +0000 (UTC) |
|
User-agent: |
Loom/3.14 (http://gmane.org/) |
I'm committing this series to add a strstr module similar to memmem (in
testing, every system that I have easy access to currently has a worst-case
quadratic strstr: mingw, cygwin, glibc, Solaris, OpenBSD; however, there is
hope that I will be successful in getting glibc and cygwin to use this patch).
I split the patch into two parts, so that after the first commit, you can
compare the similarity of memmem.c and strstr.c to make it more obvious how I
then factored that into str-two-way.h. The implementation of str-two-way.h
leaves the door wide open for byte-wise case-insensitive searching (such as
strcasestr), although I did not do that work yet. Although several gnulib
modules use strstr, only unictype/gen-ctype.c uses an arbitrary needle; and
since that file is intended more as a helper application than a library
function, I did not add any dependencies to the new strstr module. Note that
while memmem can acheive sublinear speed, strstr cannot, because it must
perform a linear search for the trailing NUL of the haystack; but the idea of a
shift table in two_way_long_needle still allows N + N/M rather than 3*N
comparisons, which can give noticeable speedups.
I'm also thinking that we do not need the c_strstr module - aside from its
comments on when it is safe to use a bytewise search even in a multibyte
locale, it behaves no differently than the POSIX specification of strstr.
However, I can see having all three of strcasestr, c-strcasestr, and
mbscasestr, since it is conceivable to want single-byte locale-dependent case
insensitivity in strcasestr which differs from the locale-independent case-
insensitivity of c_strstr or the multibyte-safe mbscasestr.
>From 96facb24aa17b8d412d2f37fa91d59282d950ebb Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Thu, 10 Jan 2008 07:38:11 -0700
Subject: [PATCH] Avoid quadratic strstr implementations.
* lib/strstr.c: New file.
* m4/strstr.m4: Likewise.
* modules/strstr: Likewise.
* modules/strstr-tests: Likewise.
* tests/test-strstr.c: Likewise.
* lib/string.in.h (rpl_strstr): Declare.
(memmem) [GNULIB_POSIXCHECK]: Document speed issue.
* m4/string_h.m4 (gl_HEADER_STRING_H_DEFAULTS): Support strstr.
* modules/string (Makefile.am): Likewise.
* MODULES.html.sh (string handling): Mention new module.
* doc/functions/strstr.texi (strstr): Document the bug.
Signed-off-by: Eric Blake <address@hidden>
---
ChangeLog | 15 ++
MODULES.html.sh | 1 +
doc/functions/strstr.texi | 6 +-
lib/string.in.h | 21 ++-
lib/strstr.c | 437 +++++++++++++++++++++++++++++++++++++++++++++
m4/string_h.m4 | 6 +-
m4/strstr.m4 | 48 +++++
modules/string | 2 +
modules/strstr | 27 +++
modules/strstr-tests | 12 ++
tests/test-strstr.c | 155 ++++++++++++++++
11 files changed, 721 insertions(+), 9 deletions(-)
create mode 100644 lib/strstr.c
create mode 100644 m4/strstr.m4
create mode 100644 modules/strstr
create mode 100644 modules/strstr-tests
create mode 100644 tests/test-strstr.c
diff --git a/ChangeLog b/ChangeLog
index 900413d..44df277 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2008-01-10 Eric Blake <address@hidden>
+
+ Avoid quadratic strstr implementations.
+ * lib/strstr.c: New file.
+ * m4/strstr.m4: Likewise.
+ * modules/strstr: Likewise.
+ * modules/strstr-tests: Likewise.
+ * tests/test-strstr.c: Likewise.
+ * lib/string.in.h (rpl_strstr): Declare.
+ (memmem) [GNULIB_POSIXCHECK]: Document speed issue.
+ * m4/string_h.m4 (gl_HEADER_STRING_H_DEFAULTS): Support strstr.
+ * modules/string (Makefile.am): Likewise.
+ * MODULES.html.sh (string handling): Mention new module.
+ * doc/functions/strstr.texi (strstr): Document the bug.
+
2008-01-10 Bruno Haible <address@hidden>
* lib/relocatable.h (relocate): State whether result is freshly
diff --git a/MODULES.html.sh b/MODULES.html.sh
index 66e0b01..1e6f16b 100755
--- a/MODULES.html.sh
+++ b/MODULES.html.sh
@@ -1671,6 +1671,7 @@ func_all_modules ()
func_module strnlen1
func_module strndup
func_module strsep
+ func_module strstr
func_module c-strstr
func_module trim
func_module fstrcmp
diff --git a/doc/functions/strstr.texi b/doc/functions/strstr.texi
index 99b3630..fe0af49 100644
--- a/doc/functions/strstr.texi
+++ b/doc/functions/strstr.texi
@@ -4,10 +4,14 @@
POSIX specification: @url{http://www.opengroup.org/susv3xsh/strstr.html}
-Gnulib module: ---
+Gnulib module: strstr
Portability problems fixed by Gnulib:
@itemize
address@hidden
+This function has quadratic instead of linear complexity on some
+platforms:
+glibc <= 2.6.1, cygwin 1.5.x, OpenBSD 4.0, Solaris 9, mingw
@end itemize
Portability problems not fixed by Gnulib:
diff --git a/lib/string.in.h b/lib/string.in.h
index 355479a..4d68cd9 100644
--- a/lib/string.in.h
+++ b/lib/string.in.h
@@ -58,8 +58,9 @@ extern void *memmem (void const *__haystack, size_t
__haystack_len,
#elif defined GNULIB_POSIXCHECK
# undef memmem
# define memmem(a,al,b,bl) \
- (GL_LINK_WARNING ("memmem is unportable - " \
- "use gnulib module memmem for portability"), \
+ (GL_LINK_WARNING ("memmem is unportable and often quadratic - " \
+ "use gnulib module memmem-simple for portability, " \
+ "and module memmem for speed" ), \
memmem (a, al, b, bl))
#endif
@@ -289,16 +290,24 @@ extern char *strsep (char **restrict __stringp, char
const *restrict __delim);
strsep (s, d))
#endif
-#if defined GNULIB_POSIXCHECK
+#if @GNULIB_STRSTR@
+# if @REPLACE_STRSTR@
+# define strstr rpl_strstr
+char *strstr (const char *haystack, const char *needle)
+ __attribute__ ((__pure__));
+# endif
+#elif defined GNULIB_POSIXCHECK
/* strstr() does not work with multibyte strings if the locale encoding is
different from UTF-8:
POSIX says that it operates on "strings", and "string" in POSIX is defined
as a sequence of bytes, not of characters. */
# undef strstr
# define strstr(a,b) \
- (GL_LINK_WARNING ("strstr cannot work correctly on character strings " \
- "in most multibyte locales - " \
- "use mbsstr if you care about internationalization"), \
+ (GL_LINK_WARNING ("strstr is quadratic on many systems, and cannot " \
+ "work correctly on character strings in most " \
+ "multibyte locales - " \
+ "use mbsstr if you care about internationalization, " \
+ "or use strstr if you care about speed"), \
strstr (a, b))
#endif
diff --git a/lib/strstr.c b/lib/strstr.c
new file mode 100644
index 0000000..80b18b0
--- /dev/null
+++ b/lib/strstr.c
@@ -0,0 +1,437 @@
+/* Copyright (C) 1991,92,93,94,96,97,98,2000,2004,2007,2008 Free Software
+ Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2, or (at your option)
+ any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License along
+ with this program; if not, write to the Free Software Foundation,
+ Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
+
+/* This particular implementation was written by Eric Blake, 2008. */
+
+#ifndef _LIBC
+# include <config.h>
+#endif
+
+/* Specification of strstr. */
+#include <string.h>
+
+#include <limits.h>
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+
+#ifndef _LIBC
+# define __builtin_expect(expr, val) (expr)
+#endif
+
+/* We use the Two-Way string matching algorithm, which guarantees
+ linear complexity with constant space. Additionally, for long
+ needles, we also use a bad character shift table similar to the
+ Boyer-Moore algorithm to achieve better performance.
+
+ See http://www-igm.univ-mlv.fr/~lecroq/string/node26.html#SECTION00260
+ and http://en.wikipedia.org/wiki/Boyer-Moore_string_search_algorithm
+*/
+
+/* Point at which computing a bad-byte shift table is likely to be
+ worthwhile. Small needles should not compute a table, since it
+ adds (1 << CHAR_BIT) + NEEDLE_LEN computations of preparation for a
+ speedup no greater than a factor of NEEDLE_LEN. The larger the
+ needle, the better the potential performance gain. On the other
+ hand, on non-POSIX systems with CHAR_BIT larger than eight, the
+ memory required for the table is prohibitive. */
+#if CHAR_BIT < 10
+# define LONG_NEEDLE_THRESHOLD 32U
+#else
+# define LONG_NEEDLE_THRESHOLD SIZE_MAX
+#endif
+
+#define MAX(a, b) ((a < b) ? (b) : (a))
+
+/* Perform a critical factorization of NEEDLE, of length NEEDLE_LEN.
+ Return the index of the first byte in the right half, and set
+ *PERIOD to the global period of the right half.
+
+ The global period of a string is the smallest index (possibly its
+ length) at which all remaining bytes in the string are repetitions
+ of the prefix (the last repetition may be a subset of the prefix).
+
+ When NEEDLE is factored into two halves, a local period is the
+ length of the smallest word that shares a suffix with the left half
+ and shares a prefix with the right half. All factorizations of a
+ non-empty NEEDLE have a local period of at least 1 and no greater
+ than NEEDLE_LEN.
+
+ A critical factorization has the property that the local period
+ equals the global period. All strings have at least one critical
+ factorization with the left half smaller than the global period.
+
+ Given an ordered alphabet, a critical factorization can be computed
+ in linear time, with 2 * NEEDLE_LEN comparisons, by computing the
+ larger of two ordered maximal suffixes. The ordered maximal
+ suffixes are determined by lexicographic comparison of
+ periodicity. */
+static size_t
+critical_factorization (const unsigned char *needle, size_t needle_len,
+ size_t *period)
+{
+ /* Index of last byte of left half, or SIZE_MAX. */
+ size_t max_suffix, max_suffix_rev;
+ size_t j; /* Index into NEEDLE for current candidate suffix. */
+ size_t k; /* Offset into current period. */
+ size_t p; /* Intermediate period. */
+ unsigned char a, b; /* Current comparison bytes. */
+
+ /* Invariants:
+ 0 <= j < NEEDLE_LEN - 1
+ -1 <= max_suffix{,_rev} < j (treating SIZE_MAX as if it were signed)
+ min(max_suffix, max_suffix_rev) < global period of NEEDLE
+ 1 <= p <= global period of NEEDLE
+ p == global period of the substring NEEDLE[max_suffix{,_rev}+1...j]
+ 1 <= k <= p
+ */
+
+ /* Perform lexicographic search. */
+ max_suffix = SIZE_MAX;
+ j = 0;
+ k = p = 1;
+ while (j + k < needle_len)
+ {
+ a = needle[j + k];
+ b = needle[max_suffix + k];
+ if (a < b)
+ {
+ /* Suffix is smaller, period is entire prefix so far. */
+ j += k;
+ k = 1;
+ p = j - max_suffix;
+ }
+ else if (a == b)
+ {
+ /* Advance through repetition of the current period. */
+ if (k != p)
+ ++k;
+ else
+ {
+ j += p;
+ k = 1;
+ }
+ }
+ else /* b < a */
+ {
+ /* Suffix is larger, start over from current location. */
+ max_suffix = j++;
+ k = p = 1;
+ }
+ }
+ *period = p;
+
+ /* Perform reverse lexicographic search. */
+ max_suffix_rev = SIZE_MAX;
+ j = 0;
+ k = p = 1;
+ while (j + k < needle_len)
+ {
+ a = needle[j + k];
+ b = needle[max_suffix_rev + k];
+ if (b < a)
+ {
+ /* Suffix is smaller, period is entire prefix so far. */
+ j += k;
+ k = 1;
+ p = j - max_suffix_rev;
+ }
+ else if (a == b)
+ {
+ /* Advance through repetition of the current period. */
+ if (k != p)
+ ++k;
+ else
+ {
+ j += p;
+ k = 1;
+ }
+ }
+ else /* a < b */
+ {
+ /* Suffix is larger, start over from current location. */
+ max_suffix_rev = j++;
+ k = p = 1;
+ }
+ }
+
+ /* Choose the longer suffix. Return the first byte of the right
+ half, rather than the last byte of the left half. */
+ if (max_suffix_rev + 1 < max_suffix + 1)
+ return max_suffix + 1;
+ *period = p;
+ return max_suffix_rev + 1;
+}
+
+/* Return the first location of NEEDLE within HAYSTACK, or NULL. This
+ method requires 0 < NEEDLE_LEN <= HAYSTACK_LEN, and is optimized
+ for NEEDLE_LEN < LONG_NEEDLE_THRESHOLD. Performance is linear,
+ with 2 * NEEDLE_LEN comparisons in preparation, and at most 3 *
+ HAYSTACK_LEN - NEEDLE_LEN comparisons in searching. */
+static char *
+two_way_short_needle (const unsigned char *haystack, size_t haystack_len,
+ const unsigned char *needle, size_t needle_len)
+{
+ size_t i; /* Index into current byte of NEEDLE. */
+ size_t j; /* Index into current window of HAYSTACK. */
+ size_t period; /* The period of the right half of needle. */
+ size_t suffix; /* The index of the right half of needle. */
+
+ /* Factor the needle into two halves, such that the left half is
+ smaller than the global period, and the right half is
+ periodic (with a period as large as NEEDLE_LEN - suffix). */
+ suffix = critical_factorization (needle, needle_len, &period);
+
+ /* Perform the search. Each iteration compares the right half
+ first. */
+ if (memcmp (needle, needle + period, suffix) == 0)
+ {
+ /* Entire needle is periodic; a mismatch can only advance by the
+ period, so use memory to avoid rescanning known occurrences
+ of the period. */
+ size_t memory = 0;
+ j = 0;
+ while (!memchr (&haystack[haystack_len], '\0',
+ j + needle_len - haystack_len)
+ && (haystack_len = j + needle_len))
+ {
+ /* Scan for matches in right half. */
+ i = MAX (suffix, memory);
+ while (i < needle_len && needle[i] == haystack[i + j])
+ ++i;
+ if (needle_len <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (memory < i + 1 && needle[i] == haystack[i + j])
+ --i;
+ if (i + 1 < memory + 1)
+ return (char *) (haystack + j);
+ /* No match, so remember how many repetitions of period
+ on the right half were scanned. */
+ j += period;
+ memory = needle_len - period;
+ }
+ else
+ {
+ j += i - suffix + 1;
+ memory = 0;
+ }
+ }
+ }
+ else
+ {
+ /* The two halves of needle are distinct; no extra memory is
+ required, and any mismatch results in a maximal shift. */
+ period = MAX (suffix, needle_len - suffix) + 1;
+ j = 0;
+ while (!memchr (&haystack[haystack_len], '\0',
+ j + needle_len - haystack_len)
+ && (haystack_len = j + needle_len))
+ {
+ /* Scan for matches in right half. */
+ i = suffix;
+ while (i < needle_len && needle[i] == haystack[i + j])
+ ++i;
+ if (needle_len <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (i != SIZE_MAX && needle[i] == haystack[i + j])
+ --i;
+ if (i == SIZE_MAX)
+ return (char *) (haystack + j);
+ j += period;
+ }
+ else
+ j += i - suffix + 1;
+ }
+ }
+ return NULL;
+}
+
+/* Return the first location of NEEDLE within HAYSTACK, or NULL. This
+ method requires 0 < NEEDLE_LEN <= HAYSTACK_LEN, and is optimized
+ for LONG_NEEDLE_THRESHOLD <= NEEDLE_LEN. Performance is linear,
+ with 3 * NEEDLE_LEN + (1U << CHAR_BIT) operations in preparation,
+ and at most 3 * HAYSTACK_LEN - NEEDLE_LEN comparisons in searching.
+ The extra initialization cost allows for as few as HAYSTACK_LEN +
+ HAYSTACK_LEN / NEEDLE_LEN. */
+static char *
+two_way_long_needle (const unsigned char *haystack, size_t haystack_len,
+ const unsigned char *needle, size_t needle_len)
+{
+ size_t i; /* Index into current byte of NEEDLE. */
+ size_t j; /* Index into current window of HAYSTACK. */
+ size_t period; /* The period of the right half of needle. */
+ size_t suffix; /* The index of the right half of needle. */
+ size_t shift_table[1U << CHAR_BIT]; /* See below. */
+
+ /* Factor the needle into two halves, such that the left half is
+ smaller than the global period, and the right half is
+ periodic (with a period as large as NEEDLE_LEN - suffix). */
+ suffix = critical_factorization (needle, needle_len, &period);
+
+ /* Populate shift_table. For each possible byte value c,
+ shift_table[c] is the distance from the last occurrence of c to
+ the end of NEEDLE, or NEEDLE_LEN if c is absent from the NEEDLE.
+ shift_table[NEEDLE[NEEDLE_LEN - 1]] contains the only 0. */
+ for (i = 0; i < 1U << CHAR_BIT; i++)
+ shift_table[i] = needle_len;
+ for (i = 0; i < needle_len; i++)
+ shift_table[needle[i]] = needle_len - i - 1;
+
+ /* Perform the search. Each iteration compares the right half
+ first. */
+ if (memcmp (needle, needle + period, suffix) == 0)
+ {
+ /* Entire needle is periodic; a mismatch can only advance by the
+ period, so use memory to avoid rescanning known occurrences
+ of the period. */
+ size_t memory = 0;
+ j = 0;
+ while (!memchr (&haystack[haystack_len], '\0',
+ j + needle_len - haystack_len)
+ && (haystack_len = j + needle_len))
+ {
+ /* Check the last byte first; if it does not match, then
+ shift to the next possible match location. */
+ size_t shift = shift_table[haystack[j + needle_len - 1]];
+ if (0 < shift)
+ {
+ if (memory && shift < period)
+ {
+ /* Since needle is periodic, but the last period has
+ a byte out of place, there can be no match until
+ after the mismatch. */
+ shift = needle_len - period;
+ memory = 0;
+ }
+ j += shift;
+ continue;
+ }
+ /* Scan for matches in right half. The last byte has
+ already been matched, by virtue of the shift table. */
+ i = MAX (suffix, memory);
+ while (i < needle_len - 1 && needle[i] == haystack[i + j])
+ ++i;
+ if (needle_len - 1 <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (memory < i + 1 && needle[i] == haystack[i + j])
+ --i;
+ if (i + 1 < memory + 1)
+ return (char *) (haystack + j);
+ /* No match, so remember how many repetitions of period
+ on the right half were scanned. */
+ j += period;
+ memory = needle_len - period;
+ }
+ else
+ {
+ j += i - suffix + 1;
+ memory = 0;
+ }
+ }
+ }
+ else
+ {
+ /* The two halves of needle are distinct; no extra memory is
+ required, and any mismatch results in a maximal shift. */
+ period = MAX (suffix, needle_len - suffix) + 1;
+ j = 0;
+ while (!memchr (&haystack[haystack_len], '\0',
+ j + needle_len - haystack_len)
+ && (haystack_len = j + needle_len))
+ {
+ /* Check the last byte first; if it does not match, then
+ shift to the next possible match location. */
+ size_t shift = shift_table[haystack[j + needle_len - 1]];
+ if (0 < shift)
+ {
+ j += shift;
+ continue;
+ }
+ /* Scan for matches in right half. The last byte has
+ already been matched, by virtue of the shift table. */
+ i = suffix;
+ while (i < needle_len - 1 && needle[i] == haystack[i + j])
+ ++i;
+ if (needle_len - 1 <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (i != SIZE_MAX && needle[i] == haystack[i + j])
+ --i;
+ if (i == SIZE_MAX)
+ return (char *) (haystack + j);
+ j += period;
+ }
+ else
+ j += i - suffix + 1;
+ }
+ }
+ return NULL;
+}
+
+/* Return the first occurrence of NEEDLE in HAYSTACK. Return HAYSTACK
+ if NEEDLE is empty, otherwise NULL if NEEDLE is not found in
+ HAYSTACK. */
+char *
+strstr (const char *haystack_start, const char *needle_start)
+{
+ const char *haystack = haystack_start;
+ const char *needle = needle_start;
+ size_t needle_len; /* Length of NEEDLE. */
+ size_t haystack_len; /* Known minimum length of HAYSTACK. */
+ bool ok = true; /* True if NEEDLE is prefix of HAYSTACK. */
+
+ /* Determine length of NEEDLE, and in the process, make sure
+ HAYSTACK is at least as long (no point processing all of a long
+ NEEDLE if HAYSTACK is too short). */
+ while (*haystack && *needle)
+ ok &= *haystack++ == *needle++;
+ if (*needle)
+ return NULL;
+ if (ok)
+ return (char *) haystack_start;
+
+ /* Reduce the size of haystack using strchr, since it has a smaller
+ linear coefficient than the Two-Way algorithm. */
+ needle_len = needle - needle_start;
+ haystack = strchr (haystack_start + 1, *needle_start);
+ if (!haystack || __builtin_expect (needle_len == 1, 0))
+ return (char *) haystack;
+ needle -= needle_len;
+ haystack_len = (haystack > haystack_start + needle_len ? 1
+ : needle_len + haystack_start - haystack);
+
+ /* Perform the search. Abstract memory is considered to be an array
+ of 'unsigned char' values, not an array of 'char' values. See
+ ISO C 99 section 6.2.6.1. */
+ if (needle_len < LONG_NEEDLE_THRESHOLD)
+ return two_way_short_needle ((const unsigned char *) haystack,
+ haystack_len,
+ (const unsigned char *) needle, needle_len);
+ return two_way_long_needle ((const unsigned char *) haystack, haystack_len,
+ (const unsigned char *) needle, needle_len);
+}
+
+#undef LONG_NEEDLE_THRESHOLD
+#undef MAX
diff --git a/m4/string_h.m4 b/m4/string_h.m4
index 99a0dab..1598c39 100644
--- a/m4/string_h.m4
+++ b/m4/string_h.m4
@@ -1,11 +1,11 @@
# Configure a GNU-like replacement for <string.h>.
-# Copyright (C) 2007 Free Software Foundation, Inc.
+# Copyright (C) 2007, 2008 Free Software Foundation, Inc.
# This file is free software; the Free Software Foundation
# gives unlimited permission to copy and/or distribute it,
# with or without modifications, as long as this notice is preserved.
-# serial 2
+# serial 3
# Written by Paul Eggert.
@@ -43,6 +43,7 @@ AC_DEFUN([gl_HEADER_STRING_H_DEFAULTS],
GNULIB_STRNLEN=0; AC_SUBST([GNULIB_STRNLEN])
GNULIB_STRPBRK=0; AC_SUBST([GNULIB_STRPBRK])
GNULIB_STRSEP=0; AC_SUBST([GNULIB_STRSEP])
+ GNULIB_STRSTR=0; AC_SUBST([GNULIB_STRSTR])
GNULIB_STRCASESTR=0; AC_SUBST([GNULIB_STRCASESTR])
GNULIB_STRTOK_R=0; AC_SUBST([GNULIB_STRTOK_R])
GNULIB_MBSLEN=0; AC_SUBST([GNULIB_MBSLEN])
@@ -78,4 +79,5 @@ AC_DEFUN([gl_HEADER_STRING_H_DEFAULTS],
HAVE_DECL_STRERROR=1; AC_SUBST([HAVE_DECL_STRERROR])
REPLACE_STRERROR=0; AC_SUBST([REPLACE_STRERROR])
REPLACE_MEMMEM=0; AC_SUBST([REPLACE_MEMMEM])
+ REPLACE_STRSTR=0; AC_SUBST([REPLACE_STRSTR])
])
diff --git a/m4/strstr.m4 b/m4/strstr.m4
new file mode 100644
index 0000000..4fe15bd
--- /dev/null
+++ b/m4/strstr.m4
@@ -0,0 +1,48 @@
+# strstr.m4 serial 1
+dnl Copyright (C) 2008 Free Software Foundation, Inc.
+dnl This file is free software; the Free Software Foundation
+dnl gives unlimited permission to copy and/or distribute it,
+dnl with or without modifications, as long as this notice is preserved.
+
+dnl Check that strstr is efficient.
+AC_DEFUN([gl_FUNC_STRSTR],
+[
+ AC_REQUIRE([gl_HEADER_STRING_H_DEFAULTS])
+ AC_CACHE_CHECK([whether strstr works in linear time],
+ [gl_cv_func_strstr_linear],
+ [AC_RUN_IFELSE([AC_LANG_PROGRAM([
+#include <string.h> /* for memmem */
+#include <stdlib.h> /* for malloc */
+#include <unistd.h> /* for alarm */
+], [[size_t m = 1000000;
+ char *haystack = (char *) malloc (2 * m + 2);
+ char *needle = (char *) malloc (m + 2);
+ void *result = 0;
+ /* Failure to compile this test due to missing alarm is okay,
+ since all such platforms (mingw) also have quadratic strstr. */
+ alarm (5);
+ /* Check for quadratic performance. */
+ if (haystack && needle)
+ {
+ memset (haystack, 'A', 2 * m);
+ haystack[2 * m] = 'B';
+ haystack[2 * m + 1] = 0;
+ memset (needle, 'A', m);
+ needle[m] = 'B';
+ needle[m + 1] = 0;
+ result = strstr (haystack, needle);
+ }
+ return !result;]])],
+ [gl_cv_func_strstr_linear=yes], [gl_cv_func_strstr_linear=no],
+ [dnl pessimistically assume the worst, since even glibc 2.6.1
+ dnl has quadratic complexity in its strstr
+ gl_cv_func_strstr_linear="guessing no"])])
+ if test "$gl_cv_func_strstr_linear" != yes; then
+ REPLACE_STRSTR=1
+ AC_LIBOBJ([strstr])
+ fi
+]) # gl_FUNC_MEMMEM
+
+# Prerequisites of lib/memmem.c.
+AC_DEFUN([gl_PREREQ_MEMMEM], [:])
+])
diff --git a/modules/string b/modules/string
index 6f3226d..431a322 100644
--- a/modules/string
+++ b/modules/string
@@ -48,6 +48,7 @@ string.h: string.in.h
-e 's|@''GNULIB_STRNLEN''@|$(GNULIB_STRNLEN)|g' \
-e 's|@''GNULIB_STRPBRK''@|$(GNULIB_STRPBRK)|g' \
-e 's|@''GNULIB_STRSEP''@|$(GNULIB_STRSEP)|g' \
+ -e 's|@''GNULIB_STRSTR''@|$(GNULIB_STRSTR)|g' \
-e 's|@''GNULIB_STRCASESTR''@|$(GNULIB_STRCASESTR)|g' \
-e 's|@''GNULIB_STRTOK_R''@|$(GNULIB_STRTOK_R)|g' \
-e 's|@''GNULIB_STRERROR''@|$(GNULIB_STRERROR)|g' \
@@ -67,6 +68,7 @@ string.h: string.in.h
-e 's|@''HAVE_DECL_STRTOK_R''@|$(HAVE_DECL_STRTOK_R)|g' \
-e 's|@''HAVE_DECL_STRERROR''@|$(HAVE_DECL_STRERROR)|g' \
-e 's|@''REPLACE_MEMMEM''@|$(REPLACE_MEMMEM)|g' \
+ -e 's|@''REPLACE_STRSTR''@|$(REPLACE_STRSTR)|g' \
-e 's|@''REPLACE_STRERROR''@|$(REPLACE_STRERROR)|g' \
-e '/definition of GL_LINK_WARNING/r $(LINK_WARNING_H)' \
< $(srcdir)/string.in.h; \
diff --git a/modules/strstr b/modules/strstr
new file mode 100644
index 0000000..4f93df9
--- /dev/null
+++ b/modules/strstr
@@ -0,0 +1,27 @@
+Description:
+strstr() function: efficiently locate first substring in a buffer.
+
+Files:
+lib/strstr.c
+m4/strstr.m4
+
+Depends-on:
+string
+stdbool
+memchr
+memcmp
+
+configure.ac:
+gl_FUNC_STRSTR
+gl_STRING_MODULE_INDICATOR([strstr])
+
+Makefile.am:
+
+Include:
+<string.h>
+
+License:
+LGPLv2+
+
+Maintainer:
+all, Eric Blake
diff --git a/modules/strstr-tests b/modules/strstr-tests
new file mode 100644
index 0000000..23e67b0
--- /dev/null
+++ b/modules/strstr-tests
@@ -0,0 +1,12 @@
+Files:
+tests/test-strstr.c
+
+Depends-on:
+
+configure.ac:
+AC_CHECK_DECLS_ONCE([alarm])
+
+Makefile.am:
+TESTS += test-strstr
+check_PROGRAMS += test-strstr
+
diff --git a/tests/test-strstr.c b/tests/test-strstr.c
new file mode 100644
index 0000000..a4cc0fc
--- /dev/null
+++ b/tests/test-strstr.c
@@ -0,0 +1,155 @@
+/*
+ * Copyright (C) 2004, 2007, 2008 Free Software Foundation
+ * Written by Bruno Haible and Eric Blake
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <string.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#define ASSERT(expr) \
+ do \
+ { \
+ if (!(expr)) \
+ { \
+ fprintf (stderr, "%s:%d: assertion failed\n", __FILE__, __LINE__); \
+ abort (); \
+ } \
+ } \
+ while (0)
+
+int
+main (int argc, char *argv[])
+{
+#if HAVE_DECL_ALARM
+ /* Declare failure if test takes too long, by using default abort
+ caused by SIGALRM. All known platforms that lack alarm also have
+ a quadratic strstr, and the replacement strstr is known to not
+ take too long. */
+ alarm (10);
+#endif
+
+ {
+ const char input[] = "foo";
+ const char *result = strstr (input, "");
+ ASSERT (result == input);
+ }
+
+ {
+ const char input[] = "foo";
+ const char *result = strstr (input, "o");
+ ASSERT (result == input + 1);
+ }
+
+ {
+ const char input[] = "ABC ABCDAB ABCDABCDABDE";
+ const char *result = strstr (input, "ABCDABD");
+ ASSERT (result == input + 15);
+ }
+
+ {
+ const char input[] = "ABC ABCDAB ABCDABCDABDE";
+ const char *result = strstr (input, "ABCDABE");
+ ASSERT (result == NULL);
+ }
+
+ {
+ const char input[] = "ABC ABCDAB ABCDABCDABDE";
+ const char *result = strstr (input, "ABCDABCD");
+ ASSERT (result == input + 11);
+ }
+
+ /* Check that a very long haystack is handled quickly if the needle is
+ short and occurs near the beginning. */
+ {
+ size_t repeat = 10000;
+ size_t m = 1000000;
+ char *needle =
+ "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
+ "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
+ char *haystack = (char *) malloc (m + 1);
+ if (haystack != NULL)
+ {
+ memset (haystack, 'A', m);
+ haystack[0] = 'B';
+ haystack[m] = '\0';
+
+ for (; repeat > 0; repeat--)
+ {
+ ASSERT (strstr (haystack, needle) == haystack + 1);
+ }
+
+ free (haystack);
+ }
+ }
+
+ /* Check that a very long needle is discarded quickly if the haystack is
+ short. */
+ {
+ size_t repeat = 10000;
+ size_t m = 1000000;
+ char *haystack =
+ "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
+ "ABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABAB";
+ char *needle = (char *) malloc (m + 1);
+ if (needle != NULL)
+ {
+ memset (needle, 'A', m);
+ needle[m] = '\0';
+
+ for (; repeat > 0; repeat--)
+ {
+ ASSERT (strstr (haystack, needle) == NULL);
+ }
+
+ free (needle);
+ }
+ }
+
+ /* Check that the asymptotic worst-case complexity is not quadratic. */
+ {
+ size_t m = 1000000;
+ char *haystack = (char *) malloc (2 * m + 2);
+ char *needle = (char *) malloc (m + 2);
+ if (haystack != NULL && needle != NULL)
+ {
+ const char *result;
+
+ memset (haystack, 'A', 2 * m);
+ haystack[2 * m] = 'B';
+ haystack[2 * m + 1] = '\0';
+
+ memset (needle, 'A', m);
+ needle[m] = 'B';
+ needle[m + 1] = '\0';
+
+ result = strstr (haystack, needle);
+ ASSERT (result == haystack + m);
+ }
+ if (needle != NULL)
+ free (needle);
+ if (haystack != NULL)
+ free (haystack);
+ }
+
+ /* Sublinear speed is only possible in memmem; strstr must examine
+ every character of haystack to find its length. */
+
+ return 0;
+}
--
1.5.3.5
>From c358da1eb79a1af4d7e4b887ca3896fb7f3fdcdb Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Thu, 10 Jan 2008 12:06:35 -0700
Subject: [PATCH] Share two-way algorithm.
* lib/str-two-way.h: New file, merged from...
* lib/memmem.c: ...here...
* lib/strstr.c: ...and here.
* modules/memmem (Files): Use it.
* modules/strstr (Files): Likewise.
Signed-off-by: Eric Blake <address@hidden>
---
ChangeLog | 7 +
lib/memmem.c | 355 +----------------------------------------
lib/str-two-way.h | 426 +++++++++++++++++++++++++++++++++++++++++++++++++
lib/strstr.c | 364 +-----------------------------------------
modules/memmem-simple | 1 +
modules/strstr | 1 +
6 files changed, 443 insertions(+), 711 deletions(-)
create mode 100644 lib/str-two-way.h
diff --git a/ChangeLog b/ChangeLog
index 44df277..7c7c885 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
2008-01-10 Eric Blake <address@hidden>
+ Share two-way algorithm.
+ * lib/str-two-way.h: New file, merged from...
+ * lib/memmem.c: ...here...
+ * lib/strstr.c: ...and here.
+ * modules/memmem (Files): Use it.
+ * modules/strstr (Files): Likewise.
+
Avoid quadratic strstr implementations.
* lib/strstr.c: New file.
* m4/strstr.m4: Likewise.
diff --git a/lib/memmem.c b/lib/memmem.c
index 622a034..186d840 100644
--- a/lib/memmem.c
+++ b/lib/memmem.c
@@ -25,361 +25,13 @@
/* Specification of memmem. */
#include <string.h>
-#include <limits.h>
-#include <stddef.h>
-#include <stdint.h>
-
#ifndef _LIBC
# define __builtin_expect(expr, val) (expr)
#endif
-/* We use the Two-Way string matching algorithm, which guarantees
- linear complexity with constant space. Additionally, for long
- needles, we also use a bad character shift table similar to the
- Boyer-Moore algorithm to achieve sub-linear performance.
-
- See http://www-igm.univ-mlv.fr/~lecroq/string/node26.html#SECTION00260
- and http://en.wikipedia.org/wiki/Boyer-Moore_string_search_algorithm
-*/
-
-/* Point at which computing a bad-byte shift table is likely to be
- worthwhile. Small needles should not compute a table, since it
- adds (1 << CHAR_BIT) + NEEDLE_LEN computations of preparation for a
- speedup no greater than a factor of NEEDLE_LEN. The larger the
- needle, the better the potential performance gain. On the other
- hand, on non-POSIX systems with CHAR_BIT larger than eight, the
- memory required for the table is prohibitive. */
-#if CHAR_BIT < 10
-# define LONG_NEEDLE_THRESHOLD 32U
-#else
-# define LONG_NEEDLE_THRESHOLD SIZE_MAX
-#endif
-
-#define MAX(a, b) ((a < b) ? (b) : (a))
-
-/* Perform a critical factorization of NEEDLE, of length NEEDLE_LEN.
- Return the index of the first byte in the right half, and set
- *PERIOD to the global period of the right half.
-
- The global period of a string is the smallest index (possibly its
- length) at which all remaining bytes in the string are repetitions
- of the prefix (the last repetition may be a subset of the prefix).
-
- When NEEDLE is factored into two halves, a local period is the
- length of the smallest word that shares a suffix with the left half
- and shares a prefix with the right half. All factorizations of a
- non-empty NEEDLE have a local period of at least 1 and no greater
- than NEEDLE_LEN.
-
- A critical factorization has the property that the local period
- equals the global period. All strings have at least one critical
- factorization with the left half smaller than the global period.
-
- Given an ordered alphabet, a critical factorization can be computed
- in linear time, with 2 * NEEDLE_LEN comparisons, by computing the
- larger of two ordered maximal suffixes. The ordered maximal
- suffixes are determined by lexicographic comparison of
- periodicity. */
-static size_t
-critical_factorization (const unsigned char *needle, size_t needle_len,
- size_t *period)
-{
- /* Index of last byte of left half, or SIZE_MAX. */
- size_t max_suffix, max_suffix_rev;
- size_t j; /* Index into NEEDLE for current candidate suffix. */
- size_t k; /* Offset into current period. */
- size_t p; /* Intermediate period. */
- unsigned char a, b; /* Current comparison bytes. */
-
- /* Invariants:
- 0 <= j < NEEDLE_LEN - 1
- -1 <= max_suffix{,_rev} < j (treating SIZE_MAX as if it were signed)
- min(max_suffix, max_suffix_rev) < global period of NEEDLE
- 1 <= p <= global period of NEEDLE
- p == global period of the substring NEEDLE[max_suffix{,_rev}+1...j]
- 1 <= k <= p
- */
-
- /* Perform lexicographic search. */
- max_suffix = SIZE_MAX;
- j = 0;
- k = p = 1;
- while (j + k < needle_len)
- {
- a = needle[j + k];
- b = needle[max_suffix + k];
- if (a < b)
- {
- /* Suffix is smaller, period is entire prefix so far. */
- j += k;
- k = 1;
- p = j - max_suffix;
- }
- else if (a == b)
- {
- /* Advance through repetition of the current period. */
- if (k != p)
- ++k;
- else
- {
- j += p;
- k = 1;
- }
- }
- else /* b < a */
- {
- /* Suffix is larger, start over from current location. */
- max_suffix = j++;
- k = p = 1;
- }
- }
- *period = p;
-
- /* Perform reverse lexicographic search. */
- max_suffix_rev = SIZE_MAX;
- j = 0;
- k = p = 1;
- while (j + k < needle_len)
- {
- a = needle[j + k];
- b = needle[max_suffix_rev + k];
- if (b < a)
- {
- /* Suffix is smaller, period is entire prefix so far. */
- j += k;
- k = 1;
- p = j - max_suffix_rev;
- }
- else if (a == b)
- {
- /* Advance through repetition of the current period. */
- if (k != p)
- ++k;
- else
- {
- j += p;
- k = 1;
- }
- }
- else /* a < b */
- {
- /* Suffix is larger, start over from current location. */
- max_suffix_rev = j++;
- k = p = 1;
- }
- }
-
- /* Choose the longer suffix. Return the first byte of the right
- half, rather than the last byte of the left half. */
- if (max_suffix_rev + 1 < max_suffix + 1)
- return max_suffix + 1;
- *period = p;
- return max_suffix_rev + 1;
-}
-
-/* Return the first location of NEEDLE within HAYSTACK, or NULL. This
- method requires 0 < NEEDLE_LEN <= HAYSTACK_LEN, and is optimized
- for NEEDLE_LEN < LONG_NEEDLE_THRESHOLD. Performance is linear,
- with 2 * NEEDLE_LEN comparisons in preparation, and at most 2 *
- HAYSTACK_LEN - NEEDLE_LEN comparisons in searching. */
-static void *
-two_way_short_needle (const unsigned char *haystack, size_t haystack_len,
- const unsigned char *needle, size_t needle_len)
-{
- size_t i; /* Index into current byte of NEEDLE. */
- size_t j; /* Index into current window of HAYSTACK. */
- size_t period; /* The period of the right half of needle. */
- size_t suffix; /* The index of the right half of needle. */
-
- /* Factor the needle into two halves, such that the left half is
- smaller than the global period, and the right half is
- periodic (with a period as large as NEEDLE_LEN - suffix). */
- suffix = critical_factorization (needle, needle_len, &period);
-
- /* Perform the search. Each iteration compares the right half
- first. */
- if (memcmp (needle, needle + period, suffix) == 0)
- {
- /* Entire needle is periodic; a mismatch can only advance by the
- period, so use memory to avoid rescanning known occurrences
- of the period. */
- size_t memory = 0;
- j = 0;
- while (j <= haystack_len - needle_len)
- {
- /* Scan for matches in right half. */
- i = MAX (suffix, memory);
- while (i < needle_len && needle[i] == haystack[i + j])
- ++i;
- if (needle_len <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (memory < i + 1 && needle[i] == haystack[i + j])
- --i;
- if (i + 1 < memory + 1)
- return (void *) (haystack + j);
- /* No match, so remember how many repetitions of period
- on the right half were scanned. */
- j += period;
- memory = needle_len - period;
- }
- else
- {
- j += i - suffix + 1;
- memory = 0;
- }
- }
- }
- else
- {
- /* The two halves of needle are distinct; no extra memory is
- required, and any mismatch results in a maximal shift. */
- period = MAX (suffix, needle_len - suffix) + 1;
- j = 0;
- while (j <= haystack_len - needle_len)
- {
- /* Scan for matches in right half. */
- i = suffix;
- while (i < needle_len && needle[i] == haystack[i + j])
- ++i;
- if (needle_len <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (i != SIZE_MAX && needle[i] == haystack[i + j])
- --i;
- if (i == SIZE_MAX)
- return (void *) (haystack + j);
- j += period;
- }
- else
- j += i - suffix + 1;
- }
- }
- return NULL;
-}
-
-/* Return the first location of NEEDLE within HAYSTACK, or NULL. This
- method requires 0 < NEEDLE_LEN <= HAYSTACK_LEN, and is optimized
- for LONG_NEEDLE_THRESHOLD <= NEEDLE_LEN. Performance is linear,
- with 3 * NEEDLE_LEN + (1U << CHAR_BIT) operations in preparation,
- and at most 2 * HAYSTACK_LEN - NEEDLE_LEN comparisons in searching.
- The extra initialization cost allows for potential sublinear
- performance O(HAYSTACK_LEN / NEEDLE_LEN). */
-static void *
-two_way_long_needle (const unsigned char *haystack, size_t haystack_len,
- const unsigned char *needle, size_t needle_len)
-{
- size_t i; /* Index into current byte of NEEDLE. */
- size_t j; /* Index into current window of HAYSTACK. */
- size_t period; /* The period of the right half of needle. */
- size_t suffix; /* The index of the right half of needle. */
- size_t shift_table[1U << CHAR_BIT]; /* See below. */
-
- /* Factor the needle into two halves, such that the left half is
- smaller than the global period, and the right half is
- periodic (with a period as large as NEEDLE_LEN - suffix). */
- suffix = critical_factorization (needle, needle_len, &period);
-
- /* Populate shift_table. For each possible byte value c,
- shift_table[c] is the distance from the last occurrence of c to
- the end of NEEDLE, or NEEDLE_LEN if c is absent from the NEEDLE.
- shift_table[NEEDLE[NEEDLE_LEN - 1]] contains the only 0. */
- for (i = 0; i < 1U << CHAR_BIT; i++)
- shift_table[i] = needle_len;
- for (i = 0; i < needle_len; i++)
- shift_table[needle[i]] = needle_len - i - 1;
-
- /* Perform the search. Each iteration compares the right half
- first. */
- if (memcmp (needle, needle + period, suffix) == 0)
- {
- /* Entire needle is periodic; a mismatch can only advance by the
- period, so use memory to avoid rescanning known occurrences
- of the period. */
- size_t memory = 0;
- j = 0;
- while (j <= haystack_len - needle_len)
- {
- /* Check the last byte first; if it does not match, then
- shift to the next possible match location. */
- size_t shift = shift_table[haystack[j + needle_len - 1]];
- if (0 < shift)
- {
- if (memory && shift < period)
- {
- /* Since needle is periodic, but the last period has
- a byte out of place, there can be no match until
- after the mismatch. */
- shift = needle_len - period;
- memory = 0;
- }
- j += shift;
- continue;
- }
- /* Scan for matches in right half. The last byte has
- already been matched, by virtue of the shift table. */
- i = MAX (suffix, memory);
- while (i < needle_len - 1 && needle[i] == haystack[i + j])
- ++i;
- if (needle_len - 1 <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (memory < i + 1 && needle[i] == haystack[i + j])
- --i;
- if (i + 1 < memory + 1)
- return (void *) (haystack + j);
- /* No match, so remember how many repetitions of period
- on the right half were scanned. */
- j += period;
- memory = needle_len - period;
- }
- else
- {
- j += i - suffix + 1;
- memory = 0;
- }
- }
- }
- else
- {
- /* The two halves of needle are distinct; no extra memory is
- required, and any mismatch results in a maximal shift. */
- period = MAX (suffix, needle_len - suffix) + 1;
- j = 0;
- while (j <= haystack_len - needle_len)
- {
- /* Check the last byte first; if it does not match, then
- shift to the next possible match location. */
- size_t shift = shift_table[haystack[j + needle_len - 1]];
- if (0 < shift)
- {
- j += shift;
- continue;
- }
- /* Scan for matches in right half. The last byte has
- already been matched, by virtue of the shift table. */
- i = suffix;
- while (i < needle_len - 1 && needle[i] == haystack[i + j])
- ++i;
- if (needle_len - 1 <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (i != SIZE_MAX && needle[i] == haystack[i + j])
- --i;
- if (i == SIZE_MAX)
- return (void *) (haystack + j);
- j += period;
- }
- else
- j += i - suffix + 1;
- }
- }
- return NULL;
-}
+#define RETURN_TYPE void *
+#define AVAILABLE(h, h_l, j, n_l) ((j) <= (h_l) - (n_l))
+#include "str-two-way.h"
/* Return the first occurrence of NEEDLE in HAYSTACK. Return HAYSTACK
if NEEDLE_LEN is 0, otherwise NULL if NEEDLE is not found in
@@ -422,4 +74,3 @@ memmem (const void *haystack_start, size_t haystack_len,
}
#undef LONG_NEEDLE_THRESHOLD
-#undef MAX
diff --git a/lib/str-two-way.h b/lib/str-two-way.h
new file mode 100644
index 0000000..3aa3a1b
--- /dev/null
+++ b/lib/str-two-way.h
@@ -0,0 +1,426 @@
+/* Byte-wise substring search, using the Two-Way algorithm.
+ Copyright (C) 2008 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+ Written by Eric Blake <address@hidden>, 2008.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2, or (at your option)
+ any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License along
+ with this program; if not, write to the Free Software Foundation,
+ Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
+
+/* Before including this file, you need to include <config.h> and
+ <string.h>, and define:
+ RESULT_TYPE A macro that expands to the return type.
+ AVAILABLE(h, h_l, j, n_l)
+ A macro that returns nonzero if there are
+ at least N_L bytes left starting at H[J].
+ H is 'unsigned char *', H_L, J, and N_L
+ are 'size_t'; H_L is an lvalue. For
+ NUL-terminated searches, H_L can be
+ modified each iteration to avoid having
+ to compute the end of H up front.
+
+ For case-insensitivity, you may optionally define:
+ CMP_FUNC(p1, p2, l) A macro that returns 0 iff the first L
+ characters of P1 and P2 are equal.
+ CANON_ELEMENT(c) A macro that canonicalizes an element right after
+ it has been fetched from one of the two strings.
+ The argument is an 'unsigned char'; the result
+ must be an 'unsigned char' as well.
+
+ This file undefines the macros documented above, and defines
+ LONG_NEEDLE_THRESHOLD.
+*/
+
+#include <limits.h>
+#include <stdint.h>
+
+/* We use the Two-Way string matching algorithm, which guarantees
+ linear complexity with constant space. Additionally, for long
+ needles, we also use a bad character shift table similar to the
+ Boyer-Moore algorithm to achieve improved (potentially sub-linear)
+ performance.
+
+ See http://www-igm.univ-mlv.fr/~lecroq/string/node26.html#SECTION00260
+ and http://en.wikipedia.org/wiki/Boyer-Moore_string_search_algorithm
+*/
+
+/* Point at which computing a bad-byte shift table is likely to be
+ worthwhile. Small needles should not compute a table, since it
+ adds (1 << CHAR_BIT) + NEEDLE_LEN computations of preparation for a
+ speedup no greater than a factor of NEEDLE_LEN. The larger the
+ needle, the better the potential performance gain. On the other
+ hand, on non-POSIX systems with CHAR_BIT larger than eight, the
+ memory required for the table is prohibitive. */
+#if CHAR_BIT < 10
+# define LONG_NEEDLE_THRESHOLD 32U
+#else
+# define LONG_NEEDLE_THRESHOLD SIZE_MAX
+#endif
+
+#define MAX(a, b) ((a < b) ? (b) : (a))
+
+#ifndef CANON_ELEMENT
+# define CANON_ELEMENT(c) c
+#endif
+#ifndef CMP_FUNC
+# define CMP_FUNC memcmp
+#endif
+
+/* Perform a critical factorization of NEEDLE, of length NEEDLE_LEN.
+ Return the index of the first byte in the right half, and set
+ *PERIOD to the global period of the right half.
+
+ The global period of a string is the smallest index (possibly its
+ length) at which all remaining bytes in the string are repetitions
+ of the prefix (the last repetition may be a subset of the prefix).
+
+ When NEEDLE is factored into two halves, a local period is the
+ length of the smallest word that shares a suffix with the left half
+ and shares a prefix with the right half. All factorizations of a
+ non-empty NEEDLE have a local period of at least 1 and no greater
+ than NEEDLE_LEN.
+
+ A critical factorization has the property that the local period
+ equals the global period. All strings have at least one critical
+ factorization with the left half smaller than the global period.
+
+ Given an ordered alphabet, a critical factorization can be computed
+ in linear time, with 2 * NEEDLE_LEN comparisons, by computing the
+ larger of two ordered maximal suffixes. The ordered maximal
+ suffixes are determined by lexicographic comparison of
+ periodicity. */
+static size_t
+critical_factorization (const unsigned char *needle, size_t needle_len,
+ size_t *period)
+{
+ /* Index of last byte of left half, or SIZE_MAX. */
+ size_t max_suffix, max_suffix_rev;
+ size_t j; /* Index into NEEDLE for current candidate suffix. */
+ size_t k; /* Offset into current period. */
+ size_t p; /* Intermediate period. */
+ unsigned char a, b; /* Current comparison bytes. */
+
+ /* Invariants:
+ 0 <= j < NEEDLE_LEN - 1
+ -1 <= max_suffix{,_rev} < j (treating SIZE_MAX as if it were signed)
+ min(max_suffix, max_suffix_rev) < global period of NEEDLE
+ 1 <= p <= global period of NEEDLE
+ p == global period of the substring NEEDLE[max_suffix{,_rev}+1...j]
+ 1 <= k <= p
+ */
+
+ /* Perform lexicographic search. */
+ max_suffix = SIZE_MAX;
+ j = 0;
+ k = p = 1;
+ while (j + k < needle_len)
+ {
+ a = CANON_ELEMENT (needle[j + k]);
+ b = CANON_ELEMENT (needle[max_suffix + k]);
+ if (a < b)
+ {
+ /* Suffix is smaller, period is entire prefix so far. */
+ j += k;
+ k = 1;
+ p = j - max_suffix;
+ }
+ else if (a == b)
+ {
+ /* Advance through repetition of the current period. */
+ if (k != p)
+ ++k;
+ else
+ {
+ j += p;
+ k = 1;
+ }
+ }
+ else /* b < a */
+ {
+ /* Suffix is larger, start over from current location. */
+ max_suffix = j++;
+ k = p = 1;
+ }
+ }
+ *period = p;
+
+ /* Perform reverse lexicographic search. */
+ max_suffix_rev = SIZE_MAX;
+ j = 0;
+ k = p = 1;
+ while (j + k < needle_len)
+ {
+ a = CANON_ELEMENT (needle[j + k]);
+ b = CANON_ELEMENT (needle[max_suffix_rev + k]);
+ if (b < a)
+ {
+ /* Suffix is smaller, period is entire prefix so far. */
+ j += k;
+ k = 1;
+ p = j - max_suffix_rev;
+ }
+ else if (a == b)
+ {
+ /* Advance through repetition of the current period. */
+ if (k != p)
+ ++k;
+ else
+ {
+ j += p;
+ k = 1;
+ }
+ }
+ else /* a < b */
+ {
+ /* Suffix is larger, start over from current location. */
+ max_suffix_rev = j++;
+ k = p = 1;
+ }
+ }
+
+ /* Choose the longer suffix. Return the first byte of the right
+ half, rather than the last byte of the left half. */
+ if (max_suffix_rev + 1 < max_suffix + 1)
+ return max_suffix + 1;
+ *period = p;
+ return max_suffix_rev + 1;
+}
+
+/* Return the first location of non-empty NEEDLE within HAYSTACK, or
+ NULL. HAYSTACK_LEN is the minimum known length of HAYSTACK. This
+ method is optimized for NEEDLE_LEN < LONG_NEEDLE_THRESHOLD.
+ Performance is guaranteed to be linear, with an initialization cost
+ of 2 * NEEDLE_LEN comparisons.
+
+ If AVAILABLE does not modify HAYSTACK_LEN (as in memmem), then at
+ most 2 * HAYSTACK_LEN - NEEDLE_LEN comparisons occur in searching.
+ If AVAILABLE modifies HAYSTACK_LEN (as in strstr), then at most 3 *
+ HAYSTACK_LEN - NEEDLE_LEN comparisons occur in searching. */
+static RETURN_TYPE
+two_way_short_needle (const unsigned char *haystack, size_t haystack_len,
+ const unsigned char *needle, size_t needle_len)
+{
+ size_t i; /* Index into current byte of NEEDLE. */
+ size_t j; /* Index into current window of HAYSTACK. */
+ size_t period; /* The period of the right half of needle. */
+ size_t suffix; /* The index of the right half of needle. */
+
+ /* Factor the needle into two halves, such that the left half is
+ smaller than the global period, and the right half is
+ periodic (with a period as large as NEEDLE_LEN - suffix). */
+ suffix = critical_factorization (needle, needle_len, &period);
+
+ /* Perform the search. Each iteration compares the right half
+ first. */
+ if (CMP_FUNC (needle, needle + period, suffix) == 0)
+ {
+ /* Entire needle is periodic; a mismatch can only advance by the
+ period, so use memory to avoid rescanning known occurrences
+ of the period. */
+ size_t memory = 0;
+ j = 0;
+ while (AVAILABLE (haystack, haystack_len, j, needle_len))
+ {
+ /* Scan for matches in right half. */
+ i = MAX (suffix, memory);
+ while (i < needle_len && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ ++i;
+ if (needle_len <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (memory < i + 1 && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ --i;
+ if (i + 1 < memory + 1)
+ return (RETURN_TYPE) (haystack + j);
+ /* No match, so remember how many repetitions of period
+ on the right half were scanned. */
+ j += period;
+ memory = needle_len - period;
+ }
+ else
+ {
+ j += i - suffix + 1;
+ memory = 0;
+ }
+ }
+ }
+ else
+ {
+ /* The two halves of needle are distinct; no extra memory is
+ required, and any mismatch results in a maximal shift. */
+ period = MAX (suffix, needle_len - suffix) + 1;
+ j = 0;
+ while (AVAILABLE (haystack, haystack_len, j, needle_len))
+ {
+ /* Scan for matches in right half. */
+ i = suffix;
+ while (i < needle_len && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ ++i;
+ if (needle_len <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (i != SIZE_MAX && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ --i;
+ if (i == SIZE_MAX)
+ return (RETURN_TYPE) (haystack + j);
+ j += period;
+ }
+ else
+ j += i - suffix + 1;
+ }
+ }
+ return NULL;
+}
+
+/* Return the first location of non-empty NEEDLE within HAYSTACK, or
+ NULL. HAYSTACK_LEN is the minimum known length of HAYSTACK. This
+ method is optimized for LONG_NEEDLE_THRESHOLD <= NEEDLE_LEN.
+ Performance is guaranteed to be linear, with an initialization cost
+ of 3 * NEEDLE_LEN + (1 << CHAR_BIT) operations.
+
+ If AVAILABLE does not modify HAYSTACK_LEN (as in memmem), then at
+ most 2 * HAYSTACK_LEN - NEEDLE_LEN comparisons occur in searching,
+ and sublinear performance O(HAYSTACK_LEN / NEEDLE_LEN) is possible.
+ If AVAILABLE modifies HAYSTACK_LEN (as in strstr), then at most 3 *
+ HAYSTACK_LEN - NEEDLE_LEN comparisons occur in searching, and
+ sublinear performance is not possible. */
+static RETURN_TYPE
+two_way_long_needle (const unsigned char *haystack, size_t haystack_len,
+ const unsigned char *needle, size_t needle_len)
+{
+ size_t i; /* Index into current byte of NEEDLE. */
+ size_t j; /* Index into current window of HAYSTACK. */
+ size_t period; /* The period of the right half of needle. */
+ size_t suffix; /* The index of the right half of needle. */
+ size_t shift_table[1U << CHAR_BIT]; /* See below. */
+
+ /* Factor the needle into two halves, such that the left half is
+ smaller than the global period, and the right half is
+ periodic (with a period as large as NEEDLE_LEN - suffix). */
+ suffix = critical_factorization (needle, needle_len, &period);
+
+ /* Populate shift_table. For each possible byte value c,
+ shift_table[c] is the distance from the last occurrence of c to
+ the end of NEEDLE, or NEEDLE_LEN if c is absent from the NEEDLE.
+ shift_table[NEEDLE[NEEDLE_LEN - 1]] contains the only 0. */
+ for (i = 0; i < 1U << CHAR_BIT; i++)
+ shift_table[i] = needle_len;
+ for (i = 0; i < needle_len; i++)
+ shift_table[CANON_ELEMENT (needle[i])] = needle_len - i - 1;
+
+ /* Perform the search. Each iteration compares the right half
+ first. */
+ if (CMP_FUNC (needle, needle + period, suffix) == 0)
+ {
+ /* Entire needle is periodic; a mismatch can only advance by the
+ period, so use memory to avoid rescanning known occurrences
+ of the period. */
+ size_t memory = 0;
+ size_t shift;
+ j = 0;
+ while (AVAILABLE (haystack, haystack_len, j, needle_len))
+ {
+ /* Check the last byte first; if it does not match, then
+ shift to the next possible match location. */
+ shift = shift_table[CANON_ELEMENT (haystack[j + needle_len - 1])];
+ if (0 < shift)
+ {
+ if (memory && shift < period)
+ {
+ /* Since needle is periodic, but the last period has
+ a byte out of place, there can be no match until
+ after the mismatch. */
+ shift = needle_len - period;
+ memory = 0;
+ }
+ j += shift;
+ continue;
+ }
+ /* Scan for matches in right half. The last byte has
+ already been matched, by virtue of the shift table. */
+ i = MAX (suffix, memory);
+ while (i < needle_len - 1 && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ ++i;
+ if (needle_len - 1 <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (memory < i + 1 && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ --i;
+ if (i + 1 < memory + 1)
+ return (RETURN_TYPE) (haystack + j);
+ /* No match, so remember how many repetitions of period
+ on the right half were scanned. */
+ j += period;
+ memory = needle_len - period;
+ }
+ else
+ {
+ j += i - suffix + 1;
+ memory = 0;
+ }
+ }
+ }
+ else
+ {
+ /* The two halves of needle are distinct; no extra memory is
+ required, and any mismatch results in a maximal shift. */
+ size_t shift;
+ period = MAX (suffix, needle_len - suffix) + 1;
+ j = 0;
+ while (AVAILABLE (haystack, haystack_len, j, needle_len))
+ {
+ /* Check the last byte first; if it does not match, then
+ shift to the next possible match location. */
+ shift = shift_table[CANON_ELEMENT (haystack[j + needle_len - 1])];
+ if (0 < shift)
+ {
+ j += shift;
+ continue;
+ }
+ /* Scan for matches in right half. The last byte has
+ already been matched, by virtue of the shift table. */
+ i = suffix;
+ while (i < needle_len - 1 && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ ++i;
+ if (needle_len - 1 <= i)
+ {
+ /* Scan for matches in left half. */
+ i = suffix - 1;
+ while (i != SIZE_MAX && (CANON_ELEMENT (needle[i])
+ == CANON_ELEMENT (haystack[i + j])))
+ --i;
+ if (i == SIZE_MAX)
+ return (RETURN_TYPE) (haystack + j);
+ j += period;
+ }
+ else
+ j += i - suffix + 1;
+ }
+ }
+ return NULL;
+}
+
+#undef AVAILABLE
+#undef CANON_ELEMENT
+#undef MAX
+#undef RETURN_TYPE
diff --git a/lib/strstr.c b/lib/strstr.c
index 80b18b0..338be82 100644
--- a/lib/strstr.c
+++ b/lib/strstr.c
@@ -25,370 +25,17 @@
/* Specification of strstr. */
#include <string.h>
-#include <limits.h>
#include <stdbool.h>
-#include <stddef.h>
-#include <stdint.h>
#ifndef _LIBC
# define __builtin_expect(expr, val) (expr)
#endif
-/* We use the Two-Way string matching algorithm, which guarantees
- linear complexity with constant space. Additionally, for long
- needles, we also use a bad character shift table similar to the
- Boyer-Moore algorithm to achieve better performance.
-
- See http://www-igm.univ-mlv.fr/~lecroq/string/node26.html#SECTION00260
- and http://en.wikipedia.org/wiki/Boyer-Moore_string_search_algorithm
-*/
-
-/* Point at which computing a bad-byte shift table is likely to be
- worthwhile. Small needles should not compute a table, since it
- adds (1 << CHAR_BIT) + NEEDLE_LEN computations of preparation for a
- speedup no greater than a factor of NEEDLE_LEN. The larger the
- needle, the better the potential performance gain. On the other
- hand, on non-POSIX systems with CHAR_BIT larger than eight, the
- memory required for the table is prohibitive. */
-#if CHAR_BIT < 10
-# define LONG_NEEDLE_THRESHOLD 32U
-#else
-# define LONG_NEEDLE_THRESHOLD SIZE_MAX
-#endif
-
-#define MAX(a, b) ((a < b) ? (b) : (a))
-
-/* Perform a critical factorization of NEEDLE, of length NEEDLE_LEN.
- Return the index of the first byte in the right half, and set
- *PERIOD to the global period of the right half.
-
- The global period of a string is the smallest index (possibly its
- length) at which all remaining bytes in the string are repetitions
- of the prefix (the last repetition may be a subset of the prefix).
-
- When NEEDLE is factored into two halves, a local period is the
- length of the smallest word that shares a suffix with the left half
- and shares a prefix with the right half. All factorizations of a
- non-empty NEEDLE have a local period of at least 1 and no greater
- than NEEDLE_LEN.
-
- A critical factorization has the property that the local period
- equals the global period. All strings have at least one critical
- factorization with the left half smaller than the global period.
-
- Given an ordered alphabet, a critical factorization can be computed
- in linear time, with 2 * NEEDLE_LEN comparisons, by computing the
- larger of two ordered maximal suffixes. The ordered maximal
- suffixes are determined by lexicographic comparison of
- periodicity. */
-static size_t
-critical_factorization (const unsigned char *needle, size_t needle_len,
- size_t *period)
-{
- /* Index of last byte of left half, or SIZE_MAX. */
- size_t max_suffix, max_suffix_rev;
- size_t j; /* Index into NEEDLE for current candidate suffix. */
- size_t k; /* Offset into current period. */
- size_t p; /* Intermediate period. */
- unsigned char a, b; /* Current comparison bytes. */
-
- /* Invariants:
- 0 <= j < NEEDLE_LEN - 1
- -1 <= max_suffix{,_rev} < j (treating SIZE_MAX as if it were signed)
- min(max_suffix, max_suffix_rev) < global period of NEEDLE
- 1 <= p <= global period of NEEDLE
- p == global period of the substring NEEDLE[max_suffix{,_rev}+1...j]
- 1 <= k <= p
- */
-
- /* Perform lexicographic search. */
- max_suffix = SIZE_MAX;
- j = 0;
- k = p = 1;
- while (j + k < needle_len)
- {
- a = needle[j + k];
- b = needle[max_suffix + k];
- if (a < b)
- {
- /* Suffix is smaller, period is entire prefix so far. */
- j += k;
- k = 1;
- p = j - max_suffix;
- }
- else if (a == b)
- {
- /* Advance through repetition of the current period. */
- if (k != p)
- ++k;
- else
- {
- j += p;
- k = 1;
- }
- }
- else /* b < a */
- {
- /* Suffix is larger, start over from current location. */
- max_suffix = j++;
- k = p = 1;
- }
- }
- *period = p;
-
- /* Perform reverse lexicographic search. */
- max_suffix_rev = SIZE_MAX;
- j = 0;
- k = p = 1;
- while (j + k < needle_len)
- {
- a = needle[j + k];
- b = needle[max_suffix_rev + k];
- if (b < a)
- {
- /* Suffix is smaller, period is entire prefix so far. */
- j += k;
- k = 1;
- p = j - max_suffix_rev;
- }
- else if (a == b)
- {
- /* Advance through repetition of the current period. */
- if (k != p)
- ++k;
- else
- {
- j += p;
- k = 1;
- }
- }
- else /* a < b */
- {
- /* Suffix is larger, start over from current location. */
- max_suffix_rev = j++;
- k = p = 1;
- }
- }
-
- /* Choose the longer suffix. Return the first byte of the right
- half, rather than the last byte of the left half. */
- if (max_suffix_rev + 1 < max_suffix + 1)
- return max_suffix + 1;
- *period = p;
- return max_suffix_rev + 1;
-}
-
-/* Return the first location of NEEDLE within HAYSTACK, or NULL. This
- method requires 0 < NEEDLE_LEN <= HAYSTACK_LEN, and is optimized
- for NEEDLE_LEN < LONG_NEEDLE_THRESHOLD. Performance is linear,
- with 2 * NEEDLE_LEN comparisons in preparation, and at most 3 *
- HAYSTACK_LEN - NEEDLE_LEN comparisons in searching. */
-static char *
-two_way_short_needle (const unsigned char *haystack, size_t haystack_len,
- const unsigned char *needle, size_t needle_len)
-{
- size_t i; /* Index into current byte of NEEDLE. */
- size_t j; /* Index into current window of HAYSTACK. */
- size_t period; /* The period of the right half of needle. */
- size_t suffix; /* The index of the right half of needle. */
-
- /* Factor the needle into two halves, such that the left half is
- smaller than the global period, and the right half is
- periodic (with a period as large as NEEDLE_LEN - suffix). */
- suffix = critical_factorization (needle, needle_len, &period);
-
- /* Perform the search. Each iteration compares the right half
- first. */
- if (memcmp (needle, needle + period, suffix) == 0)
- {
- /* Entire needle is periodic; a mismatch can only advance by the
- period, so use memory to avoid rescanning known occurrences
- of the period. */
- size_t memory = 0;
- j = 0;
- while (!memchr (&haystack[haystack_len], '\0',
- j + needle_len - haystack_len)
- && (haystack_len = j + needle_len))
- {
- /* Scan for matches in right half. */
- i = MAX (suffix, memory);
- while (i < needle_len && needle[i] == haystack[i + j])
- ++i;
- if (needle_len <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (memory < i + 1 && needle[i] == haystack[i + j])
- --i;
- if (i + 1 < memory + 1)
- return (char *) (haystack + j);
- /* No match, so remember how many repetitions of period
- on the right half were scanned. */
- j += period;
- memory = needle_len - period;
- }
- else
- {
- j += i - suffix + 1;
- memory = 0;
- }
- }
- }
- else
- {
- /* The two halves of needle are distinct; no extra memory is
- required, and any mismatch results in a maximal shift. */
- period = MAX (suffix, needle_len - suffix) + 1;
- j = 0;
- while (!memchr (&haystack[haystack_len], '\0',
- j + needle_len - haystack_len)
- && (haystack_len = j + needle_len))
- {
- /* Scan for matches in right half. */
- i = suffix;
- while (i < needle_len && needle[i] == haystack[i + j])
- ++i;
- if (needle_len <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (i != SIZE_MAX && needle[i] == haystack[i + j])
- --i;
- if (i == SIZE_MAX)
- return (char *) (haystack + j);
- j += period;
- }
- else
- j += i - suffix + 1;
- }
- }
- return NULL;
-}
-
-/* Return the first location of NEEDLE within HAYSTACK, or NULL. This
- method requires 0 < NEEDLE_LEN <= HAYSTACK_LEN, and is optimized
- for LONG_NEEDLE_THRESHOLD <= NEEDLE_LEN. Performance is linear,
- with 3 * NEEDLE_LEN + (1U << CHAR_BIT) operations in preparation,
- and at most 3 * HAYSTACK_LEN - NEEDLE_LEN comparisons in searching.
- The extra initialization cost allows for as few as HAYSTACK_LEN +
- HAYSTACK_LEN / NEEDLE_LEN. */
-static char *
-two_way_long_needle (const unsigned char *haystack, size_t haystack_len,
- const unsigned char *needle, size_t needle_len)
-{
- size_t i; /* Index into current byte of NEEDLE. */
- size_t j; /* Index into current window of HAYSTACK. */
- size_t period; /* The period of the right half of needle. */
- size_t suffix; /* The index of the right half of needle. */
- size_t shift_table[1U << CHAR_BIT]; /* See below. */
-
- /* Factor the needle into two halves, such that the left half is
- smaller than the global period, and the right half is
- periodic (with a period as large as NEEDLE_LEN - suffix). */
- suffix = critical_factorization (needle, needle_len, &period);
-
- /* Populate shift_table. For each possible byte value c,
- shift_table[c] is the distance from the last occurrence of c to
- the end of NEEDLE, or NEEDLE_LEN if c is absent from the NEEDLE.
- shift_table[NEEDLE[NEEDLE_LEN - 1]] contains the only 0. */
- for (i = 0; i < 1U << CHAR_BIT; i++)
- shift_table[i] = needle_len;
- for (i = 0; i < needle_len; i++)
- shift_table[needle[i]] = needle_len - i - 1;
-
- /* Perform the search. Each iteration compares the right half
- first. */
- if (memcmp (needle, needle + period, suffix) == 0)
- {
- /* Entire needle is periodic; a mismatch can only advance by the
- period, so use memory to avoid rescanning known occurrences
- of the period. */
- size_t memory = 0;
- j = 0;
- while (!memchr (&haystack[haystack_len], '\0',
- j + needle_len - haystack_len)
- && (haystack_len = j + needle_len))
- {
- /* Check the last byte first; if it does not match, then
- shift to the next possible match location. */
- size_t shift = shift_table[haystack[j + needle_len - 1]];
- if (0 < shift)
- {
- if (memory && shift < period)
- {
- /* Since needle is periodic, but the last period has
- a byte out of place, there can be no match until
- after the mismatch. */
- shift = needle_len - period;
- memory = 0;
- }
- j += shift;
- continue;
- }
- /* Scan for matches in right half. The last byte has
- already been matched, by virtue of the shift table. */
- i = MAX (suffix, memory);
- while (i < needle_len - 1 && needle[i] == haystack[i + j])
- ++i;
- if (needle_len - 1 <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (memory < i + 1 && needle[i] == haystack[i + j])
- --i;
- if (i + 1 < memory + 1)
- return (char *) (haystack + j);
- /* No match, so remember how many repetitions of period
- on the right half were scanned. */
- j += period;
- memory = needle_len - period;
- }
- else
- {
- j += i - suffix + 1;
- memory = 0;
- }
- }
- }
- else
- {
- /* The two halves of needle are distinct; no extra memory is
- required, and any mismatch results in a maximal shift. */
- period = MAX (suffix, needle_len - suffix) + 1;
- j = 0;
- while (!memchr (&haystack[haystack_len], '\0',
- j + needle_len - haystack_len)
- && (haystack_len = j + needle_len))
- {
- /* Check the last byte first; if it does not match, then
- shift to the next possible match location. */
- size_t shift = shift_table[haystack[j + needle_len - 1]];
- if (0 < shift)
- {
- j += shift;
- continue;
- }
- /* Scan for matches in right half. The last byte has
- already been matched, by virtue of the shift table. */
- i = suffix;
- while (i < needle_len - 1 && needle[i] == haystack[i + j])
- ++i;
- if (needle_len - 1 <= i)
- {
- /* Scan for matches in left half. */
- i = suffix - 1;
- while (i != SIZE_MAX && needle[i] == haystack[i + j])
- --i;
- if (i == SIZE_MAX)
- return (char *) (haystack + j);
- j += period;
- }
- else
- j += i - suffix + 1;
- }
- }
- return NULL;
-}
+#define RETURN_TYPE char *
+#define AVAILABLE(h, h_l, j, n_l) \
+ (!memchr ((h) + (h_l), '\0', (j) + (n_l) - (h_l)) \
+ && ((h_l) = (j) + (n_l)))
+#include "str-two-way.h"
/* Return the first occurrence of NEEDLE in HAYSTACK. Return HAYSTACK
if NEEDLE is empty, otherwise NULL if NEEDLE is not found in
@@ -434,4 +81,3 @@ strstr (const char *haystack_start, const char *needle_start)
}
#undef LONG_NEEDLE_THRESHOLD
-#undef MAX
diff --git a/modules/memmem-simple b/modules/memmem-simple
index bbe89cd..c84755d 100644
--- a/modules/memmem-simple
+++ b/modules/memmem-simple
@@ -2,6 +2,7 @@ Description:
memmem() function: locate first substring in a buffer.
Files:
+lib/str-two-way.h
lib/memmem.c
m4/memmem.m4
diff --git a/modules/strstr b/modules/strstr
index 4f93df9..251b2d7 100644
--- a/modules/strstr
+++ b/modules/strstr
@@ -2,6 +2,7 @@ Description:
strstr() function: efficiently locate first substring in a buffer.
Files:
+lib/str-two-way.h
lib/strstr.c
m4/strstr.m4
--
1.5.3.5
- strstr speedup,
Eric Blake <=