Hi Martin,
The issue should be fixed in git master.
$ git clone https://git.gnunet.org/libmicrohttpd.git
<https://git.gnunet.org/libmicrohttpd.git>
Could you check it?
--
Wishes,
Evgeny
On 28.03.2023 17:00, Evgeny Grin wrote:
Hi Martin,
Thank you for the report and the analysis.
Indeed, 4-bytes communication is not efficient at all.
It will be fixed in git master in similar way as suggested. In
adding. a log warning will be added to notify about such situations.
I'll let you know when fix landed in the master branch.
-- Evgeny
On 24.03.2023 14:36, Martin Hejnfelt wrote:
Hi,
So I'm experiencing an issue with upgraded connections running
libmicrohttpd with a thread per connection. What seemingly
happens is that after receiving the headers, the connections
read buffers are shrunk, and the write buffer is afterwards
increased to the full size of the memory pool.
This is never "undone" meaning that *if* a connection is
upgraded, the memory pool is "empty" and the upgraded
connection then starts using the emergency buffer (e_buf)
which by default has a size of 8 bytes, 4 is allocated to
write, 4 to read.
This means suddenly all communication between the application
and the library and again between the library and the upgraded
connection socket is segmented into 4 byte reads/writes.
Now as such, this doesn't necessarily post a "big" problem,
besides it for sure isn't intended functionality, however if
the connections are https based, each 4 byte "chunk" is now
encrypted, causing a massive overhead (in my case each 4 byte
chunk is then encrypted a 26 byte chunk).
The seemingly "obvious" choice I found for "easily" fixing
this, at least in my case, was to reenable the disabled
function "connection_shrink_write_buffer" and call if after
the headers have been sent, just before upgrading the
connection. In the bottom is a patch for reference that fixes
is in one case at least. I have not determined other possible
code paths that might trigger the same problem.
Best regards,
Martin Hejnfelt
avoid-upgraded-connection-memory-starvation.patch
Index: libmicrohttpd-0.9.76/src/microhttpd/connection.c
===================================================================
--- libmicrohttpd-0.9.76.orig/src/microhttpd/connection.c
<http://libmicrohttpd-0.9.76.orig/src/microhttpd/connection.c
<http://libmicrohttpd-0.9.76.orig/src/microhttpd/connection.c>>
+++ libmicrohttpd-0.9.76/src/microhttpd/connection.c
@@ -1612,8 +1612,6 @@ connection_maximize_write_buffer (struct
return c->write_buffer_size - c->write_buffer_append_offset;
}
-
-#if 0 /* disable unused function */
/**
* Shrink connection write buffer to the size of unsent data.
*
@@ -1654,10 +1652,6 @@ connection_shrink_write_buffer (struct M
c->write_buffer = new_buf;
}
-
-#endif /* unused function */
-
-
/**
* Switch connection from recv mode to send mode.
*
@@ -4700,6 +4694,8 @@ MHD_connection_handle_idle (struct MHD_C
#ifdef UPGRADE_SUPPORT
if (NULL != connection->response->upgrade_handler)
{
+ connection_shrink_write_buffer(connection);
+
connection->state = MHD_CONNECTION_UPGRADE;
/* This connection is "upgraded". Pass socket to
application. */
if (MHD_NO ==
OpenPGP_0x460A317C3326D2AE.asc