libmicrohttpd
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [libmicrohttpd] Avoid memory pool starvation for upgraded connection


From: Martin Hejnfelt
Subject: Re: [libmicrohttpd] Avoid memory pool starvation for upgraded connectionss
Date: Tue, 04 Apr 2023 10:21:38 +0200
User-agent: One.com webmail 44.2.11

Hi  Evgeny,

I just tried out SHA da42f22c64357179f3869594ca4001c7525a2a1f and it seems to work fine! Memory pool is reclaimed and repurposed appropriately seemingly.
Thank you for the fix!

Best regards,
Martin Hejnfelt


On March 31, 2023 at 10:40:01 am +02:00, Evgeny Grin <k2k@yandex.ru> wrote:
Hi Martin,

The issue should be fixed in git master.

$ git clone https://git.gnunet.org/libmicrohttpd.git

Could you check it?

--
Wishes,
Evgeny

On 28.03.2023 17:00, Evgeny Grin wrote:
Hi Martin,
Thank you for the report and the analysis.
Indeed, 4-bytes communication is not efficient at all.
It will be fixed in git master in similar way as suggested. In adding. a log warning will be added to notify about such situations.
I'll let you know when fix landed in the master branch.
-- Evgeny
On 24.03.2023 14:36, Martin Hejnfelt wrote:
Hi,

So I'm experiencing an issue with upgraded connections running libmicrohttpd with a thread per connection. What seemingly happens is that after receiving the headers, the connections read buffers are shrunk, and the write buffer is afterwards increased to the full size of the memory pool.

This is never "undone" meaning that *if* a connection is upgraded, the memory pool is "empty" and the upgraded connection then starts using the emergency buffer (e_buf) which by default has a size of 8 bytes, 4 is allocated to write, 4 to read.

This means suddenly all communication between the application and the library and again between the library and the upgraded connection socket  is segmented into 4 byte reads/writes.

Now as such, this doesn't necessarily post a "big" problem, besides it for sure isn't intended functionality, however if the connections are https based, each 4 byte "chunk" is now encrypted, causing a massive overhead (in my case each 4 byte chunk is then encrypted a 26 byte chunk).

The seemingly "obvious" choice I found for "easily" fixing this, at least in my case, was to reenable the disabled function "connection_shrink_write_buffer" and call if after the headers have been sent, just before upgrading the connection. In the bottom is a patch for reference that fixes is in one case at least. I have not determined other possible code paths that might trigger the same problem.

Best regards,
Martin Hejnfelt

avoid-upgraded-connection-memory-starvation.patch
Index: libmicrohttpd-0.9.76/src/microhttpd/connection.c
===================================================================
--- libmicrohttpd-0.9.76.orig/src/microhttpd/connection.c <http://libmicrohttpd-0.9.76.orig/src/microhttpd/connection.c>
+++ libmicrohttpd-0.9.76/src/microhttpd/connection.c
@@ -1612,8 +1612,6 @@ connection_maximize_write_buffer (struct
   return c->write_buffer_size - c->write_buffer_append_offset;
}

-
-#if 0 /* disable unused function */
/**
  * Shrink connection write buffer to the size of unsent data.
  *
@@ -1654,10 +1652,6 @@ connection_shrink_write_buffer (struct M
     c->write_buffer = new_buf;
}

-
-#endif /* unused function */
-
-
/**
  * Switch connection from recv mode to send mode.
  *
@@ -4700,6 +4694,8 @@ MHD_connection_handle_idle (struct MHD_C
#ifdef UPGRADE_SUPPORT
       if (NULL != connection->response->upgrade_handler)
       {
+        connection_shrink_write_buffer(connection);
+
         connection->state = MHD_CONNECTION_UPGRADE;
         /* This connection is "upgraded".  Pass socket to application. */
         if (MHD_NO ==


reply via email to

[Prev in Thread] Current Thread [Next in Thread]