Re: [lwip-users] PolarSSL and mbedTLS

[Noam Weissman]

Hi Jan,

I was able to trace were the error happens but I cannot figure a solution.

The SSL layer is allocation a buffer of 16K, transfers it to the underlying 
code and eventually

lwip_recvfrom is called.

Inside this function there is a do <> while loop that is supposed to collect 
the fragments into the 16K buffer
and after getting it all return to the SSL for decryption etc...

for some reason it always returns with EWOULDBLOCK error !

I changed my lwipopts and now I have 
#define MEMP_NUM_PBUF           80
#define MEM_SIZE                (22 * 1024)

As far as I understand that should be ok but I still get the error on the first 
loop iteration ?

I am sure it is an LwIP setting issue and not lack of memory 


BR,
Noam.


-----Original Message-----
From: lwip-users [mailto:address@hidden On Behalf Of Jan Menzel
Sent: Thursday, March 02, 2017 8:08 PM
To: address@hidden
Subject: Re: [lwip-users] PolarSSL and mbedTLS

Hi Noam!
        With MAX_CONTENT_LEN set to 16k you have to provide more then 32k to 
the SSL stack. With 64k ssl memory pool, that should be fine. You'll need more 
memory for extensive math in case you enable enhanced features like DHE or 
certificate validation.
        I'm not aware of any MSS related issues. Every time you call ssl_read() 
or ssl_write() some internal sm will check for any pending work to be done. If 
there is sufficient data available (in case of read) it will be returned to 
you. If not, lwips read() is called to check if new data has arrived. Data 
arrives as per TCP standard in any size and is buffered in the receive buffer 
of the SSL layer. Once a complete block (with respect to ssl) has been 
received, it is checked, decrypted and set as ready to be transferred to the 
application layer. If no data is available
ssl_read() will return something like WONT_READ. This just indicated that you 
shall try again later. So if large blocks do not arrive, you'll probably have 
to call ssl_read() a few times until all data is available. If thats not 
working for you, try if lwips read() is called in a non-blocking way. mbedtls 
default interface layer does some checking on the socked to detect if data is 
available and/or the read() call would block. I had to play a little bit around 
with errno to get it working.

        Jan

On 01.03.2017 23:19, Noam Weissman wrote:
> Hi Jan,
> 
> 
> Let me see if I understand you...  MAX_CONTENT_LEN  is defined in my 
> system to the default 16K.
> 
> I have 64K of RAM available just for the SSL use.
> 
> 
> So memory limits are not at SSL layer, I think.
> 
> 
> The SSL layer is reading a record and then the data. Data is read into 
> its own buffers. If I understand
> 
> you correctly because it tries to read as it needs 4,8 or maybe even 
> 16k and because my MSS on
> 
> TCP stack is defined to be small it is a problem.
> 
> 
> But what if MSS has a normal size or even defined as jumbo frame (8K) 
> it still will not hold
> 
> a 16K frame in one pBuf if the sender is sending a large chank. So 
> there should be something else ?
> 
> 
> Noam.
> ----------------------------------------------------------------------
> --
> *From:* lwip-users <address@hidden> on 
> behalf of Jan Menzel <address@hidden>
> *Sent:* Wednesday, March 1, 2017 11:46 PM
> *To:* address@hidden
> *Subject:* Re: [lwip-users] PolarSSL and mbedTLS
>  
> Hi Noam!
>         SSL/TLS isn't that simple. Besides encryption the data is also 
> hashed to detect data integrity issues. This places strong resource 
> requirements on the block size that is used for hashing and this is 
> where MAX_CONTENT_LEN comes into play: mbedtls needs/reserves two 
> buffers of that size, one for RX and one for TX. So if RAM usage is an 
> issue for you, reduce MAX_CONTENT_LEN and make sure you have control 
> over the other side (MAX_CONTENT_LEN is not negotiated).
>         From lwips memory concept it would be great if mbedtls would 
> just chain up pbufs until a block is complete and can be rehashed and 
> decrypted.
> I'd willing to assist in implementing this.
> 
>         Jan
> 
> On 01.03.2017 21:46, Noam Weissman wrote:
>> Hi Simon,
>> 
>> 
>> Yes it may be an issue, any ideas or a change in the settings I 
>> placed here earlier ?
>> 
>> 
>> What I am puzzled about is that SSL/TLS transfer a key over RSA. Once 
>> the key has been
>> 
>> transferred (SSL handshake) the encryption/decryption are symmetric.
>> Either using AES, DES etc..
>> 
>> 
>> So if one side wants to send 100 bytes or 500 bytes the overhead is 
>> small. Normally AES, DES
>> 
>> etc can encrypt/decrypt IN PLACE so no need for a new buffer. So 
>> memory usage should not be
>> 
>> a big difference?
>> 
>> 
>> Any ideas are welcomed 😊
>> 
>> 
>> BR,
>> 
>> Noam.
>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> ---
>> *From:* lwip-users <address@hidden> on 
>> behalf of address@hidden <address@hidden>
>> *Sent:* Wednesday, March 1, 2017 9:26 PM
>> *To:* Mailing list for lwIP users
>> *Subject:* Re: [lwip-users] PolarSSL and mbedTLS
>>  
>> I did have mbedTLS running against our httpd (no sockets) but the 
>> resource usage was rather high.
>> I'd imagine the problem could be lwIP's memory configurations here, too.
>> 
>> Simon
>> 
>> 
>> Jan Menzel wrote:
>>> Hi Noam!
>>>        I've designed a system with almost the same setup which works 
>>> well since a few years incl. firmware updates of a ~200kb.
>>>        Did you checked the memory consumption of the ip stack and 
>>> the ssl max content length setting? The default max content length 
>>> setting is IIRC 16kb, which means that data is hashed and encrypted 
>>> in chunks of up to 16kb and can only be verified and decrypted once 
>>> the entire chunk has been received. The firmware update on my system 
>>> only works if the max content length is reduced. With the default 
>>> setting I faced memory issues on LPC1768 (which has just a fraction of your 
>>> F4xx).
>>>        I also had to fiddle around a little bit with errno in the 
>>> interface between mbedtls and lwip. One last advice: carefully check 
>>> your stack usage. mbedtls uses lots of function pointers which Keils 
>>> static call graph analysis can not see and does not warn about.
>>>
>>>        Jan
>>>
>>> On 01.03.2017 14:01, Noam Weissman wrote:
>>>> Hi,
>>>>
>>>>   
>>>>
>>>> I have a client, single task using the socket API using and also 
>>>> PolarSSL for SSL support.
>>>>
>>>>   
>>>>
>>>> The client is WebSocket client and all seems to work ok.
>>>>
>>>>   
>>>>
>>>> When I try to send small messages from the server to my client all 
>>>> is working ok but when I try to push a large
>>>>
>>>> message 6K and up my ssl_read function fails with a read error?.
>>>>
>>>>   
>>>>
>>>> The ssl_read is actually calling lwip_read internally.
>>>>
>>>>   
>>>>
>>>> For some reason the SSL code is trying to read a large buffer 8-16K 
>>>> bytes and the read function fails.
>>>>
>>>>   
>>>>
>>>> Normally when we read from a socket more than is available the 
>>>> return value should be the number
>>>>
>>>> of bytes actually read and not an error ?.
>>>>
>>>>   
>>>>
>>>> The processor is STM32F427 using CCM for heap and Keil IDE
>>>>
>>>>   
>>>>
>>>> My main project uses Lwip 1.41, FreeRTOS 8.0.1 and PolarSSL 1.0.0
>>>>
>>>>   
>>>>
>>>> I have created two almost identical projects to the one I use. The 
>>>> first
>>>> uses:
>>>>
>>>> Lwip 2.01, FreeRTOS 9.0
>>>>
>>>>   
>>>>
>>>> The second project is the same as the one with Lwip 2.01 but 
>>>> instead of PolaSSL I switched to mbedTLS 2.4.0
>>>>
>>>>   
>>>>
>>>> In none secure mode everything works as expected and have no 
>>>> problems getting a large message (600K)
>>>>
>>>> In secured mode I get a read fail on the first packet ??
>>>>
>>>>   
>>>>
>>>> Anyone has an idea what I am doing wrong or what setting are not correct ??
>>>>
>>>>   
>>>>
>>>> A second question for Simon or anyone that can assist. I tried to 
>>>> set LWIP_DEBUG to 1 and my total used RAM (compiler) dropped
>>>>
>>>> about 30K ?? Why is that ?... I understood that debug should take 
>>>> more RAM not Less ?
>>>>
>>>>   
>>>>
>>>> Thanks,
>>>>
>>>> Noam.
>>>>
>>>>   
>>>>
>>>> cid:image001.jpg@01D26A92.68494F10
>>>>
>>>>       
>>>>
>>>> Noam Weissman
>>>>
>>>> Software Engineer
>>>>
>>>> SILORA R&D
>>>>
>>>> p:
>>>>
>>>>       
>>>>
>>>> +972-4-9554915 m: +972-52-5786135
>>>>
>>>> w:
>>>>
>>>>       
>>>>
>>>> www.silrd.com <http://www.silrd.com> <http://www.silrd.com>
> <http://www.silrd.com/>  e:
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> 
>> address@hidden
>> SILORA R&D - The Vision of Innovation <http://www.silrd.com/>
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> 
>> www.silrd.com <http://www.silrd.com>
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> 
>> SILORA R&D provides state of the art multimedia and switching 
>> solutions to leading companies in the ProAV industry. SILORA R&D is 
>> an innovative developer and ...
>> 
>> SILORA R&D - The Vision of Innovation <http://www.silrd.com/>
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> 
>> www.silrd.com <http://www.silrd.com>
> SILORA R&D - The Vision of Innovation <http://www.silrd.com/> 
> www.silrd.com SILORA R&D provides state of the art multimedia and 
> switching solutions to leading companies in the ProAV industry. SILORA 
> R&D is an innovative developer and ...
> 
> 
>> SILORA R&D provides state of the art multimedia and switching 
>> solutions to leading companies in the ProAV industry. SILORA R&D is 
>> an innovative developer and ...
>> 
>> 
>>>> <mailto:address@hidden>
>>>>
>>>> cid:image002.png@01D26A92.68494F10
>>>> <https://www.facebook.com/SiloraRD/>  
>>>> cid:image003.png@01D26A92.68494F10
>>>> <https://twitter.com/SiloraRD>  cid:image004.png@01D26A92.68494F10
>>>> <https://www.linkedin.com/company/silora-r&d>
>>>>
>>>>   
>>>>
>>>>   
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> lwip-users mailing list
>>>> address@hidden
>>>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>>>>
>>> _______________________________________________
>>> lwip-users mailing list
>>> address@hidden
>>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>>>
>> 
>> 
>> _______________________________________________
>> lwip-users mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>> 
>> 
>> _______________________________________________
>> lwip-users mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/lwip-users
>> 
> 
> _______________________________________________
> lwip-users mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/lwip-users
> 
> 
> _______________________________________________
> lwip-users mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/lwip-users
> 

_______________________________________________
lwip-users mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/lwip-users