Re: [lwip-users] critical section protection + timer issues questions

[Scott Taggart]

Jim,

Thanks for your responses.

I have a few clarification/comments/questions.

1) To summarize, I think that every thread must, at some point do a sys_sem, a sys_mbox or sys_mssleep call for the LWIP timer stuff to work.

2) I think that if I kludged the sys_arch timeouts stuff to just return a SINGLE timers list for ALL threads that that would break the implicit mutexing that is occurring in LWIP (i.e., LWIP assumes that a timer expiration  (i.e., callback) occurs in the same thread context that scheduled the initial timer).  My guess is that if I hacked this, the entire stack would melt down unless I started to implement my own mutex locks where needed.  Again, am I missing something here?

3) To paraphrase, you say make an initial sys_timeout call after the tcpip_init call completes.  The problem with this is that it only handles timers for THAT thread.  I have other threads that want to establish connections and pass TCP traffic – each of these must, at some point call one of the previously mentioned 3 functions for timers to work.  I find this to be an INCREDIBLE design restriction – I want my threads to wait on my own internal semaphores, etc. and not have to go into the bowels of LWIP for their o/s blocking services.  Am I missing something?  Can anyone suggest a work-around for this?  Is there a multi-threaded port of LWIP that shows how any of this is done?  For that matter, where is the repository of al the LWIP ports – I have never really found one.

Thanks,

Scott

---------------------------------------------------------

Please see below

Scott Taggart wrote:

I have two questions.  The first is regarding multi-threaded code protection and the second involves timers.

First, can someone please explain to me how the lwip code protects itself in a multithreaded environment when threads are using the “raw” interface (tcp_connect(), tcp_write(), etc.)?  For example, I see various api calls (tcp_connect() for example) end up calling into sys_timeout() which manipulates the timer linked list.  Yet there is no thread protection of any kind around this list manipulation code.  Is the calling application supposed to mutex lock all calls into LWIP?  Is it because this list is per-thread that it does not need mutex protection?

I can't really speak about the raw interface.  I've only worked in situations where the higher level interfaces were used and TCP/IP was running in its own thread.  For the most part, the fact that TCP/IP runs in its own thread provides the needed protections.

Even in this case, though, there are some worries.  I would urge you to use SYS_LIGHTWEIGHT_PROT, for example, since I don't believe that any other option provides adequate protection to the PBUF pool.  I would also advise you to avoid trying to use the same socket in two different threads, e.g., pending on a read of the socket in one thread while writing to it in another.

You should also watch out for the way that etharp gets called.  etharp_ip_output tends to get called in the context of the TCP/IP task, while etharp_arp_input is often called from an ethernet receive task.  Depending on how you implement it, the arp timer may end up firing in yet another task.  This has the potential to cause problems, since the arp list doesn't have mutex protection.  In this instance, it is fairly easy to add your own protections, since the etharp calls are going to be in code that you are adding, anyway.

Regarding timers, I am massively confused.  The notes say that sys_arch_timeouts() should return a per-thread linked list head of sys_timeouts.  What I don’t understand is what is triggering these timers to fire?  For example, I see that the macro TCP_REG() ends up calling tcp_timer_needed() which dutifully gets tcpip_tcp_timer() in the timer list.  What I don’t see is where there is any form of timer callback occurring.  Is this supposed to occur in sys_arch somewhere?  I sort of see that if there’s a semaphore or mailbox being used for that thread that the timers will be dealt with but I do not see where it is guaranteed that there is a semaphore or mailbox always in use for a given thread.  For example, if I call tcp_connect() and a resulting timer callback is put into that thread’s timer list, the SYN is sent and then tcp_connect returns.  At this point I do not see how that timer for that thread will fire (since LWIP is not waiting on a mailbox or semaphore for that connection)???  Please help me see what it is that I am missing.

You've got this right.  The timer mechansim rests on the sys.c wrappers for your OS's semaphore and mailbox mechanisms.  When you call sys_timeout, the timeout is created in the list associated with the current task.  That task needs to pend on a sys_sem, a sys_mbox, or sys_mssleep.  The timeout function ends up getting called in the context of that same task, and things work out.  If this is the timeout mechanism that you need, you're all set.  If you need something different, you're on your own.

As an example, consider the tcp timer.  The tcp/ip task gets launched by tcpip_init.  After the tcp/ip task has gotten itself initialized, it invokes a callback function to let the caller of tcpip_init know that the operation is completed.  The callback function is executing in the tcp/ip task context.  That's a perfect place to make the initial sys_timeout call to start the tcp timer.  The tcp/ip thread does use the sys stuff, so the timer will fire.  And when it does fire, the tcp timer will be running in the context of the tcp/ip thread, thus precluding any potential conflicts as it processes through the PCBs.

The tcp timer illustrates the benefits of the approach implemented in sys.c.  When you use a timer, it can be important to control the task context in which the timer code executes.  If you can get it to run in the right task, then you can often avoid resource conflicts without explicitly using mutexes. 

Thanks,

Scott




_______________________________________________

lwip-users mailing list

address@hidden

http://lists.nongnu.org/mailman/listinfo/lwip-users

--

Jim Gibbons     address@hidden 
Gibbons and Associates, Inc.    TEL: (408) 984-1441    
900 Lafayette, Suite 704, Santa Clara, CA       FAX: (408) 247-6395