Re: [libmicrohttpd] Using sendfile for file segment

[Christian Grothoff]

On Tuesday 30 November 2010 00:20:09 Eivind Sarto wrote:
> Thanks for the reply.
> But, suppose I really, really wanted an API that would allow me to send a
> response with only a segment of a file (and using sendfile). I have a
> rather demanding application where I want to store the data in large files
> and serve it to 1000s of clients where each URL will correspond to a
> segment of a large file.  I already have written and tested modules for
> standard web servers like Apache and Nginx, but neither of them have the
> process model that I need, and will not scale as well.
> 
> If I wanted to hack libmicrohttpd and add my own API, it is not obvious to
> me what that would be (I have only looked at the code for a few hours).  I
> cannot add a new function, like
>    MHD_create_response_from_fd_and_off(size, off, fd)
> because the file offset I need to pass needs to be saved in
> MHD_Connection->response_write_position, and the connection context is not
> available when the response is created.

I would have saved the offset (and size) in the response object (adding a new 
field) and added the offset from the response object to the offset that comes 
from MHD just before the sendfile call.  It should take only 5 minutes to do, 
what is work is testing and documenting it...
 
> I would be happy to hack and debug the changes I need, but I would really
> appreciate it if you could point me in the right direction so I could
> possibly avoid trying something stupid that will never work.

I'll make you a deal: you write a testcase (in the style of the other 
testcases we already have) against an API call that corresponds to what you 
had here:

>    MHD_create_response_from_fd_and_off(size, off, fd)

and I'll implement that function for you ;-).
 
Happy hacking

Christian

> Thanks,
> -eivind
> ________________________________________
> From: address@hidden
> address@hidden On Behalf Of Christian
> Grothoff address@hidden Sent: Monday, November 29, 2010 1:47 PM
> To: address@hidden
> Cc: Eivind Sarto
> Subject: Re: [libmicrohttpd] Using sendfile for file segment
> 
> You're right that this is currently not possible.  We'd need to extend the
> API to allow (additionally) passing an offset / length, then this should
> be easy to do.  I'm not opposed to having such a function in principle.
> 
> However, as long as you don't require sendfile AND partial sends, you
> should be fine with the current API.
> 
> Happy hacking,
> 
> Christian
> 
> On Monday, November 29, 2010 03:16:23 am Eivind Sarto wrote:
> > I am looking for a way to use microhttpd to send a response using
> > sendfile, but I do not wish to transfer the entire file, just a segment
> > of the file.
> > 
> > I don't see an interface for transferring just a portion of a file (at
> > some offset + length)
> > and make libmicrohttpd use the linux sendfile system call.
> > 
> > Am I missing something?
> > 
> > -eivind