Re: [Qemu-devel] [Nbd] [PATCH 3/1] doc: Propose Structured Replies extension

[Alex Bligh]

Eric,

> I guess what I need to add is that in transmission phase, most commands
> have exactly one response per request; but commands may document
> scenarios where there will be multiple responses to a single request.
> NBD_CMD_READ uses the multiple responses to make partial read and error
> handling possible

Yes, this.

> Yeah, but the reconstruction is easy; naively:
> 
> while response_magic == structured:
>    copy len-8 bytes of data from response to given offset
> response_magic == normal, read is complete

It's easy if the result is written to memory. It's not easy if the
purpose was (e.g.) to send it to a socket in a sendfile type way.
It now requires the entire response be held in memory, which wasn't
a requirement before.

> Detecting overlap or incomplete reads would requires more complexity in
> the client, but I don't know that a client has to care (the protocol is
> specifically written that a client MAY detect bad servers, but not MUST;
> a client that assumes the server is well-behaved is still compliant).

Yep

> However, you DO have a point that the server SHOULD send data in
> reasonable-size chunks; and maybe I should propose a parallel extension
> where, when negotiated between client and server, the server will
> advertise minimum and preferred I/O sizes in response to the export name
> request (for example, a server backed by a real block device may have a
> minimum of 512 bytes or 4096 bytes, and a preferred size of 64k; while a
> server backed by a normal file system may have a minimum of 1 byte);
> then put in restrictions that a server SHOULD reject read/write requests
> where offset and length are not multiples of the minimum, and that the
> server SHOULD send read chunks aligned to the preferred size (with
> exceptions for the head and tail of a larger buffer that meets minimum
> alignment but not preferred alignment).

What I'm really after is something that enables me to read 'nicely'
in a manner where I won't get fragments.

>> Also, given new commands aren't available unless you support structured
>> replies, you now have to support reassembly of replies (if you want
>> to use new features) even if all your reads are (e.g.) 1k.
> 
> Are you arguing that there should be a flag that controls whether reads
> must be in-order vs. reassembled?  Reassembly must happen either way,
> the question is whether having a way to allow out-of-order for
> efficiency, vs. defaulting to in-order for easier computation, is worth it.

No, that sounds overengineered.

More a way of guaranteeing avoiding a fragmentation on 'simple' reads.
Perhaps a 'DF' bit (don't fragment)! If the server doesn't like it, it
can always error the command.

>>> The server
>>> +  MUST NOT set the error field of a read chunk; if an error occurs, it
>>> +  MAY immediately end the sequence of structured response messages,
>>> +  MUST send the error in the concluding normal response, and SHOULD
>>> +  keep the connection open.  The final non-structured response MUST
>>> +  set an error unless the sum of data sent by all read chunks totals
>>> +  the original client length request.
>> 
>> add "and data for the entire range requested has been supplied." (I
>> know this is technically implied by the fact data cannot be duplicated).
> 
> Sure.  But keep in mind that if (when?) we add a flag for allowing the
> server to skip read chunks on holes, we'll have to tweak the wording to
> allow the server to send fewer chunks than the client's length, where
> the client must then assume zeroes for all chunks not received.

Or alternatively a chunk representing a hole. I wonder whether you
might be better to extend the chunk structure by 4 bytes to allow for
future modifications like this (e.g. NBD_CHUNK_FLAG_HOLE means
the chunk is full of zeroes and has no payload).

--
Alex Bligh

signature.asc
Description: Message signed with OpenPGP using GPGMail