[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 15/20] nvme: add support for scatter gather lists
From: |
Beata Michalska |
Subject: |
Re: [PATCH v2 15/20] nvme: add support for scatter gather lists |
Date: |
Mon, 25 Nov 2019 14:10:37 +0000 |
On Mon, 25 Nov 2019 at 06:21, Klaus Birkelund <address@hidden> wrote:
>
> On Tue, Nov 12, 2019 at 03:25:18PM +0000, Beata Michalska wrote:
> > Hi Klaus,
> >
> > On Tue, 15 Oct 2019 at 11:57, Klaus Jensen <address@hidden> wrote:
> > > +static uint16_t nvme_map_sgl(NvmeCtrl *n, QEMUSGList *qsg,
> > > + NvmeSglDescriptor sgl, uint32_t len, NvmeRequest *req)
> > > +{
> > > + const int MAX_NSGLD = 256;
> > > +
> > > + NvmeSglDescriptor segment[MAX_NSGLD];
> > > + uint64_t nsgld;
> > > + uint16_t status;
> > > + bool sgl_in_cmb = false;
> > > + hwaddr addr = le64_to_cpu(sgl.addr);
> > > +
> > > + trace_nvme_map_sgl(req->cid, NVME_SGL_TYPE(sgl.type), req->nlb, len);
> > > +
> > > + pci_dma_sglist_init(qsg, &n->parent_obj, 1);
> > > +
> > > + /*
> > > + * If the entire transfer can be described with a single data block
> > > it can
> > > + * be mapped directly.
> > > + */
> > > + if (NVME_SGL_TYPE(sgl.type) == SGL_DESCR_TYPE_DATA_BLOCK) {
> > > + status = nvme_map_sgl_data(n, qsg, &sgl, 1, &len, req);
> > > + if (status) {
> > > + goto unmap;
> > > + }
> > > +
> > > + goto out;
> > > + }
> > > +
> > > + /*
> > > + * If the segment is located in the CMB, the submission queue of the
> > > + * request must also reside there.
> > > + */
> > > + if (nvme_addr_is_cmb(n, addr)) {
> > > + if (!nvme_addr_is_cmb(n, req->sq->dma_addr)) {
> > > + return NVME_INVALID_USE_OF_CMB | NVME_DNR;
> > > + }
> > > +
> > > + sgl_in_cmb = true;
> > > + }
> > > +
> > > + while (NVME_SGL_TYPE(sgl.type) == SGL_DESCR_TYPE_SEGMENT) {
> > > + bool addr_is_cmb;
> > > +
> > > + nsgld = le64_to_cpu(sgl.len) / sizeof(NvmeSglDescriptor);
> > > +
> > > + /* read the segment in chunks of 256 descriptors (4k) */
> > > + while (nsgld > MAX_NSGLD) {
> > > + nvme_addr_read(n, addr, segment, sizeof(segment));
> > Is there any chance this will go outside the CMB?
> >
>
> Yes, there certainly was a chance of that. This has been fixed in a
> general way for both nvme_map_sgl and nvme_map_sgl_data.
>
> > > +
> > > + status = nvme_map_sgl_data(n, qsg, segment, MAX_NSGLD, &len,
> > > req);
> > > + if (status) {
> > > + goto unmap;
> > > + }
> > > +
> > > + nsgld -= MAX_NSGLD;
> > > + addr += MAX_NSGLD * sizeof(NvmeSglDescriptor);
> > > + }
> > > +
> > > + nvme_addr_read(n, addr, segment, nsgld *
> > > sizeof(NvmeSglDescriptor));
> > > +
> > > + sgl = segment[nsgld - 1];
> > > + addr = le64_to_cpu(sgl.addr);
> > > +
> > > + /* an SGL is allowed to end with a Data Block in a regular
> > > Segment */
> > > + if (NVME_SGL_TYPE(sgl.type) == SGL_DESCR_TYPE_DATA_BLOCK) {
> > > + status = nvme_map_sgl_data(n, qsg, segment, nsgld, &len,
> > > req);
> > > + if (status) {
> > > + goto unmap;
> > > + }
> > > +
> > > + goto out;
> > > + }
> > > +
> > > + /* do not map last descriptor */
> > > + status = nvme_map_sgl_data(n, qsg, segment, nsgld - 1, &len,
> > > req);
> > > + if (status) {
> > > + goto unmap;
> > > + }
> > > +
> > > + /*
> > > + * If the next segment is in the CMB, make sure that the sgl was
> > > + * already located there.
> > > + */
> > > + addr_is_cmb = nvme_addr_is_cmb(n, addr);
> > > + if ((sgl_in_cmb && !addr_is_cmb) || (!sgl_in_cmb &&
> > > addr_is_cmb)) {
> > > + status = NVME_INVALID_USE_OF_CMB | NVME_DNR;
> > > + goto unmap;
> > > + }
> > > + }
> > > +
> > > + /*
> > > + * If the segment did not end with a Data Block or a Segment
> > > descriptor, it
> > > + * must be a Last Segment descriptor.
> > > + */
> > > + if (NVME_SGL_TYPE(sgl.type) != SGL_DESCR_TYPE_LAST_SEGMENT) {
> > > + trace_nvme_err_invalid_sgl_descriptor(req->cid,
> > > + NVME_SGL_TYPE(sgl.type));
> > > + return NVME_SGL_DESCRIPTOR_TYPE_INVALID | NVME_DNR;
> > Shouldn't we handle a case here that requires calling unmap ?
>
> Woops. Fixed.
>
> > > +static uint16_t nvme_dma_read_sgl(NvmeCtrl *n, uint8_t *ptr, uint32_t
> > > len,
> > > + NvmeSglDescriptor sgl, NvmeCmd *cmd, NvmeRequest *req)
> > > +{
> > > + QEMUSGList qsg;
> > > + uint16_t err = NVME_SUCCESS;
> > > +
> > Very minor: Mixing convention: status vs error
> >
>
> Fixed by proxy in another refactor.
>
> > >
> > > +#define NVME_CMD_FLAGS_FUSE(flags) (flags & 0x3)
> > > +#define NVME_CMD_FLAGS_PSDT(flags) ((flags >> 6) & 0x3)
> > Minor: This one is slightly misleading - as per the naming and it's usage:
> > the PSDT is a field name and as such does not imply using SGLs
> > and it is being used to verify if given command is actually using
> > SGLs.
> >
>
> Ah, is this because I do
>
> if (NVME_CMD_FLAGS_PSDT(cmd->flags)) {
>
> in the code? That is, just checks for it not being zero? The value of
> the PRP or SGL for Data Transfer (PSDT) field *does* specify if the
> command uses SGLs or not. 0x0: PRPs, 0x1 SGL for data, 0x10: SGLs for
> both data and metadata. Would you prefer the condition was more
> explicit?
>
Yeah, it is just not obvious( at least to me) without referencing the spec
that non-zero value implies SGL usage. Guess a comment would be helpful
but that is not major.
Thanks,
BR
Beata
>
> Thanks!
> Klaus