Re: [Qemu-devel] Intermittant linux kernel panic on arm

[Quentin Barnes]

On Fri, Mar 09, 2007 at 07:05:27PM -0500, Rob Landley wrote:
> On Wednesday 07 March 2007 1:14 pm, Quentin Barnes wrote:
> > This is my first post to the list.  Hopefully, it will go well.
> >
> > I've been using the ARM qemu for Linux development for some basic
> > work, but wanted to expand and do more with it.  I outgrew the
> > initrd limitation and needed a disk.  Since I've been using a disk,
> > I've been getting intermittant panics during the udevd phase of
> > boot.  Once the system is up though, it's been stable.
> >
> > The panic occurs about 50%-75% the time.  I've had this problem on both
> > 0.9.0 and the 2007-03-07_05 snapshot.  I've had this problem using both
> > my own 2.6.19-1 ARM kernel made directly from kernel.org as well as
> > Aurelien Jarno's 2.6.18 Debian ARM kernel at
> > http://people.debian.org/~aurel32/arm-versatile/vmlinuz-2.6.18-4-versatile 
>
> The first thing I'd do is try to figure out what's udev doing to trigger this 
> panic?
>
> >  r7 = 00000036  r6 = 00002285  r5 = FFFFFFF7  r4 = C0BA0D20
> > [<c0089334>] (sys_ioctl+0x0/0x64) from [<c00228c0>] 
> (ret_fast_syscall+0x0/0x2c)
>
> Is there any way you can figure out which ioctl this is?
> Presumably udev read something from /sys that told it to mknod
> something in /dev.  I'm not quite sure where an ioctl comes into
> this...
>
> If you could get a small C program that triggers the panic, and a
> kernel .config you built your kernel with, that would be helpful.

I don't know if writing a small C program would trigger the panic.
The same ioctl happens earlier in the startup which doesn't panic.
However, I could still give it a try at some point if we have
no other ideas.

I ioctl is for an SG_IO which is doing a SCSI inquiry command:
==============
Breakpoint 3, scsi_dispatch_cmd (cmd=0xc7db7180) at drivers/scsi/scsi.c:475
475             struct Scsi_Host *host = cmd->device->host;
1: x/i $pc  0xc019e7f4 <scsi_dispatch_cmd+12>:  ldr     r1, [r0]
(gdb) print /c cmd->cmnd
$8 = {0x12, 0x1, 0x0, 0x0, 0xfe, 0x0 <repeats 11 times>}
==============

The SCSI inquiry command is properly formed and dispatched for a EVPD=1
to do a VPD read of 0x00.

It calls: sym_interrupt() -> sym_wakeup_done() -> sym_complete_ok().

In sym_complete_ok(), it executes:
        if (cp->phys.head.lastp != cp->goalp)
                resid = sym_compute_residual(np, cp);

cp->phys.head.lastp is 0x50000b10 and cp->goalp is 0x500007f8.  Since
they're not equal, the driver thinks there is a residual.

cp->startp is 0x500007e8 which seems to make sense to me.  I would
expect "lastp" to be between "startp" and "goalp", but it's not,
however, I'm just guessing here since I don't know SCSI at all.

Any ideas what might be wrong?


Partial contents of "cp" that leads up to panic:
==============
$17 = {phys = {head = {go = {start = 0x50000058, restart = 0x500004c0}, 
     savep = 0x500007e8, lastp = 0x50000b10, status = "\000\204\000@"}, 
[...]
 order = 0x20, odd_byte_adjustment = 0x0, nego_status = 0x1, 
 xerr_status = 0x0, extra_bytes = 0x0, 
 scsi_smsg = "\xc0 g\001\003\001\n\037\000\000\000", 
 scsi_smsg2 = '\0' <repeats 11 times>, sensecmd = "\000\000\000\000\000", 
 sv_scsi_status = 0x0, sv_xerr_status = 0x0, sv_resid = 0x0, 
 ccb_ba = 0x7db3c00, tag = 0x33, target = 0x0, lun = 0x0, link_ccbh = 0x0, 
 link_ccbq = {flink = 0xffd004fc, blink = 0xffd004fc}, startp = 0x500007e8, 
 goalp = 0x500007f8, ext_sg = 0xffffffff, ext_ofs = 0x0, to_abort = 0x0, 
 tags_si = 0x1}
==============

A strange thing to note is that this panic is only intermittent when
in graphics mode, but happens 100% of the time when qemu is in tty
console mode.  If the boot makes it past this point, this system is
really stable.  I've done hours of builds on it without it falling
over.

> Rob
> --
> Vista: Windows Millenium Second Edition

Quentin