Re: [Qemu-devel] [PATCH v3 09/19] pc: delay setting number of boot CPUs to machine_done time

[Eduardo Habkost]

On Tue, Jul 12, 2016 at 02:48:43PM +0200, Igor Mammedov wrote:
> On Tue, 12 Jul 2016 00:29:08 -0300
> Eduardo Habkost <address@hidden> wrote:
> 
> > On Wed, Jul 06, 2016 at 08:20:45AM +0200, Igor Mammedov wrote:
> > > currently present CPUs counter in CMOS only contains
> > > smp_cpus (i.e. initial CPUs specified with -smp X) and
> > > doesn't account for CPUs created with -device.
> > > If VM is started with additional CPUs added with
> > >  -device, it will hang in BIOS waiting for condition
> > >    smp_cpus == counted_cpus
> > > forever as counted_cpus will include -device CPUs as well
> > > and be more than smp_cpus.
> > > 
> > > make present CPUs counter in CMOS to count all CPUs
> > > (initial and coldplugged with -device) by delaying
> > > it to machine done time when it possible to count
> > > CPUs added with -device.  
> > 
> > Do you plan to fix the remaining code using smp_cpus? e.g.:
> perhaps after Drew's -smp refactoring or maybe during it,
> it looks like good candidate for that series,
> I'll work with Drew on that.
> 
> > 
> > 1) x86_cpu_realizefn():
> >     if (cpu->env.features[FEAT_1_EDX] & CPUID_APIC || smp_cpus > 1) {
> >             x86_cpu_apic_create(cpu, &local_err);
> >     [...]
> >   (Maybe we should simply make realizefn fail if we try to create a second 
> > CPU
> >   using -device or device_add without CPUID_APIC)
> wouldn't that break some setups that doing it but still able
> to boot?

It would, that's why we need to do it only in the case of -device
or device_add. What about something like:

  if (!(cpu->env.features[FEAT_1_EDX] & CPUID_APIC) && smp_cpus < 2 
&&total_cpus_already_created() > 0) {
      error_setg("we can't create a new VCPU without an APIC");
      return;
  }

I believe this logic should be moved to PC code, eventually. Or
at least the process should be controlled by PC code (by setting
a force-apic-creation property in X86CPU, for example).

> 
> > 
> > 2) the smp_cpus checks at hw/i386/kvmvapic.c.
> > 
> > > 
> > > Signed-off-by: Igor Mammedov <address@hidden>
> > > ---
> > >  hw/i386/pc.c | 12 +++++++++---
> > >  1 file changed, 9 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > > index 6691825..3206572 100644
> > > --- a/hw/i386/pc.c
> > > +++ b/hw/i386/pc.c
> > > @@ -471,9 +471,6 @@ void pc_cmos_init(PCMachineState *pcms,
> > >      rtc_set_memory(s, 0x5c, val >> 8);
> > >      rtc_set_memory(s, 0x5d, val >> 16);
> > >  
> > > -    /* set the number of CPU */
> > > -    rtc_set_memory(s, 0x5f, smp_cpus - 1);
> > > -
> > >      object_property_add_link(OBJECT(pcms), "rtc_state",
> > >                               TYPE_ISA_DEVICE,
> > >                               (Object **)&pcms->rtc,
> > > @@ -1157,10 +1154,19 @@ void pc_cpus_init(PCMachineState *pcms)
> > >  static
> > >  void pc_machine_done(Notifier *notifier, void *data)
> > >  {
> > > +    int i, boot_cpus = 0;
> > >      PCMachineState *pcms = container_of(notifier,
> > >                                          PCMachineState, machine_done);
> > >      PCIBus *bus = pcms->bus;
> > >  
> > > +    for (i = 0; i < pcms->possible_cpus->len; i++) {
> > > +        if (pcms->possible_cpus->cpus[i].cpu) {
> > > +            boot_cpus++;
> > > +        }
> > > +    }  
> > 
> > Any specific reason you chose to check possible_cpus instead of the
> > arch-independent CPU list from exec.c? I believe other architectures will be
> > interested in a generic way to count online CPUs instead of using smp_cpus.
> When others would need to do it we can switch to arch-independent CPU list but
> for now I'd prefer to keep using possible_cpus throughout pc.c for consistency
> reasons (so it would be the single point of bookkeeping CPUs on PC),
> and I most likely will do the same for ARM probably reusing some of PC code.

Common can be done as a follow-up, no problem. In that case,
could you move the calculation to a reusable function in pc.c?
The APIC check above would also need to check how many CPUs
already exist, for example.

-- 
Eduardo