On Thu, Oct 22, 2015 at 2:21 AM, Christian Pinto
<address@hidden> wrote:
Hello Peter,
On 07/10/2015 17:48, Peter Crosthwaite wrote:
On Mon, Oct 5, 2015 at 8:50 AM, Christian Pinto
<address@hidden> wrote:
Hello Peter,
thanks for your comments
On 01/10/2015 18:26, Peter Crosthwaite wrote:
On Tue, Sep 29, 2015 at 6:57 AM, Christian Pinto
<address@hidden> wrote:
Hi all,
This RFC patch-series introduces the set of changes enabling the
architectural elements to model the architecture presented in a
previous
RFC
letter: "[Qemu-devel][RFC] Towards an Heterogeneous QEMU".
and the OS binary image needs
to be placed in memory at model startup.
I don't see what this limitation is exactly. Can you explain more? I
do see a need to work on the ARM bootloader for AMP flows, it is a
pure SMP bootloader than assumes total control.
the problem here was to me that when we launch QEMU a binary needs to be
provided and put in memory
in order to be executed. In this patch series the slave doesn't have a
proper memory allocated when first launched.
But it could though couldn't it? Can't the slave guest just have full
access to it's own address space (probably very similar to the masters
address space) from machine init time? This seems more realistic than
setting up the hardware based on guest level information.
Actually the address space for a slave is built at init time, the thing that
is not
completely configured is the memory region modeling the RAM. Such region is
configured
in terms of size, but there is no pointer to the actual memory. The pointer
is mmap-ed later
before the slave boots.
based on what information? Is the master guest controlling this? If so
what is the real-hardware analogue for this concept where the address
map of the slave can change (i.e. be configured) at runtime?
The information about memory (fd + offset for mmap) is sent only later
when
the boot is triggered. This is also
safe since the slave will be waiting in the incoming state, and thus no
corruption or errors can happen before the
boot is triggered.
I was thinking more about your comment about slave-to-slave
interrupts. This would just trivially be a local software-generated
interrupts of some form within the slave cluster.
Sorry, I did not catch your comment at first time. You are right, if cores
are in the same cluster
a software generated interrupt is going to be enough. Of course the eventfd
based interrupts
make sense for a remote QEMU.
Is eventfd a better implementation of remote-port GPIOs as in the Xilinx work?
Re the terminology, I don't like the idea of thinking of inter-qemu
"interrupts" as whatever system we decide on should be able to support
arbitrary signals going from one QEMU to another. I think the Xilinx
work already has reset signals going between the QEMU peers.
The multi client-socket is used for the master to trigger
the boot of a slave, and also for each master-slave couple to
exchancge the
eventd file descriptors. The IDM device can be instantiated
either
as a
PCI or sysbus device.
So if everything is is one QEMU, IPIs can be implemented with just a
of registers makes the master in
"control" each of the slaves. The IDM device is already seen as a regular
device by each of the QEMU instances
involved.
I'm starting to think this series is two things that should be
decoupled. One is the abstract device(s) to facilitate your AMP, the
other is the inter-qemu communication. For the abstract device, I
guess this would be a new virtio-idm device. We should try and involve
virtio people perhaps. I can see the value in it quite separate from
modelling the real sysctrl hardware.
Interesting, which other value/usage do you see in it? For me the IDM was
meant to
It has value in prototyping with your abstract toolkit even with
homogeneous hardware. E.g. I should be able to just use single-QEMU
ARM virt machine -smp 2 and create one of these virtio-AMP setups.
Homogeneous hardware with heterogenous software using your new pieces
of abstract hardware.
It is also more practical for getting a merge of your work as you are
targetting two different audiences with the work. People intersted in
virtio can handle the new devices you create, while the core
maintainers can handle your multi-QEMU work. It is two rather big new
features.
work as an abstract system controller to centralize the management
of the slaves (boot_regs and interrupts).
But I think the implementation
should be free of any inter-QEMU awareness. E.g. from P4 of this
series:
+static void send_shmem_fd(IDMState *s, MSClient *c)
+{
+ int fd, len;
+ uint32_t *message;
+ HostMemoryBackend *backend = MEMORY_BACKEND(s->hostmem);
+
+ len = strlen(SEND_MEM_FD_CMD)/4 + 3;
+ message = malloc(len * sizeof(uint32_t));
+ strcpy((char *) message, SEND_MEM_FD_CMD);
+ message[len - 2] = s->pboot_size;
+ message[len - 1] = s->pboot_offset;
+
+ fd = memory_region_get_fd(&backend->mr);
+
+ multi_socket_send_fds_to(c, &fd, 1, (char *) message, len *
sizeof(uint32_t));
The device itself is aware of shared-memory and multi-sockets. Using
the device for single-QEMU AMP would require neither - can the IDM
device be used in a homogeneous AMP flow in one of our existing SMP
machine models (eg on a dual core A9 with one core being master and
the other slave)?
Can this be architected in two phases for greater utility, with the
AMP devices as just normal devices, and the inter-qemu communication
as a separate feature?
I see your point, and it is an interesting proposal.
What I can think here to remove the awareness of how the IDM communicates
with
the slaves, is to define a kind of AMP Slave interface. So there will be an
instance of the interface for each of the slaves, encapsulating the
communication part (being either local or based on sockets).
The AMP Slave interfaces would be what you called the AMP devices, with one
device per slave.
Do we need this hard definition of master and slave in the hardware?
Can the virtio-device be more peer-peer and the master-slave
relationship is purely implemented by the guest?
Regards,
Peter
At master side, besides the IDM, one would instantiate
as many interface devices as slaves. During the initialization the IDM will
link
with all those interfaces, and only call functions like: send_interrupt() or
boot_slave() to interact with the slaves. The interface will be the same for
both local or remote slaves, while the implementation of the methods will
differ and reside in the specific AMP Slave Interface device.
On the slave side, if the slave is remote, another instance of the
interface is instantiated so to connect to socket/eventfd.
So as an example the send_shmem_fd function you pointed could be hidden in
the
slave interface, and invoked only when the IDM will invoke the slave_boot()
function of a remote slave interface.
This would higher the level of abstraction and open the door to potentially
any
communication mechanism between master and slave, without the need to adapt
the
IDM device to the specific case. Or, eventually, to mix between local and
remote instances.
Thanks,
Christian
Regards,
Peter