Re: CentOS 8 cloud images not working under VM tests

[John Snow]

On Mon, Jun 13, 2022 at 12:36 PM John Snow <jsnow@redhat.com> wrote:
>
> On Mon, Jun 13, 2022 at 2:05 AM Thomas Huth <thuth@redhat.com> wrote:
> >
> > On 11/06/2022 01.28, John Snow wrote:
> > > This test doesn't appear to work for me:
> > >
> > > def build_image(self, img):
> > >      cimg = 
> > > self._download_with_cache("https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.3.2011-20201204.2.x86_64.qcow2";)
> > >      img_tmp = img + ".tmp"
> > >      subprocess.check_call(["ln", "-f", cimg, img_tmp])
> > >      self.exec_qemu_img("resize", img_tmp, "50G")
> > >      self.boot(img_tmp, extra_args = ["-cdrom", 
> > > self.gen_cloud_init_iso()])
> > >      self.wait_ssh()
> > >      ^^^^^^^^^^^^^^^
> > >
> > > It appears to be expecting to be able to use passwordless entry, but
> > > that doesn't appear to actually work in this case.
> > >
> > > It looks like the cloud iso generate step is supposed to handle
> > > setting up keys -- and everything appears as if it's working -- but I
> > > get SSH timeouts at this step.
> > >
> > >  From what I can see:
> > >
> > > DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o
> > > UserKnownHostsFile=/dev/null -o ConnectTimeout=1 -p 41729 -i
> > > /home/jsnow/src/qemu/bin/git/vm-test-35u779h4.tmp/id_rsa -o
> > > SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o
> > > SendEnv=no_proxy qemu@127.0.0.1 exit 0
> > > Warning: Permanently added '[127.0.0.1]:41729' (ED25519) to the list
> > > of known hosts.
> > > qemu@127.0.0.1: Permission denied 
> > > (publickey,gssapi-keyex,gssapi-with-mic).
> > >
> > > ...and the /home/jsnow/src/qemu/bin/git/vm-test-35u779h4.tmp/id_rsa
> > > file looks identical to the qemu.git/tests/keys/id_rsa file, save for
> > > a missing trailing newline.
> > >
> > >  From a subsequent run, turning SSH debug on, I see this:
> > >
> > > debug1: Offering public key:
> > > /home/jsnow/src/qemu/bin/git/vm-test-o_x2vdwo.tmp/id_rsa RSA
> > > SHA256:6TUK9PSgWR+CbTEKA6E9IyizVjt2ZW5ble/Mg4wUiao explicit
> > > debug3: send packet: type 50
> > > debug2: we sent a publickey packet, wait for reply
> > > debug3: receive packet: type 51
> > >
> > > ... Which looks like the usual kind of bog-standard "Unrecognized key"
> > > kind of answer, IIUC.
> > >
> > > Is this working for anyone else, or can anyone offer some debugging
> > > tips on what's gone wrong here?
> > It seems to work for me - maybe it's some issue with a newer version of ssh
> > on your host? (I'm still using RHEL 8 here).
> >
> > Anyway, the VM dies shortly afterwards since it tries to install some
> > additional packages, and non-Stream CentOS 8 has been disabled at the end of
> > last year. So this test is certainly broken since half a year already and
> > nobody noticed until now. I think you can either remove it, or it should get
> > updated to CentosStream instead.
> >
> >   Thomas
> >
>
> Don't really have the interest to upgrade it myself, so if it's been
> broken for half a year, out it goes.
>

It turns out it's because the setup was failing, but we use "ln" to
make a temporary image instead of cp -- so if the setup screws up,
we've permanently damaged the cached image as well. You have to delete
the original downloaded image to restore original behavior. Then, as
you say, the old image doesn't work anymore anyway. Replacing it with
a new CentOS 8 stream image works fine, if we replace the LN with CP
like the fedora recipe does. However, some of the iotests fail due to
permissions issues with docker and FUSE.

Still trying to figure out the FUSE stuff.

I tried to update aarch64, but that test seems to need even more work.
I think I will probably just delete it, it doesn't seem like anyone is
running it or looking after it.

--js