Re: [PATCH] tests/acceptance: Count multiple Tux logos displayed on fram

From:

Aleksandar Markovic

Subject:

Re: [PATCH] tests/acceptance: Count multiple Tux logos displayed on framebuffer

Date:

Tue, 18 Feb 2020 20:10:17 +0100

address@hidden

Hi Aleksandar,

On Tue, Feb 18, 2020 at 7:07 PM Aleksandar Markovic
<address@hidden> wrote:
> On Saturday, February 1, 2020, Philippe Mathieu-Daudé <address@hidden> wrote:
>>
>> Add a test that verifies that each core properly displays the Tux
>> logo on the framebuffer device.
>>
>> We simply follow the OpenCV "Template Matching with Multiple Objects"
>> tutorial, replacing Lionel Messi by Tux:
>> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
>>
>> When OpenCV and NumPy are installed, this test can be run using:
>>
>> $ avocado --show=app,framebuffer \
>> run -t cpu:i6400 \
>> tests/acceptance/machine_mips_malta.py
>> JOB ID : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
>> JOB LOG : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
>> (1/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
>> framebuffer: found Tux at position (x, y) = (0, 0)
>> PASS (3.37 s)
>> (2/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores:
>> framebuffer: found Tux at position (x, y) = (0, 0)
>> framebuffer: found Tux at position (x, y) = (88, 0)
>> framebuffer: found Tux at position (x, y) = (176, 0)
>> framebuffer: found Tux at position (x, y) = (264, 0)
>> framebuffer: found Tux at position (x, y) = (352, 0)
>> framebuffer: found Tux at position (x, y) = (440, 0)
>> framebuffer: found Tux at position (x, y) = (528, 0)
>> PASS (5.80 s)
>> (3/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores:
>> framebuffer: found Tux at position (x, y) = (0, 0)
>> framebuffer: found Tux at position (x, y) = (88, 0)
>> framebuffer: found Tux at position (x, y) = (176, 0)
>> framebuffer: found Tux at position (x, y) = (264, 0)
>> framebuffer: found Tux at position (x, y) = (352, 0)
>> framebuffer: found Tux at position (x, y) = (440, 0)
>> framebuffer: found Tux at position (x, y) = (528, 0)
>> framebuffer: found Tux at position (x, y) = (616, 0)
>> PASS (6.67 s)
>> RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME : 16.79 s
>>
>> If the AVOCADO_CV2_SCREENDUMP_PNG_PATH environment variable is set, the
>> test will save the screenshot with matched squares to it.
>>
>> Test inspired by the following post:
>> https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
>> Kernel built with the following Docker file:
>> https://github.com/philmd/qemu-testing-blob/blob/malta_i6400/mips/malta/mips64el/Dockerfile
>>
>> Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
>> ---
>> The resulting match can be visualised at https://pasteboard.co/ISIj6ZA.png
>> Generated using:
>>
>> $ AVOCADO_CV2_SCREENDUMP_PNG_PATH=/tmp/screendump_matched.png \
>> avocado --show=app run -t cpu:i6400 \
>> tests/acceptance/machine_mips_malta.py
>> JOB ID : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
>> JOB LOG : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
>> (1/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core: PASS (3.37 s)
>> (2/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores: PASS (5.80 s)
>> (3/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores: PASS (6.67 s)
>> RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME : 16.79 s
>> ---
>
>
> Reviewed-by: Aleksandar Markovic <address@hidden>
>
> I may add this file to yet another section in MAINTAINTERS (main MIPS target section) (I'll do it while applying, no need to respin), since its importance is significant (tests MIPS' smp fearure).

Hmm you mean the "MIPS TCG CPUs"? Yes why not.

Yes, I do. This test's importance far exceeds Malta support, it is a crucial test for mips smp support. I have in my backlog improving smp support, that unfortunatelly currently has certain limitations, but Leon's example still works, and this test will make sure it will continue to do so, which is of tremendous importance in my view.

Thanks again,

Aleksandar

>
> Once again, a gorgeous test!

Thanks :)

>
> Aleksandar
>
>>
>> MAINTAINERS | 1 +
>> tests/acceptance/machine_mips_malta.py | 116 +++++++++++++++++++++++++
>> 2 files changed, 117 insertions(+)
>> create mode 100644 tests/acceptance/machine_mips_malta.py
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 4ceb1ad882..65a3dfb564 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -983,6 +983,7 @@ F: hw/mips/mips_malta.c
>> F: hw/mips/gt64xxx_pci.c
>> F: include/hw/southbridge/piix.h
>> F: tests/acceptance/linux_ssh_mips_malta.py
>> +F: tests/acceptance/machine_mips_malta.py
>>
>> Mipssim
>> M: Aleksandar Markovic <address@hidden>
>> diff --git a/tests/acceptance/machine_mips_malta.py b/tests/acceptance/machine_mips_malta.py
>> new file mode 100644
>> index 0000000000..2764143d15
>> --- /dev/null
>> +++ b/tests/acceptance/machine_mips_malta.py
>> @@ -0,0 +1,116 @@
>> +# Functional tests for the MIPS Malta board
>> +#
>> +# Copyright (c) Philippe Mathieu-Daudé <address@hidden>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import gzip
>> +import logging
>> +
>> +from avocado import skipUnless
>> +from avocado_qemu import Test
>> +from avocado_qemu import wait_for_console_pattern
>> +from avocado.utils import archive
>> +
>> +
>> +NUMPY_AVAILABLE = True
>> +try:
>> + import numpy as np
>> +except ImportError:
>> + NUMPY_AVAILABLE = False
>> +
>> +CV2_AVAILABLE = True
>> +try:
>> + import cv2
>> +except ImportError:
>> + CV2_AVAILABLE = False
>> +
>> +
>> +@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> +@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>> +class MaltaMachineFramebuffer(Test):
>> +
>> + timeout = 30
>> +
>> + KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>> +
>> + def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
>> + """
>> + Boot Linux kernel and check Tux logo is displayed on the framebuffer.
>> + """
>> + screendump_path = os.path.join(self.workdir, 'screendump.pbm')
>> +
>> + kernel_url = ('https://github.com/philmd/qemu-testing-blob/raw/'
>> + 'a5966ca4b5/mips/malta/mips64el/'
>> + 'vmlinux-4.7.0-rc1.I6400.gz')
>> + kernel_hash = '096f50c377ec5072e6a366943324622c312045f6'
>> + kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>> + kernel_path = self.workdir + "vmlinux"
>> + archive.gzip_uncompress(kernel_path_gz, kernel_path)
>> +
>> + tuxlogo_url = ('https://github.com/torvalds/linux/raw/v2.6.12/'
>> + 'drivers/video/logo/logo_linux_vga16.ppm')
>> + tuxlogo_hash = '3991c2ddbd1ddaecda7601f8aafbcf5b02dc86af'
>> + tuxlogo_path = self.fetch_asset(tuxlogo_url, asset_hash=tuxlogo_hash)
>> +
>> + self.vm.set_console()
>> + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>> + 'clocksource=GIC console=tty0 console=ttyS0')
>> + self.vm.add_args('-kernel', kernel_path,
>> + '-cpu', 'I6400',
>> + '-smp', '%u' % cpu_cores_count,
>> + '-vga', 'std',
>> + '-append', kernel_command_line)
>> + self.vm.launch()
>> + framebuffer_ready = 'Console: switching to colour frame buffer device'
>> + wait_for_console_pattern(self, framebuffer_ready,
>> + failure_message='Kernel panic - not syncing')
>> + self.vm.command('human-monitor-command', command_line='stop')
>> + self.vm.command('human-monitor-command',
>> + command_line='screendump %s' % screendump_path)
>> + logger = logging.getLogger('framebuffer')
>> +
>> + match_threshold = 0.95
>> + screendump_bgr = cv2.imread(screendump_path, cv2.IMREAD_COLOR)
>> + tuxlogo_bgr = cv2.imread(tuxlogo_path, cv2.IMREAD_COLOR)
>> + result = cv2.matchTemplate(screendump_bgr, tuxlogo_bgr,
>> + cv2.TM_CCOEFF_NORMED)
>> + loc = np.where(result >= match_threshold)
>> + tuxlogo_count = 0
>> + h, w = tuxlogo_bgr.shape[:2]
>> + debug_png = os.getenv('AVOCADO_CV2_SCREENDUMP_PNG_PATH')
>> + for tuxlogo_count, pt in enumerate(zip(*loc[::-1]), start=1):
>> + logger.debug('found Tux at position (x, y) = %s', pt)
>> + cv2.rectangle(screendump_bgr, pt,
>> + (pt[0] + w, pt[1] + h), (0, 0, 255), 2)
>> + if debug_png:
>> + cv2.imwrite(debug_png, screendump_bgr)
>> + self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
>> +
>> + def test_mips_malta_i6400_framebuffer_logo_1core(self):
>> + """
>> + :avocado: tags=arch:mips64el
>> + :avocado: tags=machine:malta
>> + :avocado: tags=cpu:i6400
>> + """
>> + self.do_test_i6400_framebuffer_logo(1)
>> +
>> + def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>> + """
>> + :avocado: tags=arch:mips64el
>> + :avocado: tags=machine:malta
>> + :avocado: tags=cpu:i6400
>> + """
>> + self.do_test_i6400_framebuffer_logo(7)
>> +
>> + def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>> + """
>> + :avocado: tags=arch:mips64el
>> + :avocado: tags=machine:malta
>> + :avocado: tags=cpu:i6400
>> + """
>> + self.do_test_i6400_framebuffer_logo(8)
>> --
>> 2.21.1
>>
>>