Re: [PATCH] tests/acceptance: Count multiple Tux logos displayed on framebuffer

[Philippe Mathieu-Daudé]

Hi Aleksandar,

On Tue, Feb 18, 2020 at 7:07 PM Aleksandar Markovic
<address@hidden> wrote:
> On Saturday, February 1, 2020, Philippe Mathieu-Daudé <address@hidden> wrote:
>>
>> Add a test that verifies that each core properly displays the Tux
>> logo on the framebuffer device.
>>
>> We simply follow the OpenCV "Template Matching with Multiple Objects"
>> tutorial, replacing Lionel Messi by Tux:
>> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
>>
>> When OpenCV and NumPy are installed, this test can be run using:
>>
>>   $ avocado --show=app,framebuffer \
>>     run -t cpu:i6400 \
>>     tests/acceptance/machine_mips_malta.py
>>   JOB ID     : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
>>   JOB LOG    : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
>>    (1/3) 
>> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
>>   framebuffer: found Tux at position (x, y) = (0, 0)
>>   PASS (3.37 s)
>>    (2/3) 
>> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores:
>>   framebuffer: found Tux at position (x, y) = (0, 0)
>>   framebuffer: found Tux at position (x, y) = (88, 0)
>>   framebuffer: found Tux at position (x, y) = (176, 0)
>>   framebuffer: found Tux at position (x, y) = (264, 0)
>>   framebuffer: found Tux at position (x, y) = (352, 0)
>>   framebuffer: found Tux at position (x, y) = (440, 0)
>>   framebuffer: found Tux at position (x, y) = (528, 0)
>>   PASS (5.80 s)
>>    (3/3) 
>> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores:
>>   framebuffer: found Tux at position (x, y) = (0, 0)
>>   framebuffer: found Tux at position (x, y) = (88, 0)
>>   framebuffer: found Tux at position (x, y) = (176, 0)
>>   framebuffer: found Tux at position (x, y) = (264, 0)
>>   framebuffer: found Tux at position (x, y) = (352, 0)
>>   framebuffer: found Tux at position (x, y) = (440, 0)
>>   framebuffer: found Tux at position (x, y) = (528, 0)
>>   framebuffer: found Tux at position (x, y) = (616, 0)
>>   PASS (6.67 s)
>>   RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
>> CANCEL 0
>>   JOB TIME   : 16.79 s
>>
>> If the AVOCADO_CV2_SCREENDUMP_PNG_PATH environment variable is set, the
>> test will save the screenshot with matched squares to it.
>>
>> Test inspired by the following post:
>> https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
>> Kernel built with the following Docker file:
>> https://github.com/philmd/qemu-testing-blob/blob/malta_i6400/mips/malta/mips64el/Dockerfile
>>
>> Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
>> ---
>> The resulting match can be visualised at https://pasteboard.co/ISIj6ZA.png
>> Generated using:
>>
>>   $ AVOCADO_CV2_SCREENDUMP_PNG_PATH=/tmp/screendump_matched.png \
>>     avocado --show=app run -t cpu:i6400 \
>>     tests/acceptance/machine_mips_malta.py
>>   JOB ID     : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
>>   JOB LOG    : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
>>    (1/3) 
>> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
>>  PASS (3.37 s)
>>    (2/3) 
>> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores:
>>  PASS (5.80 s)
>>    (3/3) 
>> tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores:
>>  PASS (6.67 s)
>>   RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
>> CANCEL 0
>>   JOB TIME   : 16.79 s
>> ---
>
>
> Reviewed-by: Aleksandar Markovic <address@hidden>
>
> I may add this file to yet another section in MAINTAINTERS (main MIPS target 
> section) (I'll do it while applying, no need to respin), since its importance 
> is significant (tests MIPS' smp fearure).

Hmm you mean the "MIPS TCG CPUs"? Yes why not.

>
> Once again, a gorgeous test!

Thanks :)

>
> Aleksandar
>
>>
>>  MAINTAINERS                            |   1 +
>>  tests/acceptance/machine_mips_malta.py | 116 +++++++++++++++++++++++++
>>  2 files changed, 117 insertions(+)
>>  create mode 100644 tests/acceptance/machine_mips_malta.py
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 4ceb1ad882..65a3dfb564 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -983,6 +983,7 @@ F: hw/mips/mips_malta.c
>>  F: hw/mips/gt64xxx_pci.c
>>  F: include/hw/southbridge/piix.h
>>  F: tests/acceptance/linux_ssh_mips_malta.py
>> +F: tests/acceptance/machine_mips_malta.py
>>
>>  Mipssim
>>  M: Aleksandar Markovic <address@hidden>
>> diff --git a/tests/acceptance/machine_mips_malta.py 
>> b/tests/acceptance/machine_mips_malta.py
>> new file mode 100644
>> index 0000000000..2764143d15
>> --- /dev/null
>> +++ b/tests/acceptance/machine_mips_malta.py
>> @@ -0,0 +1,116 @@
>> +# Functional tests for the MIPS Malta board
>> +#
>> +# Copyright (c) Philippe Mathieu-Daudé <address@hidden>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import gzip
>> +import logging
>> +
>> +from avocado import skipUnless
>> +from avocado_qemu import Test
>> +from avocado_qemu import wait_for_console_pattern
>> +from avocado.utils import archive
>> +
>> +
>> +NUMPY_AVAILABLE = True
>> +try:
>> +    import numpy as np
>> +except ImportError:
>> +    NUMPY_AVAILABLE = False
>> +
>> +CV2_AVAILABLE = True
>> +try:
>> +    import cv2
>> +except ImportError:
>> +    CV2_AVAILABLE = False
>> +
>> +
>> +@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> +@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>> +class MaltaMachineFramebuffer(Test):
>> +
>> +    timeout = 30
>> +
>> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>> +
>> +    def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
>> +        """
>> +        Boot Linux kernel and check Tux logo is displayed on the 
>> framebuffer.
>> +        """
>> +        screendump_path = os.path.join(self.workdir, 'screendump.pbm')
>> +
>> +        kernel_url = ('https://github.com/philmd/qemu-testing-blob/raw/'
>> +                      'a5966ca4b5/mips/malta/mips64el/'
>> +                      'vmlinux-4.7.0-rc1.I6400.gz')
>> +        kernel_hash = '096f50c377ec5072e6a366943324622c312045f6'
>> +        kernel_path_gz = self.fetch_asset(kernel_url, 
>> asset_hash=kernel_hash)
>> +        kernel_path = self.workdir + "vmlinux"
>> +        archive.gzip_uncompress(kernel_path_gz, kernel_path)
>> +
>> +        tuxlogo_url = ('https://github.com/torvalds/linux/raw/v2.6.12/'
>> +                       'drivers/video/logo/logo_linux_vga16.ppm')
>> +        tuxlogo_hash = '3991c2ddbd1ddaecda7601f8aafbcf5b02dc86af'
>> +        tuxlogo_path = self.fetch_asset(tuxlogo_url, 
>> asset_hash=tuxlogo_hash)
>> +
>> +        self.vm.set_console()
>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>> +                               'clocksource=GIC console=tty0 console=ttyS0')
>> +        self.vm.add_args('-kernel', kernel_path,
>> +                         '-cpu', 'I6400',
>> +                         '-smp', '%u' % cpu_cores_count,
>> +                         '-vga', 'std',
>> +                         '-append', kernel_command_line)
>> +        self.vm.launch()
>> +        framebuffer_ready = 'Console: switching to colour frame buffer 
>> device'
>> +        wait_for_console_pattern(self, framebuffer_ready,
>> +                                 failure_message='Kernel panic - not 
>> syncing')
>> +        self.vm.command('human-monitor-command', command_line='stop')
>> +        self.vm.command('human-monitor-command',
>> +                        command_line='screendump %s' % screendump_path)
>> +        logger = logging.getLogger('framebuffer')
>> +
>> +        match_threshold = 0.95
>> +        screendump_bgr = cv2.imread(screendump_path, cv2.IMREAD_COLOR)
>> +        tuxlogo_bgr = cv2.imread(tuxlogo_path, cv2.IMREAD_COLOR)
>> +        result = cv2.matchTemplate(screendump_bgr, tuxlogo_bgr,
>> +                                   cv2.TM_CCOEFF_NORMED)
>> +        loc = np.where(result >= match_threshold)
>> +        tuxlogo_count = 0
>> +        h, w = tuxlogo_bgr.shape[:2]
>> +        debug_png = os.getenv('AVOCADO_CV2_SCREENDUMP_PNG_PATH')
>> +        for tuxlogo_count, pt in enumerate(zip(*loc[::-1]), start=1):
>> +            logger.debug('found Tux at position (x, y) = %s', pt)
>> +            cv2.rectangle(screendump_bgr, pt,
>> +                          (pt[0] + w, pt[1] + h), (0, 0, 255), 2)
>> +        if debug_png:
>> +            cv2.imwrite(debug_png, screendump_bgr)
>> +        self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
>> +
>> +    def test_mips_malta_i6400_framebuffer_logo_1core(self):
>> +        """
>> +        :avocado: tags=arch:mips64el
>> +        :avocado: tags=machine:malta
>> +        :avocado: tags=cpu:i6400
>> +        """
>> +        self.do_test_i6400_framebuffer_logo(1)
>> +
>> +    def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>> +        """
>> +        :avocado: tags=arch:mips64el
>> +        :avocado: tags=machine:malta
>> +        :avocado: tags=cpu:i6400
>> +        """
>> +        self.do_test_i6400_framebuffer_logo(7)
>> +
>> +    def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>> +        """
>> +        :avocado: tags=arch:mips64el
>> +        :avocado: tags=machine:malta
>> +        :avocado: tags=cpu:i6400
>> +        """
>> +        self.do_test_i6400_framebuffer_logo(8)
>> --
>> 2.21.1
>>
>>