qemu-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] ef9d91: qapi: convert to use python print fun


From: GitHub
Subject: [Qemu-commits] [qemu/qemu] ef9d91: qapi: convert to use python print function instead...
Date: Wed, 07 Feb 2018 04:05:50 -0800

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: ef9d9108917d6d5f903bca31602827e512a51c50
      
https://github.com/qemu/qemu/commit/ef9d9108917d6d5f903bca31602827e512a51c50
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qapi.py
    M scripts/qapi2texi.py
    M tests/qapi-schema/test-qapi.py

  Log Message:
  -----------
  qapi: convert to use python print function instead of statement

Python 3 no longer supports the bare "print" statement, it must be
called as a normal function with round brackets. It is possible to
opt-in to this new syntax with Python 2.6 onwards by importing the
"print_function" from the "__future__" module, making it easy to
support Python 2 and 3 in parallel.

Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 2f8480447067d6f42af52a886385284ead052af9
      
https://github.com/qemu/qemu/commit/2f8480447067d6f42af52a886385284ead052af9
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qapi.py
    M scripts/qapi2texi.py
    M tests/qapi-schema/test-qapi.py

  Log Message:
  -----------
  qapi: use items()/values() intead of iteritems()/itervalues()

The iteritems()/itervalues() methods are gone in py3, but the
items()/values() methods are still around. The latter are less
efficient than the former in py2, but this has unmeasurably
small impact on QEMU build time, so taking portability over
efficiency is a net win.

Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 38710a8994911d98acbe183a39ec3a53638de510
      
https://github.com/qemu/qemu/commit/38710a8994911d98acbe183a39ec3a53638de510
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Use OrderedDict from standard library if available

The OrderedDict class appeared in the 'collections' module
from python 2.7 onwards, so use that in preference to our
local backport if available.

Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 5f90af8e6b34f9e6b60eb05a15707a95a0febbde
      
https://github.com/qemu/qemu/commit/5f90af8e6b34f9e6b60eb05a15707a95a0febbde
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: adapt to moved location of StringIO module in py3

Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 52c4272c6c916a53cde65b997e1a4e891c14dcef
      
https://github.com/qemu/qemu/commit/52c4272c6c916a53cde65b997e1a4e891c14dcef
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: Adapt to moved location of 'maketrans' function in py3

Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 46ec4fcea95204a8e5bab9295cbfaa3606d78dc9
      
https://github.com/qemu/qemu/commit/46ec4fcea95204a8e5bab9295cbfaa3606d78dc9
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M tests/Makefile.include

  Log Message:
  -----------
  qapi: remove '-q' arg to diff when comparing QAPI output

When the qapi schema tests fail they merely print that the expected
output didn't match the actual output. This is largely useless when
trying diagnose what went wrong. Removing the '-q' arg to diff
means that it is still silent on successful tests, but when it
fails we'll see details of the incorrect output.

Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: f7a5376d4b667cf6c83c1d640e32d22456d7b5ee
      
https://github.com/qemu/qemu/commit/f7a5376d4b667cf6c83c1d640e32d22456d7b5ee
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qapi.py

  Log Message:
  -----------
  qapi: ensure stable sort ordering when checking QAPI entities

Some early python 3.x versions will have different default
ordering when calling the 'values()' method on a dict, compared
to python 2.x and later 3.x versions. Explicitly sort the items
to get a stable ordering.

Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: d4e5ec877ca698a87dabe68814c6f93668f50c60
      
https://github.com/qemu/qemu/commit/d4e5ec877ca698a87dabe68814c6f93668f50c60
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M Makefile

  Log Message:
  -----------
  qapi: force a UTF-8 locale for running Python

Python2 did not validate locale correctness when reading input data, so
would happily read UTF-8 data in non-UTF-8 locales. Python3 is strict so
if you try to read UTF-8 data in the C locale, it will raise an error
for any UTF-8 bytes that aren't representable in 7-bit ascii encoding.
e.g.

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 54: 
ordinal not in range(128)
Traceback (most recent call last):
  File "/tmp/qemu-test/src/scripts/qapi-commands.py", line 317, in <module>
    schema = QAPISchema(input_file)
  File "/tmp/qemu-test/src/scripts/qapi.py", line 1468, in __init__
    parser = QAPISchemaParser(open(fname, 'r'))
  File "/tmp/qemu-test/src/scripts/qapi.py", line 301, in __init__
    previously_included)
  File "/tmp/qemu-test/src/scripts/qapi.py", line 348, in _include
    exprs_include = QAPISchemaParser(fobj, previously_included, info)
  File "/tmp/qemu-test/src/scripts/qapi.py", line 271, in __init__
    self.src = fp.read()
  File "/usr/lib64/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]

More background on this can be seen in

  https://www.python.org/dev/peps/pep-0538/

Many distros support a new C.UTF-8 locale that is like the C locale,
but with UTF-8 instead of 7-bit ASCII. That is not entirely portable
though. This patch thus sets the LANG to "C", but overrides LC_CTYPE
to be en_US.UTF-8 locale. This gets us pretty close to C.UTF-8, but
in a way that should be portable to everywhere QEMU builds.

This patch only forces UTF-8 for QAPI scripts, since that is the one
showing the immediate error under Python3 with C locale, but potentially
we ought to force this for all python scripts used in the build process.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 31d8f92e646f7d4cfbb4ffab440ab41a3c838fd3
      
https://github.com/qemu/qemu/commit/31d8f92e646f7d4cfbb4ffab440ab41a3c838fd3
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/signrom.py

  Log Message:
  -----------
  scripts: ensure signrom treats data as bytes

Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: c21965a0c8b979c306e927f158257e5b0fa3a1f9
      
https://github.com/qemu/qemu/commit/c21965a0c8b979c306e927f158257e5b0fa3a1f9
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: allow use of python 3

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 74af2e59d26712aa673832ec03ec6eac53066c03
      
https://github.com/qemu/qemu/commit/74af2e59d26712aa673832ec03ec6eac53066c03
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: remove unused import

Removing 'import sys' as it's not used anywhere.

Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: af99fa9fe22ba1c6139d1fbe6a66f3b01d575cab
      
https://github.com/qemu/qemu/commit/af99fa9fe22ba1c6139d1fbe6a66f3b01d575cab
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: better control of created files

To launch a VM, we need to create basically two files: the monitor
socket (if it's a UNIX socket) and the qemu log file.

For the qemu log file, we currently just open the path, which will
create the file if it does not exist or overwrite the file if it does
exist.

For the monitor socket, if it already exists, we are currently removing
it, even if it's not created by us.

This patch moves to _pre_launch() the responsibility to create a
temporary directory to host the files so we can remove the whole
directory on _post_shutdown().

Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: d301bccf7323ff3069e46f600aa7b31dbbc2f2f7
      
https://github.com/qemu/qemu/commit/d301bccf7323ff3069e46f600aa7b31dbbc2f2f7
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: refactor launch()

This is just a refactor to separate the exception handler from the
actual launch procedure, improving the readability and making future
maintenances in this piece of code easier.

Reviewed-by: Fam Zheng <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 04a963b4953be6c7f1899cfe0a0a11d03292c18b
      
https://github.com/qemu/qemu/commit/04a963b4953be6c7f1899cfe0a0a11d03292c18b
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: always cleanup on shutdown()

Currently we only cleanup on shutdown() if the VM is running.

To make sure we will always cleanup, this patch makes the
self._load_io_log() and the self._post_shutdown() to
always be called on shutdown(), regardless the VM running state.

Reviewed-by: Fam Zheng <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 17589cae908222d572953d4c85f72aa833e87d58
      
https://github.com/qemu/qemu/commit/17589cae908222d572953d4c85f72aa833e87d58
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: use poll() instead of 'returncode'

The 'returncode' Popen attribute is not guaranteed to be updated. It
actually depends on a call to either poll(), wait() or communicate().

On the other hand, poll() will: "Check if child process has terminated.
Set and return returncode attribute."

Let's use the poll() to check whether the process is running and to get
the updated process exit code, when the process is finished.

Reviewed-by: Fam Zheng <address@hidden>
eviewed-by: Eduardo Habkost <address@hidden>
Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: c58b535f8396f7ab71dc7074713c99ec042da95b
      
https://github.com/qemu/qemu/commit/c58b535f8396f7ab71dc7074713c99ec042da95b
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: cleanup redundant calls in launch()

Now that shutdown() is guaranteed to always execute self._load_io_log()
and self._post_shutdown(), their calls in 'except' became redundant and
we can safely replace it by a call to shutdown().

Reviewed-by: Fam Zheng <address@hidden>
Reviewed-by: Eduardo Habkost <address@hidden>
Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 156dc7b1740d9f71baf4aef277d7f812f1a784ba
      
https://github.com/qemu/qemu/commit/156dc7b1740d9f71baf4aef277d7f812f1a784ba
  Author: Amador Pahim <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M scripts/qemu.py

  Log Message:
  -----------
  qemu.py: don't launch again before shutdown()

If a VM is launched, files are created and a cleanup is required before
a new launch. This cleanup is executed by shutdown(), so shutdown() must
be called even if the VM is manually terminated (i.e. using kill).

This patch creates a control to make sure launch() will not be executed
again if shutdown() is not called after the previous launch().

Signed-off-by: Amador Pahim <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 280b8da3b842177de538de4f73d3d63db077e39b
      
https://github.com/qemu/qemu/commit/280b8da3b842177de538de4f73d3d63db077e39b
  Author: Miika S <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M qapi/ui.json
    M ui/keycodemapdb

  Log Message:
  -----------
  input: add missing JIS keys to virtio input

keycodemapdb updated to add the QKeyCodes muhenkan and katakanahiragana

Signed-off-by: Miika S <address@hidden>
Message-Id: <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: df25920903036d0381ecf7f946c9ea100d002f80
      
https://github.com/qemu/qemu/commit/df25920903036d0381ecf7f946c9ea100d002f80
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M ui/keycodemapdb

  Log Message:
  -----------
  ui: update keycodemapdb to get py3 fixes

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 8e73a3c5fc0b57c3b7b4f3f79135570cb41d84a4
      
https://github.com/qemu/qemu/commit/8e73a3c5fc0b57c3b7b4f3f79135570cb41d84a4
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M .travis.yml

  Log Message:
  -----------
  travis: improve python version test coverage

Currently travis declares ancient python 2.4 is desired. Update that to
2.6 which is the oldest version any targetted distros still needs. If we
just list a python 3 version at the top level this will double the
number of travis jobs we run which is unreasonable.

So arbitrarily pick the clang test matrix entries to build with python
3.0 and 3.6, to extend coverage of python versions, without increasing
job count or build time.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 356dc290f0ef930a24b6af3a0908b1cb58ad47c9
      
https://github.com/qemu/qemu/commit/356dc290f0ef930a24b6af3a0908b1cb58ad47c9
  Author: Daniel P. Berrange <address@hidden>
  Date:   2018-02-05 (Mon, 05 Feb 2018)

  Changed paths:
    M tests/docker/dockerfiles/fedora.docker

  Log Message:
  -----------
  docker: change Fedora images to run with python3

Fedora has switched to Python 3 by default, so it makes sense to use that
for testing QEMU builds, so we get testing of Python 3 compatibility.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: bc2943d6caf787e1c9a5f3109cdb98f37630b89e
      
https://github.com/qemu/qemu/commit/bc2943d6caf787e1c9a5f3109cdb98f37630b89e
  Author: Peter Maydell <address@hidden>
  Date:   2018-02-06 (Tue, 06 Feb 2018)

  Changed paths:
    M .travis.yml
    M Makefile
    M configure
    M qapi/ui.json
    M scripts/qapi.py
    M scripts/qapi2texi.py
    M scripts/qemu.py
    M scripts/signrom.py
    M tests/Makefile.include
    M tests/docker/dockerfiles/fedora.docker
    M tests/qapi-schema/test-qapi.py
    M ui/keycodemapdb

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' 
into staging

Python queue, 2018-02-05

# gpg: Signature made Mon 05 Feb 2018 23:07:57 GMT
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <address@hidden>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request: (21 commits)
  docker: change Fedora images to run with python3
  travis: improve python version test coverage
  ui: update keycodemapdb to get py3 fixes
  input: add missing JIS keys to virtio input
  qemu.py: don't launch again before shutdown()
  qemu.py: cleanup redundant calls in launch()
  qemu.py: use poll() instead of 'returncode'
  qemu.py: always cleanup on shutdown()
  qemu.py: refactor launch()
  qemu.py: better control of created files
  qemu.py: remove unused import
  configure: allow use of python 3
  scripts: ensure signrom treats data as bytes
  qapi: force a UTF-8 locale for running Python
  qapi: ensure stable sort ordering when checking QAPI entities
  qapi: remove '-q' arg to diff when comparing QAPI output
  qapi: Adapt to moved location of 'maketrans' function in py3
  qapi: adapt to moved location of StringIO module in py3
  qapi: Use OrderedDict from standard library if available
  qapi: use items()/values() intead of iteritems()/itervalues()
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/20e0d439a6de...bc2943d6caf7

reply via email to

[Prev in Thread] Current Thread [Next in Thread]